- Author detailed, task-based multi-turn conversations and rubrics aligned with project specifications.
- Test and refine conversation drafts against frontier large language models, iterating to meet quality and difficulty requirements.
- Deliver comprehensive evaluation assets including transcripts, target behaviors, binary rubrics, and supporting evidence.
- Ensure strict fidelity to evolving project specs while maintaining high throughput and attention to detail.
- Validate and calibrate outputs with team leads and quality control as guidelines change.
- Work independently and consistently, meeting expected output rates for deliverable completion.
Requirements
- Native-level written English with exceptional clarity, structure, and attention to detail.
- Prior experience in data annotation, RLHF, SFT, evaluation, or prompt engineering for AI systems.
- Working knowledge of frontier LLM behaviors and common model failure patterns.
- Demonstrated ability to interpret and apply highly detailed specifications without supervision.
- Strong critical thinking and analytical skills in writing-heavy or analysis-heavy domains.
Application Process
- Easy Apply on LinkedIn
- Participate in resume evaluation & interview stage
LLM Conversation Quality Assurance Specialist employer: Crossing Hurdles
Crossing Hurdles is an exceptional employer that fosters a collaborative and innovative work culture, making it an ideal place for a Kubernetes Cloud Engineer & CI/CD Specialist. With a strong emphasis on employee growth, we offer continuous learning opportunities and the chance to work with cutting-edge technologies in a dynamic environment across the UK. Our commitment to work-life balance and a supportive team atmosphere ensures that you can thrive both personally and professionally.
