Architect preference data collection protocols, reward model validation benchmarks, and adversarial red-teaming pipelines for frontier generative AI labs.
Responsibilities
- Design evaluation rubrics, pairwise ranking schemas, and prompt suites for complex reasoning models
- Engineer automated pipelines detecting hallucinations, reward hacking, and sycophancy in LLM outputs
- Direct specialist expert pods (programmers, physicians, legal scholars) generating high-nuance synthetic training data
- Co-author research findings on data curation efficiency and alignment methodologies
Requirements
- Demonstrated experience with LLM evaluation benchmarks (MMLU, GSM8k, Arena pairwise ranking)
- Deep grasp of RLHF, DPO, PPO, and SFT alignment frameworks and prompt engineering
- Track record of authoring precise technical guidelines and evaluation rubrics
- Publications or notable contributions to open-source alignment projects is a strong plus
- Direct collaboration with top frontier AI researchers and foundation model labs
- Competitive Silicon Valley-calibrated compensation and equity
- Unlimited paid time off (PTO) with a mandatory minimum 4 weeks policy
- Parental leave (up to 16 weeks fully paid)
YOUR NEXT OPPORTUNITY
Machine Learning & AI · AI Engineering in London employer: Engai
Engai is an attractive employer due to its focus on cutting-edge AI research and development. The company provides competitive compensation, unlimited PTO, and a supportive work culture that encourages collaboration with industry experts.
