Operations · London · Full-time · Senior level
This role owns the quality standard for the human labelled data the company delivers, and runs both halves of it: automated evaluation built with models and statistics, and a distributed audit team of about ten people. A strong candidate is hands on in Python and SQL, comfortable with sampling and agreement statistics, and able to write rubrics clear enough for a remote team to follow without supervision. The job also asks for someone who can hold a standard under commercial pressure and show the evidence behind it.
How it fits the AI era
Much of the screening work here is meant to be done by models, with LLM based judging and anomaly detection replacing linear growth in headcount. What stays human is the judgement on hard cases, the calibration of the standard, and the decision about how much manual coverage to keep. The role sits at the point where automated measurement and human ground truth feed each other.
How it builds AI experience
The work offers direct practice in LLM as judge screening, model assisted QA and drift detection on real production annotation data, plus building gold sets that train and validate those models. Because the company owns the platform, the measurement built here can become product capability rather than internal tooling, which is a rare chance to shape evaluation features for physical AI customers.
Worth showing
- Python and SQL
- LLM as judge evaluation
- Sampling and agreement statistics
- Writing quality rubrics
This is our own read, written from the employer's posting and the company's Spotlight. The employer's posting is the source of truth for the role.
About Encord
A data platform that helps AI teams manage, label and clean the video, sensor and image data that trains robots, self-driving cars and other physical AI systems.
#J-18808-Ljbffr
Quality Systems Lead employer: present of AI
H Company is an exceptional employer for Product Designers, offering a dynamic work environment in London where innovation meets collaboration. With a strong focus on employee growth, the company encourages continuous learning and experimentation, particularly in the evolving field of AI design. The inclusive culture fosters open communication with users and cross-functional teams, ensuring that every designer's voice is heard and valued, making it a truly rewarding place to advance your career.
