AI Assurance Lead in London

We’re working with a specialist UK technology business delivering advanced AI solutions into Defence, Government and other highly regulated environments. They’re looking for an experienced Machine Learning Engineer or Data Scientist to lead technical AI safety, model evaluation and assurance across a growing portfolio of AI products. This is not a governance-only role. Around 75% of the position is hands-on technical work, testing whether models, pipelines and AI agents perform safely and reliably in real-world environments.

You’ll take technical ownership of AI evaluation and assurance, working closely with engineering, product and data science teams. Responsibilities will include:

• Designing model evaluation frameworks and test harnesses
• Evaluating and benchmarking LLMs against real-world use cases
• Measuring areas including precision, recall, grounding, calibration and hallucination
• Red teaming systems for prompt injection, jailbreaks, data leakage and tool misuse
• Stress-testing models using adversarial and out-of-distribution data
• Testing agent guardrails, permissions and containment
• Building automated evaluation into CI/CD pipelines
• Monitoring deployed models for drift and regression
• Supporting fine-tuning and model adaptation
• Producing technical assurance evidence and presenting findings to senior stakeholders and customers

You’ll also contribute to the wider AI assurance programme, including internal audit, certification and customer assurance activity.

What we’re looking for:

You’ll bring a strong Machine Learning Engineering or Data Science background alongside hands-on experience taking AI systems into production. Key experience includes:

• Production Python
• PyTorch, Hugging Face Transformers or similar
• Strong hands-on LLM experience
• RAG, vector search, prompting, tool use and agent orchestration
• Designing and running model evaluations
• Cloud deployment, containers and CI/CD
• Understanding AI attack surfaces and failure modes
• Applied statistics and experimental design

Experience in AI safety, adversarial ML, AI red teaming, technical assurance or Defence/Secure Government environments would be highly advantageous.

A rare opportunity to combine hands-on AI engineering with safety and assurance, working on technically challenging systems where reliability and trust genuinely matter.