At a Glance
- Tasks: Run evaluations on cutting-edge AI systems and automate testing pipelines.
- Company: Join a pioneering team working with top AI labs like OpenAI and Google DeepMind.
- Benefits: Competitive salary, flexible work environment, and opportunities for professional growth.
- Other info: Dynamic role with access to groundbreaking AI models and innovative projects.
- Why this job: Be at the forefront of AI research and influence deployment decisions.
- Qualifications: Strong Python skills and a passion for optimising processes.
The predicted salary is between 60000 - 80000 £ per year.
Application deadline We are conducting interviews actively and aim to fill this role as soon as we find someone suitable.
About the opportunity We develop and run evaluations that help assess the risks posed by scheming AIs.
You will get to work with frontier labs like Open AI, Anthropic, and Google Deep Mind and be amongst the first to interact with new models before anyone else.
The ideal candidate loves rigorously testing frontier AI models, and enjoys building efficient pipelines and automating them.
You will have the opportunity to Run pre-deployment evaluation campaigns on the most capable AI systems in the world.
We partner with multiple labs, giving you access to a breadth of models that no single AI lab could offer.
You\'ll be among the first people to interact with new models before anyone else.
Deep dive into AI cognition.
Scan through thousands of model transcripts to surface behavioral patterns that no one has ever observed before.
These patterns are often deeply surprising and fascinating to study, e. g. the non-standard language and the reward-seeking reasoning described in our anti-scheming paper.
Build new evaluations for frontier risks , from designing novel test environments to scaling them across hundreds of distinct scenarios.
Work directly with frontier AI developers.
Share your findings, engage with their feedback, and see your evaluations directly inform deployment decisions for the most capable AI systems in the world.
Automate and improve the evaluation pipeline.
We already use automation across building, running, and analyzing evals.
Rapid progress in agent capabilities opens up radically new possibilities, and you\'ll have the freedom to rethink and reshape the pipeline as they emerge.
Key requirements Software engineering skills: Our entire stack uses Python.
We\'re looking for candidates with strong software engineering experience.
Ideally, you have experience shipping and maintaining production Python code, and know how to factor messy problems into clean abstractions that others can use and extend.
Process optimisation: You always try to improve workflows.
Pre-deployment evaluations are very fast paced so ideally you love shaving friction off your workflows wherever possible.
Data Analysis
