At a Glance
- Tasks: Join our Red Team to research and evaluate AI safety measures.
- Company: Be part of the world's leading AI Security Institute, shaping global AI governance.
- Benefits: Enjoy competitive salary, flexible working, and generous leave policies.
- Other info: Dynamic environment with opportunities for growth and influence.
- Why this job: Make a real impact on AI safety while collaborating with top experts.
- Qualifications: PhD students or recent graduates in machine learning or related fields.
The predicted salary is between 65000 - 145000 £ per year.
About the AI Security Institute
The AI Security Institute is the world's largest and best-funded team dedicated to understanding advanced AI risks and translating that knowledge into action. We’re in the heart of the UK government with direct lines to No. 10 (the Prime Minister's office), and we work with frontier developers and governments globally. We’re here because governments are critical for advanced AI going well, and UK AISI is uniquely positioned to mobilise them. With our resources, unique agility and international influence, this is the best place to shape both AI development and government action.
The deadline for applying to this role is Tuesday 8th December, end of day, anywhere on Earth.
About the team
The Red Team conducts cutting-edge research to evaluate the safety measures of frontier AI systems, sharing our findings directly with leading AI companies, UK officials and allied governments. Our work shapes how the most powerful AI systems are deployed - you can read about it in The New York Times, The Economist, and The Telegraph. Demand for this work is growing faster than we can currently meet - we're scaling to address that gap.
The Red Team’s work is crucial to ensuring advanced AI systems are developed and deployed safely. We are composed of three specialised sub-teams:
- Alignment – developing methods to stress-test whether models reliably act as intended.
- Control – researching methods for stress-testing and evaluating the measures that identify and prevent agents taking harmful actions.
- Misuse - stress-testing safeguards against misuse of dangerous capabilities.
While each team has its own focus, we share methodologies, infrastructure and tooling across the wider team.
Role Description
This is an exciting opportunity for motivated and talented researchers to join AISI's Red Team, contributing to priority research objectives within a fast-paced environment in government. We are looking for exceptional candidates currently studying for a PhD, or postdoctoral researchers who have submitted their PhD within the last 2 years, in machine learning, computer science, computer security or a related field. We will by default consider this a 6-month residency but are flexible to shorten or extend per candidate preferences and availability. Following an end of residency review, promising candidates may be offered the opportunity to join their preferred team in a full-time employee role.
We are particularly interested in candidates who have one or more of the following:
- Research experience in adversarial machine learning, AI security, alignment, AI control or evaluations.
- Experience with optimisation methods such as reinforcement learning, fine-tuning or evolutionary methods.
- Familiarity with loss-of-control threat models, such as deceptive alignment, research sabotage or reward-seeking.
- Hands-on experience red teaming or jailbreaking AI systems.
- Experience building something with LLMs, such as an evaluation, an agent or a fine-tuned model.
- Experience using ML frameworks such as PyTorch or Hugging Face.
- Shared research or technical work publicly, for example a paper, preprint, open-source tool or blog post.
What you’ll be doing
As a resident on one of our sub teams, you:
- Work closely with research scientists to decide which questions matter most and how to test them.
- Work on a defined project with support from an experienced research engineer or scientist.
- Help design and run attacks and evaluations on frontier AI systems to find weaknesses in their safety measures.
- Write code and help build tools to run experiments at scale.
- Analyse experiment results carefully and turn them into clear findings for technical and policy audiences.
- Keep up with new attack techniques and the research literature.
- Learn how frontier AI models are evaluated and red teamed in practice.
Illustrative projects
The projects below are examples of the kind of work you might do:
- Misuse – Create and improve novel automated research approaches for developing better red-teaming algorithms.
- Alignment – Build methods that automate the identification of harmful or unsanctioned model behaviour.
- Control – Develop better ways to evaluate AI monitors.
Person specification
You may be a good fit if you have some of the following skills, experience and attitudes:
- Experience running experiments or analysing data.
- Good research habits.
- Curiosity about how systems fail.
- Clear written and spoken communication.
- Openness to feedback.
- Motivation to reduce risks from advanced AI.
Highly Desirable
We additionally may factor in experience with particular areas like:
- Published or co-authored research on adversarial ML, jailbreaks, data poisoning, AI safeguards, alignment, control or evaluations.
- Previous research experience in ML or AI safety.
- A background in security research.
- Experience fine-tuning or modifying open-weight models.
- Familiarity with the alignment or AI control literature.
- Experience testing the security of AI agents.
This might not be the role for you if:
- You want to lead or set research strategy from day one.
- You prefer well-defined requirements and long planning cycles.
- You are uncomfortable in a collaborative research environment.
Motivated candidates are encouraged to apply even if they don't meet all of the above criteria.
Selection process
The interview process may vary from candidate to candidate, however, you should expect a typical process to include some technical proficiency tests, discussions with a cross-section of our team at AISI and conversations with your team lead.
What We Offer
- Impact you couldn't have anywhere else.
- Incredibly talented, mission-driven and supportive colleagues.
- Direct influence on how frontier AI is governed and deployed globally.
- Opportunity to shape the first & best-resourced public-interest research team focused on AI security.
Resources & access
- Pre-release access to multiple frontier models and ample compute.
- Extensive operational support.
- Work with experts across national security, policy, AI research and adjacent sciences.
Growth & autonomy
- If you’re talented and driven, you’ll own important problems early.
- 5 days off and annual stipends for learning and development.
- Freedom to pursue research bets without product pressure.
Life & family
- Modern central London office, or similar government offices in other cities.
- Hybrid working, flexibility for occasional remote work abroad.
- At least 25 days’ annual leave, 8 public holidays, extra team-wide breaks and 3 days off for volunteering.
- Generous paid parental leave.
- Excellent pension provision.
Salary
Annual salary is benchmarked to role scope and relevant experience. Most offers land between £65,000 and £145,000 made up of a base salary plus a technical allowance.
Additional Information
Use of AI in Applications: Artificial Intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience.
Security: Successful candidates must undergo a criminal record check and get baseline personnel security standard (BPSS) clearance before they can be appointed.
Nationality requirements: We may be able to offer roles to applicants from any nationality or background.
Working for the Civil Service: The Civil Service Code sets out the standards of behaviour expected of civil servants.
Diversity and Inclusion: The Civil Service is committed to attract, retain and invest in talent wherever it is found.
Research Residency – Red Team in London employer: AI Security Institute
The AI Security Institute is an exceptional employer, offering a dynamic work culture that fosters innovation and collaboration among talented professionals. Located in a vibrant research environment, employees benefit from continuous growth opportunities, competitive compensation, and the chance to contribute to cutting-edge projects in AI security. Join us to be part of a mission-driven team that values your expertise and encourages professional development.