Lead Research Scientist/Engineer (Embedded Evaluations)

Lead Research Scientist/Engineer (Embedded Evaluations)

Full-Time 63000 - 77000 £ / year (est.) Hybrid
A

At a Glance

  • Tasks: Lead embedded evaluations at frontier AI labs and conduct impactful research.
  • Company: Join Apollo Research, a leader in AI safety and alignment.
  • Benefits: Enjoy competitive salary, unlimited vacation, and professional development budget.
  • Other info: Flexible working hours and visa sponsorship available for exceptional candidates.
  • Why this job: Make a real difference in AI safety while working with cutting-edge technology.
  • Qualifications: Strong analytical skills and experience in project management are essential.

The predicted salary is between 63000 - 77000 £ per year.

Application deadline: We are conducting interviews actively and aim to fill this role as soon as we find someone suitable.

ABOUT THE OPPORTUNITY

The OpenAI/Hugging Face incident may be the biggest warning shot we get before catastrophic loss-of-control. In its aftermath, Anthropic and OpenAI publicly committed to giving embedded evaluators an unprecedented level of access. Embedded evaluations are the best existing method to find out whether alignment is on track. With OpenAI predicting an automated researcher by March 2028, we don't have time to start with an MVP embedded evaluation and iteratively improve it. The first embedded evaluations will set the standard and we have to get it right. Apollo has taken steps toward becoming an embedded evaluator at frontier labs. We want to hire experienced senior researchers with a strong track record of getting things done to execute our embedded evaluation plans.

KEY RESPONSIBILITIES

  • On-site embedded evaluators: You will spend weeks or months embedded at a frontier AI lab, where you will conduct interviews, review the lab’s evidence, gather new evidence, and write a report. You will execute a playbook quickly and adapt to unforeseen circumstances. We hold our embedded evaluators to the highest standards, so you will likely have to first prove yourself within Apollo before being sent into an embedded campaign. That is, you will probably start on the applied research team to gain context on embedded evaluations. For this team, we prefer candidates with the right to work in the US who can be based in San Francisco.
  • Applied researchers: You will be the force‑multiplier behind embedded evaluations. You will prepare on‑site evaluators before they are embedded and support them while they are embedded. You will do the conceptual work to determine what evidence would support a safety case, and you will write the playbook for on‑site evaluators to optimize how fast they can gather evidence. You will develop tooling for embedded evaluators, both ahead‑of‑time and urgently on demand. You will be the bridge connecting fundamental research and the on‑site evaluators. For example, you might convert the methodology of "Measuring Reward‑Seeking via Contrastive Belief Updates" into turnkey code and procedures. This team can work from any of our offices. Both are high‑stakes teams. On‑site embedded evaluators should be exceptionally strong at quick execution under high pressure, and very strong at planning and research. Applied researchers should be exceptionally strong at planning and research, and very strong at quick execution under high pressure. For both teams, we are excited for experienced candidates who have owned ambitious projects end‑to‑end under tight deadlines.

KEY REQUIREMENTS

  • We don’t require a formal background or industry experience and welcome self‑taught candidates.
  • Data Analysis & Pattern Recognition: You can extract signal from large, messy datasets. You're comfortable with quantitative analysis and know when qualitative assessment is more appropriate. You can identify anomalies and unexpected model behaviours.
  • Writing for high‑stakes audiences: You have experience writing about qualitative and quantitative findings for different stakeholders. You can take a pile of unorganized technical information and communicate it effectively to lab safety teams, lab leadership, governments, or the public.
  • AI power‑user: You’re capable of using AI to accelerate your work, technical or otherwise. You have experience using different models, know which ones to use for which tasks, when not to use AI, and always experiment with new AI workflows.
  • Expertise in scheming‑related topics: You have a strong conceptual understanding of ideas like metagaming, evaluation awareness, deception, and reward‑seeking. You are familiar with classic literature like "Is Power‑Seeking AI an Existential Risk?" and modern research.
  • Hands‑on experience with frontier RL: Labs' post‑training pipelines are likely far more complicated than open‑source versions like Olmo 3. Embedded evaluators need to triage this complexity to reach a verdict on safety claims such as "Training did not incentivize goals or drives that would motivate scheming." Your deep knowledge of frontier RL will accelerate embedded evaluations.
  • Project management experience: You have owned an ambitious project end‑to‑end under a hard deadline. You have previously managed strong ICs and can keep research projects on schedule despite unexpected delays or surprising experiment results. On‑site embedded evaluators will need to keep months‑long evaluations on track, and applied researchers will need to make plans that are robust to changing circumstances and short turnarounds.
  • Professionalism: Your professional conduct is unimpeachable. You know when to tactfully resolve disagreements with partners and when to stand firm in negotiations, and you are excellent at both of those. Your previous boss trusted you to autonomously handle all interactions with partners for long periods.
  • Mission commitment: Apollo estimates that RSI will happen soon, and we are extremely concerned that AGI will catastrophically not be aligned with human values. Hence we have an "intense, but in a good way" culture. We will ask a lot from you, but in return you will personally have a large impact on loss‑of‑control risk.

NICE TO HAVES

  • Software engineering skills: Strong software engineering skills correlate highly with effective execution, even in an era of AI agents. Our entire stack uses Python.
  • Prior embedded expertise: You have previous experience in professional services like auditing or consulting. You’ve learned lessons from working on‑site with clients that will generalize to working with frontier AI labs.

We want to emphasize that people who feel they don’t fulfill all of these characteristics but think they would be a good fit for the position, nonetheless, are strongly encouraged to apply. We believe that excellent candidates can come from a variety of backgrounds and are excited to give you opportunities to shine.

BENEFITS

This role offers a market‑competitive salary, equity, and competitive benefits. Depending on your experience, you may be offered a Lead or individual contributor position. Salary: Recognised experts who can lead projects of significant scope: USD 350,000 - USD 590,000 or £230,000 - £395,000. Strong individual contributors: USD 245,000 - USD 405,000 or £160,000 - £270,000.

  • Unlimited vacation
  • Unlimited sick leave
  • Up to 6 months of paid parental leave
  • Comprehensive health, dental and vision insurance
  • Retirement savings with competitive employer matching (e.g. 401(k) for US employees)
  • Lunch, dinner, and snacks are provided for all employees on workdays
  • Paid work trips, including staff retreats, business trips, and relevant conferences
  • A yearly $1,000 (USD) professional development budget
  • Relocation support and visa fees (if applicable). We are able to sponsor visas to the UK and US.

LOGISTICS

Time Allocation: Full‑time. Location: This is an in‑person role working out of our London or San Francisco office. We prefer on‑site embedded evaluators to be permanently located in San Francisco. Applied researchers can work from any of our offices. We offer flexible working hours and WFH arrangements. Remote candidates will only be considered if they are exceptional and can travel at least 1 week/month to either office.

Visa sponsorship: We sponsor visas in both the UK and US. Sponsorship isn't guaranteed for every role or candidate, but if we make you an offer, we'll work with you to find the right visa route.

ABOUT APOLLO RESEARCH

The rapid rise in AI capabilities offers tremendous opportunities, but also presents significant risks. At Apollo Research, we’re primarily concerned with risks from Loss of Control, i.e. risks coming from the model itself rather than e.g. humans misusing the AI. We’re particularly concerned with deceptive alignment / scheming, a phenomenon where a model appears to be aligned but is, in fact, misaligned and capable of evading human oversight. We work on the detection of scheming (e.g., building evaluations and novel evaluation techniques), the science of scheming (e.g., model organisms and the study of scaling trends), and scheming mitigations (e.g., control). We closely work with multiple frontier AI companies, e.g. to test their models before deployment and collaborate on fundamental research. At Apollo, we aim for a culture that emphasizes truth‑seeking, being goal‑oriented, giving and receiving constructive feedback, and being friendly and helpful.

ABOUT THE TEAM

Alex Meinke leads the science of scheming team’s research agenda with guidance from Marius Hobbhahn, though team members lead individual projects. You will mostly work with the science of scheming team as well as our team of software engineers, but you will likely sometimes interact with the governance team to translate technical knowledge into concrete recommendations.

Equality Statement: Apollo Research is an Equal Opportunity Employer. We value diversity and are committed to providing equal opportunities to all, regardless of age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, or sexual orientation.

About the interview process: Our multi‑stage process includes a screening interview, 3 technical interviews, and a final interview with Marius (CEO). The technical interviews will be closely related to tasks the candidate would do on the job. There are no LeetCode‑style general coding interviews. If you want to prepare for the interviews, we suggest (a) thinking about what embedded evaluators should investigate, (b) building a tool that would be useful for embedded evaluators and running it on suitable proxy data, and/or (c) writing a report based on mock findings.

Your Privacy and Fairness in Our Recruitment Process: We are committed to protecting your data, ensuring fairness, and adhering to workplace fairness principles in our recruitment process. To enhance hiring efficiency, we use AI‑powered tools to assist with tasks such as resume screening. These tools are designed and deployed in compliance with internationally recognized AI governance frameworks. Your personal data is handled securely and transparently. We adopt a human‑centred approach: all resumes are screened by a human and final hiring decisions are made by our team. If you have questions about how your data is processed or wish to report concerns about fairness, please contact us at info@apolloresearch.ai.

Lead Research Scientist/Engineer (Embedded Evaluations) employer: Apollo Research

Apollo Research is an exceptional employer, offering a dynamic work environment where innovation meets security in the rapidly evolving AI landscape. With competitive salaries, unlimited vacation, and a strong emphasis on professional development, employees are empowered to grow and thrive. The collaborative culture fosters truth-seeking and constructive feedback, making it an ideal place for those passionate about tackling novel challenges in AI security.

A

Contact Details:

Apollo Research Recruitment Team

StudySmarter Expert Advice🤫

We think this is how you could land Lead Research Scientist/Engineer (Embedded Evaluations)

✨Get Involved in Research Communities

Dive headfirst into the scientific research world by joining relevant communities and forums. Engage in discussions, share your insights, and even attend conferences or seminars in your field. This not only boosts your visibility but can also lead to potential job opportunities—don't forget to connect with like-minded folks!

✨Show Off Your Research Projects

Have you worked on any cool research projects? Make it easy for potential employers to see your work by creating a portfolio or a personal website. This way, when you apply for roles like the one at Apollo Research, you can point them to your projects and publications, showcasing your expertise directly.

✨Utilise Professional Networks

Networking is key in scientific research. Join professional bodies or organisations related to your field. They often have job boards and resources tailored for job seekers. Make connections with professionals who may know about openings or can give you tips on landing a full-time position.

✨Keep Your Eyes on Openings & Apply Directly

Don’t just rely on job boards! Keep an eye on the careers section of the websites of companies like Apollo Research. Apply directly through their website because sometimes they post jobs there before anywhere else. Plus, it shows your proactive approach!

We think you need these skills to ace Lead Research Scientist/Engineer (Embedded Evaluations)

Data Analysis
Pattern Recognition
Writing for High-Stakes Audiences
AI Power-User
Expertise in Scheming-Related Topics
Hands-On Experience with Frontier RL
Project Management

Some tips for your application 🫡

Highlight Your Research Experience:When applying for a full-time role in scientific research, make sure to emphasise your research experience prominently in your CV. Share specific projects you’ve worked on, the methodologies you used, and any significant findings. If you’ve published papers or presented at conferences, definitely include that too – it shows you’re on it in the academic world!

Tailor Your Cover Letter to the Research Area:Your cover letter should reflect your passion for the specific area of research at Apollo Research. Mention relevant experiences that align with the organisation’s goals or projects. This shows that you’ve done your homework and are genuinely interested in the position – plus, it helps us see how you’d fit into the team dynamics.

Showcase Your Data Analysis Skills:In scientific research, data analysis skills are a big deal! Make sure to detail any relevant analytical tools or software you’re familiar with, like R, Python, or statistical packages. Employers are keen to know you can handle the data-heavy elements of the role, so add specific examples where you’ve used these skills effectively.

Discuss Your Future Research Goals:In your motivation section, it’s a great idea to talk about your future research goals and how they align with the work being done at Apollo Research. This shows that you’re not just looking for any job, but rather a chance to contribute meaningfully to the field. We love to see applicants who are forward-thinking and enthusiastic about their research journey!

How to prepare for a job interview at Apollo Research

✨Showcase Your Research Skills

In scientific research, it’s crucial to demonstrate your ability to design and conduct experiments. Come armed with examples of past projects where you've developed hypotheses, collected data, and analysed results. Be ready to discuss any specific methodologies or tools you’ve used, like PCR techniques or statistical software.

✨Prepare for Technical Questions

Expect some technical questions specific to your field. Make sure you're up to speed with recent advancements in scientific research related to the role at Apollo Research. Brush up on concepts relevant to their projects and be prepared to discuss how you would approach a specific research problem or challenge they might face.

✨Know Your Publications

If you've authored or co-authored any papers, be prepared to discuss them! Highlighting your contributions to published research can really set you apart. It shows not only your expertise but also your ability to communicate complex ideas clearly, which is key in scientific research roles.

✨Exhibit Your Team Spirit

In full-time roles, collaboration is often at the heart of scientific research. Prepare examples that show how you've successfully worked in teams, dealt with conflicts, or contributed to group projects. We want to know how you can work effectively with the team at Apollo Research to drive research projects forward.