Overview
In this role you will design and run AI evaluation pipelines to test mental health tools, using simulated users and automated judges. You’ll collaborate with partners to adapt tooling for diverse countries and languages, and contribute to research across PATH’s AI portfolio. You may lead small research initiatives and publish results, helping to ensure safe, effective AI for health at scale. This position sits in PATH’s London project office and involves occasional international travel.
Responsibilities
- Design and run benchmarking pipelines for AI evaluation in global mental health
- Scale evaluations by creating simulated users and multi-session conversations; score transcripts with automated judges and analyze results
- Collaborate with technical partners to adapt tooling for local contexts and products
- Ensure results are traceable, reproducible, and publication-ready
- Develop and maintain tools (e.g., user simulation, integration with expert input) and produce well-tested, documented code for open release
- Contribute to PATH’s AI for Health research portfolio with engineering support and ideas; potentially lead discrete research projects and publish findings
- Provide knowledge transfer to partners and stakeholders to enable independent evaluations
Key requirements
- BSc or MSc in computer science, software engineering, or related field, or equivalent practical experience
- Strong Python with well-tested, maintainable code (version control, testing, CI)
- Experience evaluating AI models via evaluation pipelines or large-scale research contributions
- Experience with cloud infrastructure, containers, and data pipelines (AWS, GCP, Azure; Docker)
- Ability to work independently with external partners and communicate technical work to clinicians, researchers, and policy teams
- Ability to handle sensitive mental health data with care
- Independent and proactive work style
- Clear written and verbal communication
- Collaborative mindset and ability to work with cross-functional teams
- Python programming
- Cloud infrastructure (AWS, GCP, or Azure)
- Containers (Docker)
Research Engineer, AI for Health in London employer: PATH
PATH is an exceptional employer that champions health equity and offers a collaborative work culture where interns can make a meaningful impact. With a focus on employee growth, the internship provides valuable experience in partnership strategy while working alongside dedicated professionals in a supportive environment. Located in the heart of the EU, this role not only offers a competitive stipend but also the opportunity to contribute to global health initiatives that truly matter.
