At a Glance
- Tasks: Develop evaluation frameworks for AI in clinical operations and global development.
- Company: Join a leading pharma organisation at the forefront of AI innovation.
- Benefits: Two-year postdoctoral appointment with opportunities for hands-on AI experience.
- Other info: Collaborate with experts across disciplines and contribute to cutting-edge AI projects.
- Why this job: Make a real impact by shaping how AI is evaluated in clinical settings.
- Qualifications: PhD or equivalent experience in clinical, biomedical, or regulatory fields required.
The predicted salary is between 35000 - 45000 £ per year.
We are deploying generative AI agents across clinical operations and global development workflows: clinical data review, anomaly detection in trial data, protocol Q&A for site teams, Informed Consent Form generation and translation, and protocol element document assembly. Each system needs rigorous evaluation. In specialized clinical contexts, "good" is not a single number. It depends on the user, the regulatory context, the therapeutic area, and the cost profile of errors. We are looking for a postdoctoral researcher to help us develop the evaluation frameworks, rubrics, and quality systems that determine when these AI agents are ready for our scientists, clinical operations specialists, and site teams to rely on.
Key Responsibilities
- Develop evaluation frameworks and rubrics for AI agents operating in clinical operations and global development contexts.
- Define quality dimensions for specialized outputs, e.g. clinical data queries, anomaly detection, protocol Q&A responses, ICF content, regulatory documents, that reflect how clinical-ops users actually judge quality.
- Design and run human evaluation protocols with clinical, regulatory, and site-facing subject matter experts.
- Build benchmark datasets and evaluation pipelines that the broader GenAI organization uses across AI applications.
- Partner with our AI engineering teams to translate evaluation findings into measurable improvement.
- Contribute to the methodology of how a regulated pharma organization evaluates AI in clinical work.
What we're looking for
- Domain expertise: Strong understanding of clinical operations and global development workflows, e.g. clinical data management, monitoring, regulatory documentation, site operations, or related areas.
- Familiarity with how clinical-ops specialists, monitors, regulatory authors, and site teams actually do their work and assess quality.
- PhD or equivalent research experience in a clinical, biomedical, regulatory, biostatistics, or closely related field.
- Enthusiasm for AI agents: Genuine interest in working hands-on with generative AI and agentic systems. Curiosity about how LLMs behave in specialized domains and where they fail. Comfort prototyping with modern AI tooling (LLM APIs, vector databases, evaluation frameworks).
- Aptitude for evaluation systems: Demonstrated ability to develop frameworks, rubrics, and structured assessment protocols. Experience designing evaluations with multiple quality dimensions, expert raters, or non-trivial inter-rater reliability considerations. Strong analytical and writing skills; comfort with ambiguity and unsolved problems.
- Practical skills: Python and the modern ML/data stack. Comfort working across disciplines, e.g. clinical operations, regulatory affairs, AI engineering, data science.
Practical details
- Duration: 2-year postdoctoral appointment.
- Reports to: [TBD].
- Works closely with Generative AI Teams, Scientific Fellow, clinical operations and regulatory subject matter experts across J&J global development.
Post Doctoral Researcher, Evaluation & Standards in Clinical Operations employer: Johnson & Johnson
As a leading innovator in the deployment of generative AI within clinical operations, we offer a dynamic work environment that fosters collaboration and creativity. Our commitment to employee growth is evident through our robust training programmes and opportunities to engage with cutting-edge technology, ensuring that you can make a meaningful impact in the field of clinical research. Located in a vibrant area, our inclusive culture promotes diversity and encourages a healthy work-life balance, making us an exceptional employer for those seeking rewarding careers in healthcare and technology.
StudySmarter Expert Advice🤫
We think this is how you could land Post Doctoral Researcher, Evaluation & Standards in Clinical Operations
✨Tip Number 1
Network like a pro! Reach out to professionals in clinical operations and AI through LinkedIn or industry events. We can’t stress enough how personal connections can open doors that applications alone can’t.
✨Tip Number 2
Show off your expertise! Prepare a portfolio or presentation that highlights your research and evaluation frameworks. When you get the chance to chat with potential employers, let them see your passion and knowledge in action.
✨Tip Number 3
Practice makes perfect! Conduct mock interviews with friends or mentors who understand the field. We want you to feel confident discussing your experience and how it relates to the role of a Post Doctoral Researcher.
✨Tip Number 4
Apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who take the initiative to engage directly with us.
We think you need these skills to ace Post Doctoral Researcher, Evaluation & Standards in Clinical Operations
Some tips for your application 🫡
Tailor Your Application:Make sure to customise your CV and cover letter for the role. Highlight your experience in clinical operations and any relevant research you've done. We want to see how your background aligns with our needs!
Showcase Your Skills:Don’t just list your qualifications; demonstrate them! Use specific examples of frameworks or evaluation systems you've developed. This helps us understand your practical skills and how you can contribute to our team.
Be Clear and Concise:When writing your application, keep it straightforward. Avoid jargon unless it's necessary, and make sure your points are easy to follow. We appreciate clarity as much as complexity!
Apply Through Our Website:We encourage you to submit your application through our website. It’s the best way for us to receive your details and ensures you’re considered for the role. Plus, it’s super easy!
How to prepare for a job interview at Johnson & Johnson
✨Know Your Stuff
Make sure you brush up on your knowledge of clinical operations and global development workflows. Understand how clinical data management, regulatory documentation, and site operations work. This will help you answer questions confidently and show that you're the right fit for the role.
✨Show Your Passion for AI
Demonstrate your enthusiasm for generative AI and how it can transform clinical operations. Be ready to discuss your experiences with AI tools and your curiosity about their behaviour in specialised domains. This will highlight your genuine interest in the field.
✨Prepare Evaluation Frameworks
Since the role involves developing evaluation frameworks and rubrics, come prepared with examples of how you've approached similar tasks in the past. Think about the quality dimensions you would consider and be ready to discuss your methodology.
✨Practice Problem-Solving
Expect some scenario-based questions that test your analytical skills and ability to handle ambiguity. Practise articulating your thought process when faced with complex problems, especially in relation to AI evaluations and clinical contexts. This will showcase your critical thinking abilities.
