At a Glance
- Tasks: Lead automated adversarial testing for cutting-edge AI platforms and ensure their safety and compliance.
- Company: Join a forward-thinking tech company in London focused on AI innovation.
- Benefits: Attractive salary, flexible working options, and opportunities for professional growth.
- Other info: Dynamic role with a chance to shape the future of AI technology.
- Why this job: Be at the forefront of AI security and make a significant impact in the tech industry.
- Qualifications: Experience in regulated finance and strong Python skills are essential.
The predicted salary is between 80000 - 100000 £ per year.
Lead AI Red Teaming & QA Engineer to design and execute automated adversarial testing for our enterprise Agentic AI platforms. You will move beyond traditional software QA to build continuous safety pipelines, ensuring our non-deterministic LLM agents, RAG systems, and tool integrations are secure, resilient, and compliant before production release.
Key Responsibilities
- Automated Adversarial Testing: Build and integrate automated red teaming suites into CI/CD pipelines using frameworks like Garak, Pyrit, and AgentDojo to enforce strict safety release gates.
- AI Evaluation Frameworks: Develop metrics and continuous testing for core AI risks, including hallucinations, memorisation, algorithmic bias, uncertainty, and model drift.
- Regulatory Compliance Evidence: Map threat models (OWASP LLM Top 10, Agentic threats) to automated test cases. Produce the technical testing evidence required by EU AI Act Article 15, DORA, and FCA Operational Resilience guidelines.
- Centralised AI-BOM Platform: Own the enterprise AI Bill of Materials (AI-BOM), tracking model lineages, dataset versions, and signed artifacts as a centralized evaluation service.
Required Technical Skills
- Regulated Finance: Proven experience testing software within FCA, DORA, or EU AI Act frameworks.
- AWS Bedrock Ecosystem: Hands‑on experience configuring, testing, and bypassing Bedrock Guardrails, Agents, and Knowledge Bases (RAG).
- AI Security & Fundamentals: Solid understanding of Foundation Models, tool usage (function calling), OWASP LLM Top 10, and NIST AI RMF.
- Automation Stack: Strong Python development skills, experience with AI eval tools (Garak, Pyrit, Ragas), and building complex CI/CD test pipelines.
Lead AI Red Teaming & QA Engineer. Job in London LilyLifestyle Jobs employer: United Cerebral Palsy of Georgia
At our London-based company, we pride ourselves on being an exceptional employer that fosters innovation and growth in the rapidly evolving field of AI. Our collaborative work culture encourages creativity and continuous learning, providing employees with ample opportunities to develop their skills while contributing to cutting-edge projects. With a focus on regulatory compliance and safety, we offer a unique environment where you can make a meaningful impact in the world of AI technology.
Contact Details:
United Cerebral Palsy of Georgia Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land Lead AI Red Teaming & QA Engineer. Job in London LilyLifestyle Jobs
✨Tip Number 1
Network like a pro! Reach out to folks in the AI and QA space on LinkedIn or at industry events. A friendly chat can open doors that a CV just can't.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your automated testing projects or any cool AI tools you've built. This is your chance to shine beyond the application.
✨Tip Number 3
Prepare for the interview by brushing up on the latest trends in AI security and compliance. We want you to be ready to discuss how you can tackle challenges like algorithmic bias and model drift.
✨Tip Number 4
Don't forget to apply through our website! It’s the best way to ensure your application gets the attention it deserves. Plus, we love seeing candidates who are proactive!
We think you need these skills to ace Lead AI Red Teaming & QA Engineer. Job in London LilyLifestyle Jobs
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Lead AI Red Teaming & QA Engineer role. Highlight your experience with automated testing, CI/CD pipelines, and any relevant frameworks like Garak or Pyrit. We want to see how your skills align with our needs!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you're passionate about AI security and how your background fits into our mission at StudySmarter. Keep it engaging and relevant to the job description.
Showcase Your Technical Skills:Don’t forget to highlight your technical skills in your application. Mention your experience with AWS Bedrock, Python development, and any specific AI evaluation tools you've used. We love seeing concrete examples of your work!
Apply Through Our Website:We encourage you to apply through our website for the best chance of getting noticed. It’s super easy, and you’ll be able to submit all your materials in one go. Plus, we keep an eye on applications that come through our site!
How to prepare for a job interview at United Cerebral Palsy of Georgia
✨Know Your Tech Inside Out
Make sure you’re well-versed in the technical skills listed in the job description. Brush up on your Python development skills and get familiar with frameworks like Garak, Pyrit, and AgentDojo. Being able to discuss your hands-on experience with these tools will show that you’re not just a theory person but someone who can get things done.
✨Understand Regulatory Frameworks
Since this role involves working within FCA, DORA, and EU AI Act frameworks, it’s crucial to understand these regulations. Prepare to discuss how you’ve navigated compliance in past projects and be ready to explain how you would approach regulatory challenges in adversarial testing.
✨Showcase Your Problem-Solving Skills
Be prepared to tackle hypothetical scenarios during the interview. Think about how you would build automated red teaming suites or develop metrics for AI risks. This is your chance to demonstrate your critical thinking and problem-solving abilities, so don’t hold back!
✨Ask Insightful Questions
Interviews are a two-way street, so come armed with questions that show your interest in the company and the role. Ask about their current challenges with AI security or how they envision the future of their AI platforms. This not only shows your enthusiasm but also helps you gauge if the company is the right fit for you.
