At a Glance
- Tasks: Lead automated adversarial testing for cutting-edge AI platforms and ensure their security and compliance.
- Company: Join a forward-thinking tech company in London focused on AI innovation.
- Benefits: Competitive salary, flexible working hours, and opportunities for professional growth.
- Other info: Dynamic team environment with a focus on continuous learning and career advancement.
- Why this job: Make a real impact in AI safety and work with the latest technologies.
- Qualifications: Experience in regulated finance and strong Python development skills required.
The predicted salary is between 80000 - 100000 £ per year.
Lead AI Red Teaming & QA Engineer to design and execute automated adversarial testing for our enterprise Agentic AI platforms. You will move beyond traditional software QA to build continuous safety pipelines, ensuring our non-deterministic LLM agents, RAG systems, and tool integrations are secure, resilient, and compliant before production release.
Key Responsibilities
- Automated Adversarial Testing: Build and integrate automated red teaming suites into CI/CD pipelines using frameworks like Garak, Pyrit, and AgentDojo to enforce strict safety release gates.
- AI Evaluation Frameworks: Develop metrics and continuous testing for core AI risks, including hallucinations, memorisation, algorithmic bias, uncertainty, and model drift.
- Regulatory Compliance Evidence: Map threat models (OWASP LLM Top 10, Agentic threats) to automated test cases. Produce the technical testing evidence required by EU AI Act Article 15, DORA, and FCA Operational Resilience guidelines.
- Centralised AI-BOM Platform: Own the enterprise AI Bill of Materials (AI-BOM), tracking model lineages, dataset versions, and signed artifacts as a centralized evaluation service.
Required Technical Skills
- Regulated Finance: Proven experience testing software within FCA, DORA, or EU AI Act frameworks.
- AWS Bedrock Ecosystem: Hands‑on experience configuring, testing, and bypassing Bedrock Guardrails, Agents, and Knowledge Bases (RAG).
- AI Security & Fundamentals: Solid understanding of Foundation Models, tool usage (function calling), OWASP LLM Top 10, and NIST AI RMF.
- Automation Stack: Strong Python development skills, experience with AI eval tools (Garak, Pyrit, Ragas), and building complex CI/CD test pipelines.
Senior AI Red Teaming and Quality Assurance Engineer. Job in London LilyLifestyle Jobs employer: United Cerebral Palsy of Georgia
At LilyLifestyle, we pride ourselves on being an innovative leader in AI technology, offering a dynamic work environment in the heart of London. Our commitment to employee growth is reflected in our continuous learning culture, where you will have the opportunity to work with cutting-edge tools and frameworks while contributing to meaningful projects that shape the future of AI safety and compliance. Join us to be part of a collaborative team that values creativity, diversity, and the pursuit of excellence.
Contact Details:
United Cerebral Palsy of Georgia Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land Senior AI Red Teaming and Quality Assurance Engineer. Job in London LilyLifestyle Jobs
✨Tip Number 1
Network like a pro! Reach out to folks in the AI and QA space on LinkedIn or at industry events. A friendly chat can open doors that a CV just can't.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your automated testing projects or any cool AI tools you've built. This is your chance to shine beyond the written application.
✨Tip Number 3
Prepare for interviews by brushing up on the latest trends in AI security and compliance. We want to see you confidently discuss how you’d tackle challenges like algorithmic bias or model drift.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets the attention it deserves. Plus, we love seeing candidates who are proactive!
We think you need these skills to ace Senior AI Red Teaming and Quality Assurance Engineer. Job in London LilyLifestyle Jobs
Some tips for your application 🫡
Tailor Your CV:Make sure your CV speaks directly to the job description. Highlight your experience with automated adversarial testing and any relevant frameworks like Garak or Pyrit. We want to see how your skills align with our needs!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you're passionate about AI security and how your background in regulated finance makes you a perfect fit for us. Keep it engaging and personal!
Showcase Your Projects:If you've worked on any relevant projects, don’t hold back! Include links or descriptions of your work with CI/CD pipelines or AI evaluation frameworks. We love seeing practical examples of your skills in action.
Apply Through Our Website:We encourage you to apply through our website for a smoother process. It helps us keep track of applications better and ensures you get noticed. Plus, it’s super easy to do!
How to prepare for a job interview at United Cerebral Palsy of Georgia
✨Know Your Tech Inside Out
Make sure you’re well-versed in the technical skills listed in the job description. Brush up on your Python development skills and be ready to discuss your experience with frameworks like Garak and Pyrit. Being able to talk confidently about your hands-on experience with AWS Bedrock and AI security fundamentals will definitely impress.
✨Showcase Your Problem-Solving Skills
Prepare to discuss specific examples where you've tackled challenges in automated adversarial testing or regulatory compliance. Think of scenarios where you’ve built CI/CD pipelines or developed metrics for AI risks. This will demonstrate your ability to think critically and apply your knowledge in real-world situations.
✨Understand the Regulatory Landscape
Familiarise yourself with the EU AI Act, FCA guidelines, and DORA. Be prepared to explain how you’ve navigated these regulations in past roles. Showing that you can map threat models to automated test cases will highlight your understanding of compliance and safety in AI.
✨Ask Insightful Questions
Interviews are a two-way street! Prepare thoughtful questions about the company’s approach to AI safety and how they integrate red teaming into their processes. This not only shows your interest but also gives you a chance to assess if the company aligns with your values and career goals.
