At a Glance
- Tasks: Lead automated adversarial testing for cutting-edge AI platforms and ensure their security and compliance.
- Company: Join a forward-thinking tech company in London focused on AI innovation.
- Benefits: Competitive salary, flexible working hours, and opportunities for professional growth.
- Other info: Dynamic team environment with a focus on continuous learning and career advancement.
- Why this job: Be at the forefront of AI safety and make a significant impact in the tech industry.
- Qualifications: Experience in regulated finance testing and strong Python development skills required.
The predicted salary is between 80000 - 100000 £ per year.
Lead AI Red Teaming & QA Engineer to design and execute automated adversarial testing for our enterprise Agentic AI platforms. You will move beyond traditional software QA to build continuous safety pipelines, ensuring our non-deterministic LLM agents, RAG systems, and tool integrations are secure, resilient, and compliant before production release.
Key Responsibilities
- Automated Adversarial Testing: Build and integrate automated red teaming suites into CI/CD pipelines using frameworks like Garak, Pyrit, and AgentDojo to enforce strict safety release gates.
- AI Evaluation Frameworks: Develop metrics and continuous testing for core AI risks, including hallucinations, memorisation, algorithmic bias, uncertainty, and model drift.
- Regulatory Compliance Evidence: Map threat models (OWASP LLM Top 10, Agentic threats) to automated test cases. Produce the technical testing evidence required by EU AI Act Article 15, DORA, and FCA Operational Resilience guidelines.
- Centralised AI-BOM Platform: Own the enterprise AI Bill of Materials (AI-BOM), tracking model lineages, dataset versions, and signed artifacts as a centralized evaluation service.
Required Technical Skills
- Regulated Finance: Proven experience testing software within FCA, DORA, or EU AI Act frameworks.
- AWS Bedrock Ecosystem: Hands-on experience configuring, testing, and bypassing Bedrock Guardrails, Agents, and Knowledge Bases (RAG).
- AI Security & Fundamentals: Solid understanding of Foundation Models, tool usage (function calling), OWASP LLM Top 10, and NIST AI RMF.
- Automation Stack: Strong Python development skills, experience with AI eval tools (Garak, Pyrit, Ragas), and building complex CI/CD test pipelines.
Lead AI Red Teaming & QA Engineer. Job in London LilyLifestyle Jobs employer: United Cerebral Palsy of Georgia
At our London-based company, we pride ourselves on being an exceptional employer that fosters a culture of innovation and collaboration. As a Lead AI Red Teaming & QA Engineer, you will have access to cutting-edge technology and the opportunity to work alongside industry experts, all while enjoying a supportive environment that prioritises employee growth and development. With competitive benefits and a commitment to regulatory compliance, we ensure that our team members are equipped to thrive in their roles and contribute meaningfully to the future of AI safety.
Contact Details:
United Cerebral Palsy of Georgia Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land Lead AI Red Teaming & QA Engineer. Job in London LilyLifestyle Jobs
✨Tip Number 1
Network like a pro! Reach out to folks in the AI and QA space on LinkedIn or at industry events. A friendly chat can open doors that a CV just can't.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your automated testing projects, especially those using frameworks like Garak or Pyrit. This will give potential employers a taste of what you can do.
✨Tip Number 3
Prepare for interviews by brushing up on the latest in AI security and compliance. Be ready to discuss how you've tackled challenges related to algorithmic bias or model drift in past roles.
✨Tip Number 4
Don't forget to apply through our website! We love seeing candidates who are genuinely interested in joining us at StudySmarter. It shows you're keen and ready to dive into the role!
We think you need these skills to ace Lead AI Red Teaming & QA Engineer. Job in London LilyLifestyle Jobs
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Lead AI Red Teaming & QA Engineer role. Highlight your experience with automated testing, CI/CD pipelines, and any relevant frameworks like Garak or Pyrit. We want to see how your skills align with our needs!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you're passionate about AI security and how your background fits into our mission at StudySmarter. Keep it engaging and relevant to the job description.
Showcase Your Technical Skills:Don’t hold back on showcasing your technical skills! Mention your hands-on experience with AWS Bedrock, Python development, and any AI evaluation tools you've used. We love seeing concrete examples of your work!
Apply Through Our Website:We encourage you to apply through our website for a smoother application process. It helps us keep track of your application and ensures you don’t miss out on any important updates from us!
How to prepare for a job interview at United Cerebral Palsy of Georgia
✨Know Your Tech Inside Out
Make sure you’re well-versed in the tools and frameworks mentioned in the job description, like Garak, Pyrit, and AgentDojo. Brush up on your Python skills and be ready to discuss how you've used these technologies in past projects.
✨Understand Regulatory Compliance
Familiarise yourself with the EU AI Act, FCA, and DORA guidelines. Be prepared to explain how you’ve navigated compliance in previous roles and how you would approach mapping threat models to automated test cases.
✨Showcase Your Problem-Solving Skills
Prepare examples of how you’ve tackled challenges in automated adversarial testing or AI evaluation. Think about specific instances where you identified risks like algorithmic bias or model drift and how you addressed them.
✨Ask Insightful Questions
At the end of the interview, don’t shy away from asking questions that show your interest in the role and the company. Inquire about their current safety pipelines or how they measure success in AI security. This shows you’re engaged and thinking critically about the position.
