At a Glance
- Tasks: Lead automated adversarial testing for cutting-edge AI platforms and ensure their security and compliance.
- Company: Join a forward-thinking tech company in London focused on AI innovation.
- Benefits: Competitive salary, flexible working hours, and opportunities for professional growth.
- Other info: Dynamic team environment with excellent career advancement opportunities.
- Why this job: Make a real impact in AI safety and work with the latest technologies.
- Qualifications: Experience in regulated finance testing and strong Python development skills required.
The predicted salary is between 80000 - 100000 £ per year.
Lead AI Red Teaming & QA Engineer to design and execute automated adversarial testing for our enterprise Agentic AI platforms. You will move beyond traditional software QA to build continuous safety pipelines, ensuring our non-deterministic LLM agents, RAG systems, and tool integrations are secure, resilient, and compliant before production release.
Key Responsibilities
- Automated Adversarial Testing: Build and integrate automated red teaming suites into CI/CD pipelines using frameworks like Garak, Pyrit, and AgentDojo to enforce strict safety release gates.
- AI Evaluation Frameworks: Develop metrics and continuous testing for core AI risks, including hallucinations, memorisation, algorithmic bias, uncertainty, and model drift.
- Regulatory Compliance Evidence: Map threat models (OWASP LLM Top 10, Agentic threats) to automated test cases. Produce the technical testing evidence required by EU AI Act Article 15, DORA, and FCA Operational Resilience guidelines.
- Centralised AI-BOM Platform: Own the enterprise AI Bill of Materials (AI-BOM), tracking model lineages, dataset versions, and signed artifacts as a centralized evaluation service.
Required Technical Skills
- Regulated Finance: Proven experience testing software within FCA, DORA, or EU AI Act frameworks.
- AWS Bedrock Ecosystem: Hands-on experience configuring, testing, and bypassing Bedrock Guardrails, Agents, and Knowledge Bases (RAG).
- AI Security & Fundamentals: Solid understanding of Foundation Models, tool usage (function calling), OWASP LLM Top 10, and NIST AI RMF.
- Automation Stack: Strong Python development skills, experience with AI eval tools (Garak, Pyrit, Ragas), and building complex CI/CD test pipelines.
Senior AI Red Teaming and Quality Assurance Engineer. Job in London LilyLifestyle Jobs employer: United Cerebral Palsy of Georgia
At LilyLifestyle, we pride ourselves on being an innovative leader in AI technology, offering a dynamic work environment in the heart of London. Our commitment to employee growth is reflected in our continuous learning culture, where you will have the opportunity to work with cutting-edge tools and frameworks while contributing to meaningful projects that ensure the safety and compliance of our AI systems. Join us for a rewarding career where your expertise in AI Red Teaming and Quality Assurance will be valued and nurtured.
Contact Details:
United Cerebral Palsy of Georgia Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land Senior AI Red Teaming and Quality Assurance Engineer. Job in London LilyLifestyle Jobs
✨Tip Number 1
Network like a pro! Get out there and connect with folks in the AI and tech scene. Attend meetups, webinars, or even local events. You never know who might have the inside scoop on job openings or can put in a good word for you.
✨Tip Number 2
Show off your skills! Create a portfolio or GitHub repository showcasing your projects, especially those related to automated adversarial testing or AI security. This gives potential employers a taste of what you can do beyond just a CV.
✨Tip Number 3
Prepare for interviews by brushing up on the latest trends in AI and regulatory compliance. Be ready to discuss how you would tackle challenges like algorithmic bias or model drift. We want to see your thought process!
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive about their job search.
We think you need these skills to ace Senior AI Red Teaming and Quality Assurance Engineer. Job in London LilyLifestyle Jobs
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the role of Senior AI Red Teaming and QA Engineer. Highlight your experience with automated testing, CI/CD pipelines, and any relevant frameworks like Garak or Pyrit. We want to see how your skills align with our needs!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you're passionate about AI security and how your background fits into our mission at StudySmarter. Be specific about your achievements and how they relate to the job description.
Showcase Relevant Projects:If you've worked on projects that involve adversarial testing or compliance with regulations like the EU AI Act, make sure to mention them. We love seeing real-world applications of your skills, so don’t hold back!
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for the role. Plus, it shows you’re keen on joining the StudySmarter team!
How to prepare for a job interview at United Cerebral Palsy of Georgia
✨Know Your Tech Inside Out
Make sure you’re well-versed in the tools and frameworks mentioned in the job description, like Garak, Pyrit, and AgentDojo. Brush up on your Python skills and be ready to discuss how you've used these technologies in past projects.
✨Understand Regulatory Compliance
Familiarise yourself with the EU AI Act, FCA, and DORA guidelines. Be prepared to explain how you’ve ensured compliance in previous roles, and think of examples where you mapped threat models to automated test cases.
✨Showcase Your Problem-Solving Skills
Prepare to discuss specific challenges you've faced in AI testing and how you overcame them. Highlight your experience with adversarial testing and continuous safety pipelines, as this will demonstrate your ability to think critically and adapt.
✨Ask Insightful Questions
At the end of the interview, don’t shy away from asking questions about the company’s approach to AI security and quality assurance. This shows your genuine interest in the role and helps you gauge if it’s the right fit for you.
