At a Glance
- Tasks: Design and implement automated testing frameworks for cutting-edge AI systems.
- Company: Join a forward-thinking tech company in the heart of London.
- Benefits: Competitive salary, hybrid work model, and opportunities for career growth.
- Other info: Dynamic team environment with a focus on innovation and collaboration.
- Why this job: Be at the forefront of AI technology and make a real impact.
- Qualifications: Strong Python skills and experience with test automation frameworks.
The predicted salary is between 60000 - 80000 £ per year.
This is a Fixed Term Contract until the end of 2026 but is highly likely to extend. We are looking for an AI Test Automation Engineer to join on a fixed-term contract until the end of the year, with potential to extend. The role is based in London, with some office presence required.
You will help design and implement automated testing and evaluation frameworks for MCP Servers, LLM-based systems, and agentic AI workflows, ensuring accuracy, safety, reliability, and performance across complex AI-enabled platforms.
Key Responsibilities- Develop automated testing frameworks for MCP Servers and related AI systems.
- Design evaluation strategies for LLM accuracy, safety, and reliability.
- Build automated tests using Python, Pytest, and BDD frameworks.
- Identify and address AI failure modes including hallucination, latency, and incorrect tool usage.
- Work with engineering, QA, and product teams to define quality metrics and acceptance criteria.
- Support Agile delivery, ensuring testing aligns with sprint goals.
- Produce reporting on quality outcomes, risks, and improvement areas.
- Maintain documentation for test cases, evaluation pipelines, and validation strategies.
- Strong Python programming experience for test automation and evaluation.
- Expertise in Pytest and familiarity with BDD tools such as Behave or Cucumber.
- Knowledge of LLM evaluation approaches such as RAGAS, DeepEval, or custom evaluation pipelines.
- Understanding of agentic AI issues including hallucination, tool misuse, and performance bottlenecks.
- Experience testing AI workflows, distributed systems, or microservices environments.
- Strong knowledge of Agile delivery and CI/CD quality integration.
- Excellent communication, analytical, and problem-solving skills.
- Experience with Model Context Protocol (MCP) or agent orchestration solutions.
- Exposure to observability, monitoring, or logging tools for AI systems.
- Background in API and service integration testing.
- Knowledge of containerised and cloud-native environments.
- Experience in enterprise AI automation or intelligent platform engineering.
Test Automation Specialist in London employer: Robson Bale
Join a forward-thinking company that values innovation and collaboration, where as a Test Automation Specialist in London, you will be at the forefront of developing cutting-edge AI testing frameworks. Our hybrid work culture promotes flexibility and work-life balance, while our commitment to employee growth ensures you have access to continuous learning opportunities and career advancement. With a focus on meaningful projects and a supportive team environment, this role offers a unique chance to contribute to the future of AI technology.
StudySmarter Expert Advice🤫
We think this is how you could land Test Automation Specialist in London
✨Tip Number 1
Network like a pro! Reach out to folks in the industry, attend meetups, and connect with potential colleagues on LinkedIn. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your test automation projects, especially those using Python and Pytest. This gives you a chance to demonstrate your expertise and makes you stand out from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on common test automation scenarios and AI-related challenges. Practice explaining your thought process and how you tackle issues like hallucination or performance bottlenecks.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive about their job search!
We think you need these skills to ace Test Automation Specialist in London
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Test Automation Specialist role. Highlight your Python programming skills and experience with Pytest and BDD tools. We want to see how your background aligns with our needs!
Showcase Your Projects:Include specific projects where you've developed automated testing frameworks or worked with AI systems. This gives us a clear picture of your hands-on experience and problem-solving skills in action.
Be Clear and Concise:When writing your cover letter, keep it clear and to the point. Explain why you're excited about the role and how you can contribute to our team. We appreciate straightforward communication!
Apply Through Our Website:Don’t forget to apply through our website! It’s the best way for us to receive your application and ensures you’re considered for the role. We can’t wait to hear from you!
How to prepare for a job interview at Robson Bale
✨Know Your Tech Inside Out
Make sure you brush up on your Python skills and get familiar with Pytest and BDD frameworks like Behave or Cucumber. Be ready to discuss how you've used these tools in past projects, as well as any specific challenges you've faced in test automation.
✨Understand AI Evaluation Techniques
Familiarise yourself with LLM evaluation approaches such as RAGAS and DeepEval. Be prepared to explain how you would design evaluation strategies for accuracy, safety, and reliability in AI systems, as this will show your depth of knowledge in the field.
✨Showcase Your Agile Experience
Since the role involves Agile delivery, be ready to talk about your experience working in Agile teams. Highlight how you've aligned testing with sprint goals and contributed to quality metrics and acceptance criteria in previous roles.
✨Communicate Clearly and Confidently
Strong communication skills are key for this position. Practice explaining complex technical concepts in simple terms, as you'll need to collaborate with engineering, QA, and product teams. Confidence in your problem-solving approach will also impress your interviewers.
