Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Test and validate cutting-edge AI applications using modern technologies.
- Company: Join a forward-thinking tech company focused on AI innovation.
- Benefits: Competitive salary, flexible work options, and opportunities for professional growth.
- Other info: Collaborative environment with strong career advancement potential.
- Why this job: Be at the forefront of AI technology and ensure its quality and reliability.
- Qualifications: Experience in software testing, Python automation, and familiarity with AI systems.
The predicted salary is between 50000 - 70000 £ per year.
We are seeking a skilled and detail-oriented AI Test Engineer to test, validate, and assure the quality of advanced AI-powered applications built using modern LLM, RAG, and Agentic AI architectures. The ideal candidate will have strong hands-on experience in software testing, Python-based test automation, API testing, and practical exposure to AI systems built with frameworks such as LangChain, LangGraph, and OpenAI ecosystems. You will work closely with AI Engineers and Developers to ensure AI applications are reliable, accurate, secure, and production-ready. The role will focus on validating AI behaviours, testing Retrieval-Augmented Generation (RAG) pipelines, evaluating AI agents, verifying embeddings and search quality, and ensuring end-to-end quality across AI workflows and integrations.
Key Responsibilities
- AI Application Testing
- Test AI applications built using Large Language Models (LLMs).
- Validate the behaviour, reliability, and output quality of AI-powered applications and APIs.
- Design and execute functional, regression, integration, and end-to-end test cases for AI systems.
- Work closely with AI Engineers to identify defects, edge cases, and system weaknesses early in the development lifecycle.
- LLM Testing & Validation
- Test LLM-based features for response quality, consistency, accuracy, and robustness.
- Validate prompt behaviour and test prompt variations across different business scenarios.
- Perform negative testing, boundary testing, and exception testing for AI outputs.
- Support evaluation of hallucination risk, response relevance, and output stability.
- RAG & Knowledge System Testing
- Test Retrieval-Augmented Generation (RAG) pipelines end to end.
- Validate document ingestion, chunking, embedding generation, retrieval quality, and final answer relevance.
- Verify semantic search behaviour across vector databases and knowledge sources.
- Work with AI Engineers to identify retrieval failures, context mismatch issues, and ranking problems.
- Agentic AI & Workflow Testing
- Test AI Agents built using LangChain and LangGraph.
- Validate agent workflows, tool usage, state transitions, memory behaviour, and multi-step execution paths.
- Test decision points, fallback handling, retry logic, and workflow orchestration behaviour.
- Ensure agentic systems behave safely and predictably under expected and unexpected conditions.
- API, Integration & System Testing
- Test AI-driven microservices, APIs, and enterprise integrations.
- Validate data flow between AI components, external tools, vector databases, and business applications.
- Perform integration testing across AI services, backend components, and user-facing workflows.
- Verify system behaviour in production-like environments.
- Quality Assurance & Automation
- Develop automated test cases and reusable test suites for AI applications.
- Build test data, test scenarios, and validation frameworks for structured and unstructured AI use cases.
- Support continuous testing within development and release cycles.
- Track, document, and communicate defects clearly to engineering and product teams.
- AI Safety, Performance & Observability
- Test AI systems for reliability, latency, failure handling, and performance under load.
- Validate safety controls, content filtering, output guardrails, and error-handling behaviour.
- Support monitoring, evaluation, and observability of AI systems in development and production.
- Contribute to quality benchmarks, acceptance criteria, and release readiness reviews for AI features.
- Engineering Collaboration
- Collaborate closely with AI Developers, Product teams, DevOps, and Data teams.
- Participate in requirement reviews, solution discussions, and test planning sessions.
- Ensure testing is aligned with business requirements and technical design.
- Promote high testing standards and a quality-first engineering approach across AI delivery.
Required Technical Skills (High-Level)
- Software Testing / QA Engineering
- Python
- API Testing
- Test Automation
- Large Language Models (LLMs)
- Retrieval-Augmented Generation (RAG)
- LangChain
- LangGraph
- AI Agent Testing
- OpenAI API Integration
- Embeddings & Vector Databases
- Prompt Testing / AI Output Validation
- Integration Testing
- Regression Testing
- Specification-Driven Testing
Additional Preferred Skills
- Experience with FAISS / Chroma / Pinecone / Weaviate
- Experience testing FastAPI / Flask based services
- Understanding of Transformer-based AI systems
- Experience with Azure / AWS / GCP
- Knowledge of CI/CD pipelines for automated testing
- Experience with LLM evaluation frameworks
- Exposure to AI safety testing, red-teaming, or guardrail validation
- Understanding of performance, load, and reliability testing for AI systems
AI Software/QA Test Engineer in England employer: Diagonal Matrix Ltd
Contact Detail:
Diagonal Matrix Ltd Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land AI Software/QA Test Engineer in England
✨Tip Number 1
Network like a pro! Reach out to folks in the AI and software testing community on LinkedIn or at meetups. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your testing projects, especially those involving LLMs or RAG. This gives potential employers a taste of what you can do and sets you apart from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on common testing scenarios related to AI applications. Think about how you'd validate AI behaviours or test API integrations, and be ready to discuss your thought process.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive about their job search!
We think you need these skills to ace AI Software/QA Test Engineer in England
Some tips for your application 🫡
Tailor Your Application: Make sure to customise your CV and cover letter to highlight your experience with AI testing, Python automation, and any relevant frameworks like LangChain or OpenAI. We want to see how your skills align with the role!
Showcase Your Projects: If you've worked on any AI projects or testing scenarios, don’t hold back! Share specific examples that demonstrate your hands-on experience and problem-solving skills. This helps us see your practical knowledge in action.
Be Clear and Concise: When writing your application, keep it straightforward. Use clear language and bullet points where possible to make it easy for us to read. We appreciate a well-structured application that gets straight to the point!
Apply Through Our Website: We encourage you to submit your application directly through our website. It’s the best way for us to receive your details and ensures you’re considered for the role. Plus, it’s super easy!
How to prepare for a job interview at Diagonal Matrix Ltd
✨Know Your AI Stuff
Make sure you brush up on your knowledge of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Agentic AI. Familiarise yourself with frameworks like LangChain and LangGraph, as well as the OpenAI ecosystem. Being able to discuss these topics confidently will show that you're serious about the role.
✨Showcase Your Testing Skills
Prepare to talk about your hands-on experience with software testing and Python-based test automation. Be ready to share specific examples of how you've designed and executed test cases, especially for AI applications. Highlight any experience you have with API testing and integration testing, as this will be crucial for the job.
✨Understand the AI Workflow
Get a good grasp of the end-to-end AI workflow, from validating AI behaviours to testing RAG pipelines. Think about how you would approach testing various components, such as embeddings and search quality. This understanding will help you answer questions about how you would ensure the reliability and accuracy of AI applications.
✨Collaborate Like a Pro
Since the role involves working closely with AI Engineers and Developers, be prepared to discuss your collaboration skills. Share examples of how you've worked in teams to identify defects and improve testing processes. Emphasising your ability to communicate effectively with technical teams will set you apart.