b Overview /b p In this role you will design and implement automated QA strategies for GenAI features, ensuring reliable, accurate, and trustworthy AI-powered workflows. You will work across evaluations, APIs, UIs, and performance tests to validate non-deterministic AI systems. You’ll shape evaluation frameworks for GenAI metrics like accuracy and safety, and help scale testing within a cross-functional AI-focused team. This is a hands-on, impact-driven position at a leading AI-infused planning platform that serves Fortune 50 customers. You will translate complex AI behavior into robust test coverage and actionable insights. /p b Responsibilities /b ul li Develop and execute end-to-end automated test coverage across evaluations, APIs, UIs, and performance tests using Python /li li Implement testing strategies for GenAI features, including conversational AI and agentic systems /li li Maintain automated test suites for prompt testing and regression tests to detect model behavior changes /li li Support evaluation frameworks measuring GenAI performance across accuracy, relevance, safety, consistency, and latency /li li Curate high-quality test datasets and golden examples representing diverse user scenarios /li li Assist in adversarial testing to identify hallucinations or biases and support production alerting integration /li li Utilise internal tools and testing frameworks to help developers validate GenAI code and assist with UAT /li li Document testing procedures and known issues for automated coverage workstream /li /ul b Key requirements /b ul li Solid QA or test engineering experience with exposure to AI/ML systems /li li Hands-on Python coding experience /li li Knowledge of GenAI technologies including LLMs, prompt engineering, and AI application patterns /li li Experience with test automation frameworks and scripting (Python, Pytest, API, UI Testing) /li li Understanding of functional, integration, regression, performance, and security testing /li li Ability to execute test cases for non-deterministic systems /li li Experience with API testing tools (Postman, REST Assured) and backend testing /li li Familiarity with CI/CD pipelines and automated testing integration /li /ul ul li Python /li li Pytest /li li API testing /li li UI testing /li li Postman /li li REST Assured /li /ul
SDET - AI employer: Anaplan
Anaplan is an exceptional employer that fosters a Winning Culture, where innovation and collaboration thrive. Located in London, we offer a dynamic work environment that champions diversity and provides ample opportunities for professional growth, ensuring that every employee feels valued and inspired to contribute to our mission of transforming business decision-making. With a commitment to inclusivity and a focus on celebrating achievements, Anaplan is the perfect place for those looking to make a meaningful impact while advancing their careers.
