Senior QAT (Quality Assurance Tester)
Generative AI/RAG Platform (UK Public Sector)
Salary: £65'000 per annum
Location: Newcastle, one day a week in the office
Security clearance: SC eligible, no SC needed
Overview
We are seeking an experienced Senior Quality Assurance Tester to lead testing strategy and execution for a Generative AI platform for a UK government department. This role goes beyond traditional functional QA. It includes evaluating AI output quality (answer accuracy, retrieval relevance, hallucination rate) alongside conventional API, integration and non-functional testing.
Key Responsibilities
Test Strategy & Planning
Define and own the test strategy across functional, non-functional and AI-specific quality dimensions (answer quality, retrieval accuracy, hallucination rate), covering RAG pipelines, prompt templates and agentic workflows.
AI Output Evaluation
Design and run structured evaluations of LLM outputs, working with engineering to track answer quality and hallucination rate over time, and flag regressions before release.
API & Integration Testing
Carry out manual and automated API testing (for example with REST Assured or similar tools) across the application logic that connects RAG pipelines to Legacy and multi-cloud systems.
Automation & CI/CD Integration
Build and maintain automated test suites integrated into CI/CD pipelines (for example GitLab or Jenkins), enabling shift-left testing across the delivery life cycle.
Non-Functional Testing
Carry out performance testing (for example k6, Gatling or JMeter) and accessibility testing, so the platform is robust, responsive and inclusive under real government usage conditions.
Responsible AI Testing
Test that guardrails and human-in-the-loop checkpoints defined by policy and technical leads work correctly and consistently. This is a verification role, not a policy-setting one.
Agile Collaboration
Work within agile delivery teams, engaging with Product, Engineering and Policy stakeholders to embed testing throughout the sprint rather than treating it as a final gate.
Governance & Standards Compliance
Make sure testing and release evidence aligns with the GOV.UK Service Standard and departmental governance, supporting evidence-gathering for...
Senior QAT (Quality Assurance Tester) in Newcastle upon Tyne employer: scrumconnect ltd
Scrumconnect Consulting is an exceptional employer that prioritises the growth and development of its employees, offering a supportive work culture in Newcastle. With a commitment to diversity and inclusion, as well as being a Disability Confident employer, we provide meaningful opportunities for career advancement and continuous learning, making it an ideal place for passionate individuals to thrive in their recruitment careers.
