At a Glance
- Tasks: Define and implement evaluation frameworks for AI-integrated services and develop automated test pipelines.
- Company: Global organisation leading in AI enablement and platform engineering.
- Benefits: Competitive daily rate, hybrid work model, and exposure to cutting-edge AI challenges.
- Other info: Collaborative environment with opportunities for career growth in AI engineering.
- Why this job: Join a transformative AI initiative and influence long-term quality and governance standards.
- Qualifications: Strong Python skills, experience with LLM evaluation, and understanding of agentic AI systems.
A global organisation is building a centralised AI enablement and platform engineering function focused on delivering scalable, secure, and governed AI capabilities across the enterprise. This role sits within a programme delivering enterprise-grade agentic AI infrastructure, including internal AI assistants, retrieval and search services, extensibility frameworks, and governance tooling.
The programme is focused on delivering a production-grade internal agentic AI platform, including:
- Development of an enterprise AI assistant capable of reasoning, planning, and tool orchestration
- Operation of enterprise retrieval, search, and grounding services for approved data sources
- Delivery of a secure internal gateway layer providing discovery, observability, policy enforcement, and lifecycle management for AI-integrated services
- Design and development of AI-integrated services and reusable capabilities that safely expose internal and third-party systems to AI agents
- Establishment of evaluation, governance, and quality-control frameworks to support scalable and compliant deployment of AI capabilities
Key Responsibilities:
- Define and implement evaluation frameworks covering correctness, safety, reliability, and regression impact for AI-integrated services
- Develop and maintain automated test pipelines for agentic workflows, including tool orchestration and multi-step execution paths
- Identify, evaluate, and mitigate AI system failure modes such as hallucinations, invalid inputs, latency issues, and inappropriate tool usage
- Produce testing and governance evidence required for internal approval and operational processes
- Collaborate closely with ML Engineers and platform teams to embed testability and evaluation capabilities into AI services
- Contribute to the long-term quality assurance and governance strategy for enterprise-wide AI platform adoption
Essential Skills and Experience:
- Strong Python development experience, particularly for automation and test frameworks
- Experience with LLM and RAG evaluation tooling, frameworks, or custom evaluation pipelines
- Expertise in automated testing across unit, integration, and regression testing environments
- Good understanding of agentic AI systems, associated risks, and operational failure modes
- Ability to assess technical solutions against governance, audit, and security requirements
- Experience working within regulated or highly governed engineering environments
What’s on Offer:
- Opportunity to contribute to large-scale enterprise AI transformation initiatives
- Exposure to cutting-edge AI platform engineering and governance challenges
- Collaborative environment working alongside platform engineers, ML specialists, and architecture teams
- Influence over the development of long-term AI quality and governance standards
- Opportunity to shape scalable AI engineering practices within a complex enterprise environment
AI Platform QA Engineer for Enterprise AI Governance employer: Trades Workforce Solutions
Join a leading global organisation at the forefront of AI transformation, where you will play a pivotal role in shaping enterprise-grade AI capabilities. With a collaborative work culture that values innovation and quality, you'll have the opportunity to work alongside top-tier professionals in a hybrid environment in Central London, fostering both personal and professional growth. Enjoy competitive rates and the chance to influence long-term governance standards in a dynamic and supportive setting.
Contact Details:
Trades Workforce Solutions Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land AI Platform QA Engineer for Enterprise AI Governance
✨Tip Number 1
Network like a pro! Reach out to folks in the AI and engineering space on LinkedIn or at meetups. You never know who might have the inside scoop on job openings or can put in a good word for you.
✨Tip Number 2
Show off your skills! Create a portfolio or GitHub repository showcasing your Python projects, especially those related to automation and testing. This gives potential employers a taste of what you can do.
✨Tip Number 3
Prepare for interviews by brushing up on common QA scenarios and AI system challenges. Think about how you would tackle issues like hallucinations or latency problems, and be ready to discuss your thought process.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive about their job search!
We think you need these skills to ace AI Platform QA Engineer for Enterprise AI Governance
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the role of AI Platform QA Engineer. Highlight your Python development experience and any relevant automated testing skills. We want to see how your background aligns with our needs!
Showcase Your Experience:In your cover letter, showcase your experience with LLM and RAG evaluation tooling. Share specific examples of how you've tackled challenges in automated testing or governance frameworks. This helps us see your problem-solving skills in action!
Be Clear and Concise:When writing your application, keep it clear and concise. Use bullet points where possible to make it easy for us to read. We appreciate straightforward communication that gets to the point!
Apply Through Our Website:Don’t forget to apply through our website! It’s the best way for us to receive your application and ensures you’re considered for the role. We can’t wait to hear from you!
How to prepare for a job interview at Trades Workforce Solutions
✨Know Your AI Stuff
Make sure you brush up on your knowledge of agentic AI systems and their associated risks. Be ready to discuss how you've tackled issues like hallucinations or latency in past projects. This will show that you understand the technical challenges and can contribute effectively.
✨Showcase Your Python Skills
Since strong Python development experience is essential, prepare to demonstrate your automation and testing frameworks. Bring examples of your work or even a small project that highlights your coding skills. This will give the interviewers confidence in your technical abilities.
✨Understand the Governance Landscape
Familiarise yourself with governance, audit, and security requirements in regulated environments. Be prepared to discuss how you've assessed technical solutions against these standards in previous roles. This will show that you can navigate the complexities of enterprise AI governance.
✨Collaborate and Communicate
This role involves working closely with ML Engineers and platform teams, so highlight your collaboration skills. Share examples of how you've successfully worked in teams to embed testability and evaluation capabilities into AI services. Good communication can set you apart!
