At a Glance
- Tasks: Define evaluation frameworks and develop automated test pipelines for AI services.
- Company: Global organisation leading in AI enablement and platform engineering.
- Benefits: Competitive daily rate, hybrid work model, and exposure to cutting-edge AI challenges.
- Other info: Collaborative culture with opportunities for professional growth and influence.
- Why this job: Join a transformative AI initiative and shape the future of enterprise AI governance.
- Qualifications: Strong Python skills and experience with automated testing in regulated environments.
A global organisation is building a centralised AI enablement and platform engineering function focused on delivering scalable, secure, and governed AI capabilities across the enterprise. This role sits within a programme delivering enterprise-grade agentic AI infrastructure, including internal AI assistants, retrieval and search services, extensibility frameworks, and governance tooling.
The programme is focused on delivering a production-grade internal agentic AI platform, including:
- Development of an enterprise AI assistant capable of reasoning, planning, and tool orchestration
- Operation of enterprise retrieval, search, and grounding services for approved data sources
- Delivery of a secure internal gateway layer providing discovery, observability, policy enforcement, and lifecycle management for AI-integrated services
- Design and development of AI-integrated services and reusable capabilities that safely expose internal and third-party systems to AI agents
- Establishment of evaluation, governance, and quality-control frameworks to support scalable and compliant deployment of AI capabilities
The programme currently follows a centrally delivered model while evolving towards a federated contribution approach over time.
Key Responsibilities
- Define and implement evaluation frameworks covering correctness, safety, reliability, and regression impact for AI-integrated services
- Develop and maintain automated test pipelines for agentic workflows, including tool orchestration and multi-step execution paths
- Identify, evaluate, and mitigate AI system failure modes such as hallucinations, invalid inputs, latency issues, and inappropriate tool usage
- Produce testing and governance evidence required for internal approval and operational processes
- Collaborate closely with ML Engineers and platform teams to embed testability and evaluation capabilities into AI services
- Contribute to the long-term quality assurance and governance strategy for enterprise-wide AI platform adoption
Essential Skills and Experience
- Strong Python development experience, particularly for automation and test frameworks
- Experience with LLM and RAG evaluation tooling, frameworks, or custom evaluation pipelines
- Expertise in automated testing across unit, integration, and regression testing environments
- Good understanding of agentic AI systems, associated risks, and operational failure modes
- Ability to assess technical solutions against governance, audit, and security requirements
- Experience working within regulated or highly governed engineering environments
What’s on Offer
- Opportunity to contribute to large-scale enterprise AI transformation initiatives
- Exposure to cutting-edge AI platform engineering and governance challenges
- Collaborative environment working alongside platform engineers, ML specialists, and architecture teams
- Influence over the development of long-term AI quality and governance standards
- Opportunity to shape scalable AI engineering practices within a complex enterprise environment
AI Platform Quality Engineer - EnterpriseScale & Governance employer: Broughton Group
Join a leading global organisation at the forefront of AI transformation, where you will play a pivotal role in shaping enterprise-grade AI capabilities. With a collaborative work culture in Central London, you will benefit from exposure to cutting-edge technology, opportunities for professional growth, and the chance to influence long-term quality and governance standards in a dynamic environment. Embrace the unique advantage of working in a hybrid model that promotes work-life balance while contributing to meaningful projects that drive innovation.
StudySmarter Expert Advice🤫
We think this is how you could land AI Platform Quality Engineer - EnterpriseScale & Governance
✨Tip Number 1
Network like a pro! Reach out to folks in the AI and engineering space on LinkedIn or at meetups. We all know that sometimes it’s not just what you know, but who you know that can get you in the door.
✨Tip Number 2
Prepare for those interviews by brushing up on your Python skills and understanding of AI systems. We recommend doing mock interviews with friends or using online platforms to get comfortable with common questions related to quality engineering.
✨Tip Number 3
Showcase your projects! If you've worked on any AI-related projects or have experience with automated testing frameworks, make sure to highlight them during interviews. We want to see your hands-on experience and how you tackle real-world problems.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love seeing candidates who are proactive about their job search!
We think you need these skills to ace AI Platform Quality Engineer - EnterpriseScale & Governance
Some tips for your application 🫡
Tailor Your CV:Make sure your CV reflects the skills and experiences that match the job description. Highlight your Python development experience and any work with AI systems, as these are key for us.
Craft a Compelling Cover Letter:Use your cover letter to tell us why you're passionate about AI and quality engineering. Share specific examples of your past work that align with our goals in building scalable and secure AI capabilities.
Showcase Your Testing Expertise:Since this role involves automated testing and evaluation frameworks, be sure to detail your experience in these areas. Mention any tools or frameworks you've used that relate to LLM and RAG evaluation.
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for this exciting opportunity in AI platform engineering.
How to prepare for a job interview at Broughton Group
✨Know Your AI Stuff
Make sure you brush up on your knowledge of agentic AI systems and their associated risks. Be ready to discuss how you've tackled issues like hallucinations or latency in past projects. This shows you're not just familiar with the theory but have practical experience too.
✨Showcase Your Python Skills
Since strong Python development experience is essential, prepare to demonstrate your skills. Bring examples of automation and test frameworks you've built or worked with. If you can, share specific challenges you faced and how you overcame them using Python.
✨Understand Evaluation Frameworks
Familiarise yourself with evaluation frameworks for AI-integrated services. Be prepared to discuss how you would define and implement these frameworks, focusing on correctness, safety, and reliability. This will show that you understand the importance of quality assurance in AI.
✨Collaborative Mindset
This role involves working closely with ML Engineers and platform teams, so highlight your teamwork skills. Share examples of how you've collaborated in the past, especially in regulated environments. Emphasising your ability to embed testability into AI services will make you stand out.
