At a Glance
- Tasks: Design automated testing strategies for cutting-edge AI systems and validate their performance.
- Company: Join a leading financial services organisation revolutionising AI technology.
- Benefits: Competitive daily rate, hybrid working model, and opportunity to work on innovative projects.
- Other info: 6-month contract with excellent career growth potential in a dynamic environment.
- Why this job: Make an impact in the AI field while ensuring quality and security in financial services.
- Qualifications: Experience in large-scale test automation and AI system validation required.
We’re supporting a financial services organisation building an enterprise AI platform and are looking for an AI QA Engineer to help define how large scale automated testing operates across a RAG-enabled ecosystem and help ensure their AI platforms are accurate, secure and production ready. You’ll be responsible for validating LLM and RAG-based AI systems, building automated testing approaches around AI behaviours and helping define how AI quality, hallucinations, security and observability are measured at scale.
What you’ll be doing:
- Designing automated testing strategies for AI/LLM systems
- Validating RAG pipelines and retrieval accuracy
- Building frameworks to test AI quality, consistency and performance
- Measuring and assessing hallucinations and model behaviour
- Testing AI security boundaries, permissions and access controls
- Supporting observability and reporting through Datadog dashboards
- Working with engineering teams to define AI quality standards
- Testing agentic workflows and integrations including MCP environments
What they’re looking for:
- Strong background in large-scale test automation/QA engineering
- Previous experience testing AI systems, LLMs or GenAI applications
- Experience validating RAG architectures
- Knowledge of hallucination testing/evaluation frameworks
- Exposure to AWS Bedrock and Python
- Understanding of AI security, access controls and governance
- Familiarity with observability tooling (Datadog ideal)
- Bonus: MCP/agent frameworks experience
This is a 6-month contract paying £650 a day (inside IR35/umbrella rate). The client operate a hybrid working model and you will be expected to work 3 days a week in their London-based office.
QA Engineer (AI) - Contract - Financial Services in London employer: Vertus Partners
Join a forward-thinking financial services organisation that prioritises innovation and quality in the rapidly evolving AI landscape. With a strong commitment to employee development, you will have access to cutting-edge projects and collaborative work culture, all while enjoying the flexibility of a hybrid working model in the vibrant city of London. This role not only offers competitive compensation but also the opportunity to contribute to meaningful advancements in AI technology, making it an ideal environment for passionate professionals seeking to make an impact.
StudySmarter Expert Advice🤫
We think this is how you could land QA Engineer (AI) - Contract - Financial Services in London
✨Tip Number 1
Network like a pro! Reach out to folks in the financial services and AI sectors on LinkedIn. Join relevant groups, attend meetups, and don’t be shy about asking for informational interviews. You never know who might have the inside scoop on job openings!
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your automated testing strategies and any projects related to AI systems you've worked on. This is your chance to demonstrate your expertise in validating RAG pipelines and testing AI quality. Make it easy for potential employers to see what you can do!
✨Tip Number 3
Prepare for those interviews! Brush up on your knowledge of AI security, access controls, and observability tools like Datadog. Be ready to discuss your experience with large-scale test automation and how you’ve tackled challenges in previous roles. Confidence is key!
✨Tip Number 4
Apply through our website! We’re always on the lookout for talented QA Engineers like you. By applying directly, you’ll ensure your application gets the attention it deserves. Plus, you’ll be one step closer to joining a team that’s shaping the future of AI in financial services!
We think you need these skills to ace QA Engineer (AI) - Contract - Financial Services in London
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience in large-scale test automation and AI systems. We want to see how your skills align with the role, so don’t be shy about showcasing your relevant projects!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you’re passionate about AI QA engineering and how your background makes you the perfect fit for our team. Keep it engaging and to the point.
Showcase Your Technical Skills:Don’t forget to mention your experience with tools like AWS Bedrock, Python, and Datadog. We’re looking for someone who can hit the ground running, so highlight any relevant technical expertise you have!
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for this exciting opportunity. Let’s get started on this journey together!
How to prepare for a job interview at Vertus Partners
✨Know Your AI Stuff
Make sure you brush up on your knowledge of AI systems, especially LLMs and RAG architectures. Be ready to discuss your previous experiences in testing these technologies and how you've approached challenges like hallucination testing.
✨Showcase Your Automation Skills
Prepare to talk about your experience with large-scale test automation. Have examples ready that demonstrate your ability to design automated testing strategies, particularly for AI/LLM systems. Highlight any frameworks you've built or contributed to.
✨Familiarise with Tools
Get comfortable with the tools mentioned in the job description, especially Datadog for observability. If you have experience with AWS Bedrock or Python, be sure to mention it and how you've used these tools in your past roles.
✨Collaborative Mindset
Since you'll be working closely with engineering teams, emphasise your teamwork skills. Share examples of how you've collaborated to define quality standards or improve testing processes in previous projects.
