At a Glance
- Tasks: Design automated testing strategies for cutting-edge AI systems and validate their performance.
- Company: Join a leading financial services organisation revolutionising AI technology.
- Benefits: Competitive daily rate, hybrid working model, and opportunity to work on innovative projects.
- Other info: 6-month contract with potential for growth in a dynamic environment.
- Why this job: Make an impact in the AI field while ensuring quality and security in financial services.
- Qualifications: Experience in large-scale test automation and familiarity with AI systems and RAG architectures.
The predicted salary is between 156000 - 156000 € per year.
We’re supporting a financial services organisation building an enterprise AI platform and are looking for an AI QA Engineer to help define how large scale automated testing operates across a RAG-enabled ecosystem and help ensure their AI platforms are accurate, secure and production ready.
You’ll be responsible for validating LLM and RAG-based AI systems, building automated testing approaches around AI behaviours and helping define how AI quality, hallucinations, security and observability are measured at scale.
What you’ll be doing:
- Designing automated testing strategies for AI/LLM systems
- Validating RAG pipelines and retrieval accuracy
- Building frameworks to test AI quality, consistency and performance
- Measuring and assessing hallucinations and model behaviour
- Testing AI security boundaries, permissions and access controls
- Supporting observability and reporting through Datadog dashboards
- Working with engineering teams to define AI quality standards
- Testing agentic workflows and integrations including MCP environments
What they’re looking for:
- Strong background in large-scale test automation/QA engineering
- Previous experience testing AI systems, LLMs or GenAI applications
- Experience validating RAG architectures
- Knowledge of hallucination testing/evaluation frameworks
- Exposure to AWS Bedrock and Python
- Understanding of AI security, access controls and governance
- Familiarity with observability tooling (Datadog ideal)
- Bonus: MCP/agent frameworks experience
This is a 6-month contract paying £650 a day (inside IR35/umbrella rate). The client operate a hybrid working model and you will be expected to work 3 days a week in their London-based office.
QA Engineer (AI) - Contract - Financial Services employer: Vertus Partners
Join a forward-thinking financial services organisation that prioritises innovation and quality in the rapidly evolving AI landscape. With a strong commitment to employee development, you will have access to cutting-edge projects and collaborative work culture, all while enjoying the flexibility of a hybrid working model in the vibrant city of London. This role not only offers competitive compensation but also the opportunity to contribute to meaningful advancements in AI technology, making it an ideal environment for passionate professionals seeking to make an impact.
StudySmarter Expert Advice🤫
We think this is how you could land QA Engineer (AI) - Contract - Financial Services
✨Tip Number 1
Network like a pro! Reach out to folks in the financial services and AI sectors on LinkedIn. Join relevant groups and engage in discussions. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your previous work with AI systems, especially any automated testing strategies you've designed. This will give potential employers a clear view of what you can bring to the table.
✨Tip Number 3
Prepare for interviews by brushing up on common QA scenarios related to AI and LLMs. Be ready to discuss your experience with RAG architectures and how you’ve tackled challenges like hallucination testing. Confidence is key!
✨Tip Number 4
Don’t forget to apply through our website! We’re always looking for talented individuals like you. Plus, it’s a great way to ensure your application gets the attention it deserves.
We think you need these skills to ace QA Engineer (AI) - Contract - Financial Services
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience with large-scale test automation and AI systems. We want to see how your skills align with the job description, so don’t be shy about showcasing your relevant projects!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you’re passionate about AI QA engineering and how your background makes you a perfect fit for our team. Keep it engaging and relevant to the role.
Showcase Your Technical Skills:Don’t forget to mention your experience with tools like AWS Bedrock, Python, and Datadog. We’re looking for someone who can hit the ground running, so highlight any specific projects or achievements that demonstrate your expertise.
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for this exciting opportunity. Plus, it’s super easy!
How to prepare for a job interview at Vertus Partners
✨Know Your AI Inside Out
Make sure you brush up on your knowledge of AI systems, especially LLMs and RAG architectures. Be ready to discuss specific testing strategies you've used in the past and how they can apply to the role. This shows you're not just familiar with the concepts but have practical experience.
✨Showcase Your Automation Skills
Prepare examples of automated testing frameworks you've built or worked with. Highlight your experience with Python and any relevant tools like Datadog. Being able to articulate your approach to designing automated tests for AI behaviours will set you apart from other candidates.
✨Understand Security and Governance
Since the role involves testing AI security boundaries, make sure you can discuss access controls and governance in detail. Bring up any previous experiences where you had to ensure compliance and security in your testing processes. This will demonstrate your awareness of the critical aspects of AI quality.
✨Be Ready for Technical Questions
Expect technical questions that dive deep into your understanding of hallucination testing and evaluation frameworks. Prepare to explain how you would measure and assess model behaviour. Practising these responses will help you feel more confident during the interview.
