Location: Hybrid / London 3 days
Length: Initially 6 months
Role Overview
We are seeking an experienced AI Engineer to establish and scale quality assurance, validation, and evaluation frameworks for enterprise AI and analytics solutions. The role will focus on ensuring the accuracy, safety, reliability, and performance of semantic models, knowledge graphs, derived data assets, and AI-powered query systems.
Working closely with AI Engineers, Data Engineers, Analytics Engineers, and Product teams, you will build automated testing and evaluation capabilities that enable continuous validation of AI-generated outputs across SQL, Cypher, semantic querying, and hybrid retrieval systems.
Key Responsibilities
Build and maintain golden-question datasets to benchmark AI solution quality and consistency.
Develop automated evaluation frameworks for semantic mappings, measures, derived assets, SQL/Cypher generation, graph retrieval, hybrid search, and multi-intent question answering.
Design and execute validation tests covering data equivalence, join integrity, graph projection accuracy, query safety, access controls, regression, performance, and concurrency.
Create contract, integration, and end-to-end test suites across mocked services, ClickHouse, BigQuery, and Neo4j environments.
Embed testing and evaluation processes into CI/CD pipelines to support continuous delivery and rapid feedback.
Establish repeatable validation across changes to data, semantic models, knowledge graphs, prompts, configurations, and derived assets.
Monitor quality metrics, identify regressions, and work with engineering teams to resolve defects and improve system reliability.
Support AI governance by ensuring outputs are accurate, secure, auditable, and compliant with organisational standards.
Strong experience in software testing, QA engineering, or platform validation.
Experience developing automated testing frameworks using Python.
Understanding of AI, LLM, RAG, semantic layer, or knowledge graph architectures.
Experience validating SQL-based analytical systems and complex data transformations.
Knowledge of graph databases and query languages such as Neo4j and Cypher.
Familiarity with modern data platforms including ClickHouse, BigQuery, Snowflake, Databricks, or similar environments.
Experience creating regression, integration, contract, and end-to-end test suites.
Understanding of CI/CD tooling such as GitHub Actions, Azure DevOps, GitLab CI, or Jenkins.
Strong analytical and problem-solving capabilities.
AI Engineer in London employer: Experis Austria
Experis Austria is an exceptional employer, offering a dynamic work culture that fosters collaboration and innovation in the heart of London. With a focus on employee growth, we provide ample opportunities for professional development and skill enhancement, ensuring that our team members thrive in their careers while contributing to impactful projects in the oil and gas sector. Our hybrid work model promotes a healthy work-life balance, making it an ideal environment for those seeking meaningful and rewarding employment.
