ML/AI Test Analyst

Who are Valcon: Valcon is a north-western European consulting, technology and data company based in the UK, the Netherlands, Denmark, Sweden, and Croatia. Valcon is private equity backed and in recent years has bought together some of the most exciting companies from these countries to form one of the brightest and most exciting consultancies around. We’re one of the fastest growing companies across Europe and have no plans to stop.

What do we do: Our mission is to combine premium consulting with deep technology and data knowledge to add value to our clients.

Overview of Technology Practice: We’re at the forefront of empowering the future of work, helping organisations of all sizes achieve their goals. For large enterprises, we eliminate manual labour through automation and intelligent tools, boosting knowledge worker productivity. Across the board, we build high-quality, performant and user-friendly solutions that leverage the latest technologies to deliver exceptional results for both scale-ups and large companies. Our Technology Practice has 5 capabilities:

• IT Architecture
• Cloud Engineering
• Software Engineering
• Intelligent Process Automation
• Quality Management

We are seeking a detail-oriented and technically skilled AI/ML Test Engineer to ensure the quality, reliability, and robustness of AI systems. This role involves designing and executing comprehensive test strategies for machine learning models and data pipelines, identifying issues across APIs, datasets, and model outputs, and driving quality throughout the AI/ML lifecycle. The ideal candidate will combine strong problem-solving abilities with a deep understanding of data science, software testing, and automation frameworks.

Key Responsibilities:

• Diagnose and debug failures in AI systems, including technical bugs, data quality issues, or model limitations.
• Design and implement creative test scenarios that push the limits of AI/ML models.
• Assess and articulate the impact of model behavior on end-user experience and business objectives.
• Create, manage, and validate large and diverse test datasets, including synthetic and adversarial data.
• Automate model and API testing using frameworks such as pytest, unittest, or behave.
• Collaborate with data scientists and engineers to understand ML pipeline components and potential failure points.
• Evaluate model robustness against noisy, manipulated, or adversarial inputs.
• Perform load and performance testing on deployed ML models to ensure production readiness.
• Conduct UAT (User Acceptance Testing) and manage defect triaging and resolution.
• Create and maintain detailed test plans, test cases, and test scripts.
• Report test coverage, defects, and progress via tools such as Jira and test dashboards.
• Contribute to test strategy and drive improvements in QA processes for AI/ML products.

Required Skills & Experience:

• AI/ML and Data Testing: Proficient in identifying data issues, distribution shifts, and dataset bias. Understanding of various data formats (CSV, JSON, Parquet) used in ML workflows. Familiarity with ML frameworks: TensorFlow, PyTorch, scikit-learn. Able to validate outputs against ground truth and expected behavior.
• Programming & Tools: Strong Python skills, especially for test automation, data analysis (Pandas, NumPy), and API testing. Familiar with Java for integration and component testing (optional but beneficial). Experience with REST API testing tools: Postman, RestAssured. Understanding of API documentation standards (Swagger/OpenAPI) and HTTP protocol essentials.
• Testing & QA Processes: Skilled in test case design, scripting, execution, and defect lifecycle. Experience using Python-based test frameworks (pytest, unittest, etc.). Exposure to test management tools like Jira and structured reporting. Experience in leading QA efforts and coordinating test activities.
• Cloud & Deployment: Familiarity with cloud-based ML services: AWS SageMaker, Azure ML, GCP Vertex AI, etc. Understanding of deployment pipelines, serverless components (e.g., Lambda, Step Functions).
• Data Science Collaboration: Exposure to Jupyter Notebooks, visualization libraries (e.g., Matplotlib, Seaborn). Knowledge of synthetic data generation, data augmentation, and perturbation techniques.

Preferred Qualifications:

• Bachelor’s or Master’s in Computer Science, Data Science, Software Engineering, or related field.
• 3+ years in QA/testing roles, with at least 1–2 years focused on AI/ML systems.
• Certifications in ML, cloud services, or test automation frameworks are a plus.

Soft Skills:

• Strong analytical and communication skills.
• Ability to translate technical findings into business insights.
• Self-starter with an innovative mindset for tackling complex testing challenges.