AI Evaluations Engineer in Manchester

AI Evaluations Engineer in Manchester

Manchester Full-Time 60000 - 80000 £ / year (est.) Hybrid
ConnexAI

At a Glance

  • Tasks: Measure and improve AI systems by designing evaluations and building datasets.
  • Company: Join a leading tech firm focused on cutting-edge AI innovations.
  • Benefits: Competitive salary, flexible work options, and opportunities for professional growth.
  • Other info: Collaborative environment with a focus on real-world applications and career advancement.
  • Why this job: Make a real impact on AI quality and user experience in production.
  • Qualifications: Strong Python skills and experience with ML systems required.

The predicted salary is between 60000 - 80000 £ per year.

This role sits at the centre of how we measure and improve AI systems in production. You’ll define what good performance means across LLMs, ASR, TTS, and full speech-to-speech pipelines, and build the datasets, metrics, and evaluation systems that make AI quality measurable and comparable in the real world. You’ll work closely with engineering and product teams to ensure model changes lead to real improvements in user experience, not just better offline benchmarks.

What you’ll do:

  • Design and run evaluations across LLM, ASR, TTS, and speech-to-speech systems
  • Build real-world datasets and test cases from production behaviour and edge cases
  • Define metrics and scorecards for model and system quality
  • Benchmark internal models against external and frontier systems
  • Evaluate full pipelines (ASR → LLM → TTS), not just individual models
  • Build Python tools to automate evaluation workflows
  • Create internal leaderboards, red-teaming setups, and regression tests
  • Work with engineers and product teams to diagnose system failures
  • Turn vague product goals into measurable evaluation frameworks

What this role is about:

  • Defining and measuring AI quality in production systems
  • Turning real user behaviour into structured evaluation signals
  • Ensuring model changes improve real-world performance
  • Understanding why AI systems fail, not just whether they do

What good looks like:

  • You can translate improved quality into measurable metrics
  • You think in terms of system impact (before vs after), not just accuracy
  • You’re comfortable working across code, data, and production systems
  • You care about real-world behaviour, not just benchmarks

Core skills:

  • Strong Python (scripting, data analysis, tooling)
  • Experience with ML systems, evaluation, or experimentation
  • Understanding of LLMs or speech systems (ASR / TTS)
  • Ability to design test cases and structured datasets
  • Comfortable working with engineers and product teams

Nice to have:

  • Experience with LLM evaluation or benchmarking
  • Exposure to speech or multimodal systems
  • Familiarity with production APIs or ML systems
  • Experience with automated testing or CI-style workflows

AI Evaluations Engineer in Manchester employer: ConnexAI

At ConnexAI, we pride ourselves on being an excellent employer by fostering a collaborative and innovative work culture that empowers our employees to thrive. Our Manchester office offers a dynamic environment where you can engage in meaningful AI projects while benefiting from continuous learning and growth opportunities. Join us to be part of a global team dedicated to shaping the future of language technology, all while enjoying the excitement of working at the forefront of AI advancements.

ConnexAI

Contact Details:

ConnexAI Recruitment Team

StudySmarter Expert Advice🤫

We think this is how you could land AI Evaluations Engineer in Manchester

✨Get Involved in Data Science Meetups

Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like ConnexAI!

✨Show Off Your Projects

Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like AI Evaluations Engineer at ConnexAI.

✨Leverage Professional Networks

Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like ConnexAI.

✨Apply Directly through Our Website

When you find a suitable opening like AI Evaluations Engineer at ConnexAI, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!

We think you need these skills to ace AI Evaluations Engineer in Manchester

Python
Data Analysis
Machine Learning Systems
Evaluation and Experimentation
Understanding of LLMs
ASR (Automatic Speech Recognition)
TTS (Text-to-Speech)

Some tips for your application 🫡

Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!

Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!

Craft a Tailored Cover Letter:For a full-time role at ConnexAI, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.

Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at ConnexAI. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!

How to prepare for a job interview at ConnexAI

✨Brush Up on Your Statistics

For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!

✨Showcase Your Projects

Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!

✨Get Comfortable with Python and R

Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at ConnexAI!

✨Prepare for Case Studies

Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.