Machine Learning Engineer in London

Competitive salary dependent on experience. Company performance-related bonus, healthcare insurance & wellbeing benefits.

About Liv-ex: We offer a multitude of business services covering trading opportunities, data, logistics and various automation technologies; aimed at a diverse group of wine businesses, from ambitious young start-ups to established merchants and traders. Our aim is to make the wine trade more transparent, efficient and safe, for the benefit of our members and the market as a whole. We are hardworking, committed and action oriented, retaining a valued neutrality in the market. Founded in 2000, Liv-ex has grown to serve a growing number of members in the B2B sector, with an ever-expanding range of services. We help our members and other stakeholders to better understand the fine wine market and identify profit opportunities.

Summary Purpose: We are seeking an experienced Machine Learning Engineer to build the technical foundation for our AI-driven wine exchange platform. While our Data Scientists focus on designing and fine-tuning complex models (NLP, Forecasting, Recommendations), your mission is to productionise, scale, and serve these models with high availability and low latency. You will own the MLOps infrastructure on Databricks and AWS, building robust pipelines that process millions of records and serve real‑time predictions to our global trading platform. You will bridge the gap between experimentation and production software engineering, ensuring our systems are reliable, secure, and maintainable.

Responsibilities:

• Productionise ML Pipelines: Engineer robust, scalable data and ML pipelines using PySpark on Databricks to power our Entity Matching and Recommender systems.
• Implement MLOps Best Practices: Design and maintain CI/CD workflows for machine learning, automating model training, testing, and deployment using MLflow and Databricks Asset Bundles.
• Model Serving & Deployment: Deploy models to production using Mosaic AI Model Serving (or similar serverless endpoints), optimising for throughput and low latency.
• Infrastructure Management: Manage our underlying data and ML infrastructure on AWS (S3, Lambda) and Databricks, including Unity Catalog governance and Vector Search indexes.
• Performance Optimisation: Profile and optimize Spark jobs and inference code to reduce cloud costs (DBUs) and improve processing speed.
• Monitoring & Observability: Implement comprehensive monitoring for model drift, data quality, and system health to ensure 99.9% availability.
• Collaboration: Work closely with Data Scientists to take models from “notebook prototype” to “production service,” and with Software Engineers to integrate API endpoints into the core Liv-ex platform.

What We’re Looking For:

• Expert Python Engineer: Production-grade programming skills (typing, testing, modular design) with experience refactoring research code.
• Databricks & Spark: Deep proficiency with PySpark and the Databricks ecosystem (Delta Lake, Unity Catalog, Workflows/Jobs).
• Cloud Native (AWS): Strong experience with AWS core services (S3, IAM, Lambda) and Infrastructure-as-Code principles (Terraform or similar is a plus).
• MLOps & Tools: Hands‑on experience with MLflow (registry, tracking), Docker/Containerization, and CI/CD tools (GitHub Actions, Jenkins, or similar).
• Deployment Patterns: Experience with different serving patterns: Real‑time (REST APIs), Batch inference, and Streaming.
• Vector Search: Familiarity with deploying and scaling vector databases (e.g., Databricks Vector Search, Qdrant, Weaviate, Pinecone) for semantic search applications.
• Model Understanding: Sufficient understanding of NLP and Regressors to debug inference issues, even if you aren’t training the models yourself.
• Educational Background: Bachelor’s degree in Computer Science, Engineering, or a related field.
• Experience: 5+ years in Data Engineering or Machine Learning Engineering.
• “Builder” Mindset: You care deeply about code quality, testing, and system architecture. You prefer automating tasks over manual execution.
• Production Scars: You have broken things in production and learned how to fix them. You understand why “it runs on my laptop” is not enough.

Why Join Liv-ex:

• Own the Stack: You will be the primary engineer defining our MLOps architecture on a modern Databricks/AWS stack.
• High Impact: Your work directly enables our new AI products to function at scale.
• Modern Tooling: Work with the latest features in the Databricks ecosystem (Mosaic AI, Serverless, Unity Catalog).