Staff Data Engineer, AI Evaluation in England, London

At Wayve we're committed to creating a diverse, fair and respectful culture that is inclusive of everyone based on their unique skills and perspectives. About Us Founded in 2017, Wayve is the leading developer of Embodied AI technology. Our advanced AI software and foundation models enable vehicles to perceive, understand, and navigate any complex environment, enhancing the usability and safety of automated driving systems. Our vision is to create autonomy that propels the world forward. Our intelligent, mapless, and hardware-agnostic AI products are designed for automakers, accelerating the transition from assisted to automated driving. In our fast-paced environment big problems ignite us—we embrace uncertainty, leaning into complex challenges to unlock groundbreaking solutions. We aim high and stay humble in our pursuit of excellence, constantly learning and evolving as we pave the way for a smarter, safer future. At Wayve, your contributions matter. We value diversity, embrace new perspectives, and foster an inclusive work environment; we back each other to deliver impact.

Impact expected Wayve's machine learning-first approach relies on high-quality, well-structured data. The Evaluation Workflows and Measurement teams build tools and pipelines that power model evaluation at scale. As we scale our evaluation approaches and tooling, we need to process massive volumes of test data efficiently and reliably. This Data Engineer will be embedded in the AI Evaluation division to ensure our evaluation and analytics pipelines are robust, performant, and future-proof. Their work will strengthen our data foundations for fast decision-making, accelerate the availability of large-scale image and video analytics, and help us rapidly integrate and leverage data from external partners - enabling faster iteration across both offline and on-road evaluation.

Challenges you will own:

• Build scalable and reliable data and analytics pipelines to process and enrich over 1 million hours of driving video data annually and supply mission-critical data to stakeholders across the business.
• Unlock rapid insights by architecting and optimising analytics pipelines that drive company wide development and decision-making.
• Collaborate across functions - including research engineers, simulation experts, robotics engineers, data scientists and safety drivers - to deliver and visualise enriched data.
• Improve pipeline observability, validation, and fault tolerance for production-grade robustness.
• Reduce tech debt and simplify orchestration across Flyte, Databricks, and Azure-based infrastructure.

Example Projects:

• Design and optimise distributed data pipelines to handle large-scale video and image data processing.
• Re-design and optimise existing analytics pipelines.
• Collaborate with the data platform team to integrate pipelines with Databricks for governance and compliance - and unlock massive scale for offline evaluation from third party datasets.
• Shape evaluation data to support future use cases like Retrieval-Augmented Generation (RAG) and natural language analytics.

What we are looking for in our candidate:

• Essential
• Proficiency in Python and SQL, with experience in frameworks like Pandas, PySpark, and NumPy for large-scale data processing.
• Expertise in debugging and optimising distributed systems with a focus on scalability and reliability.
• Proven ability to design and implement scalable, fault-tolerant ETL pipelines with minimal manual intervention.
• Knowledge of data modelling best practices, including the medallion architecture or comparable frameworks.
• Experience in workflow orchestration using Flyte, dbt, Airflow, or Prefect.
• Strong understanding of unit, integration, and data validation testing using tools like Pytest or Great Expectations.
• Familiarity with cloud infrastructure (preferably Azure) for managing pipelines and storage.
• Ability to collaborate closely with stakeholders to understand requirements and shape data pipelines to meet user needs effectively.

• Desirable
• 5+ years of experience in a data engineering or similar role.
• Experience with Docker, Kubernetes, Databricks.
• Familiarity with shaping data for AI/LLM-based systems.

This is a full-time role based in our office in London. At Wayve we want the best of all worlds so we operate a hybrid working policy that combines time together in our offices and workshops to fuel innovation, culture, relationships and learning, and time spent working from home. We operate core working hours so you can determine the schedule that works best for you and your team. We understand that everyone has a unique set of skills and experiences and that not everyone will meet all of the requirements listed above. If you're passionate about self-driving cars and think you have what it takes to make a positive impact on the world, we encourage you to apply.