Staff Data Engineer in London

We are seeking a Staff Data Engineer to join our Emerging Tech team and define the data architecture powering our multimodal AI platform. You will set the technical vision and drive the implementation of scalable data pipelines, lakehouse infrastructure, and data platform capabilities that enable content intelligence, semantic search, recommendation, and personalization at scale — while raising the engineering bar across the data team.

A SNAPSHOT OF YOUR RESPONSIBILITIES

• Act as the technical anchor for your product pillar — collaborate closely with product leaders, ML engineers, backend engineers, editorial, and merchandising teams to translate ambiguous goals into clear technical designs, and communicate decisions effectively to both technical and non-technical stakeholders.
• Design and hands-on implement high-throughput batch and streaming data pipelines for multimodal content — including media segments, metadata, transcripts, and engagement signals — and architect the data models and ML feature stores that support them.
• Build scalable data ingestion frameworks across heterogeneous sources including media processing systems, AI inference services, and user engagement events; partner with ML engineers to define feature-ready data contracts for model training and inference, including embedding generation and vector storage.
• Own data governance, lineage tracking, and quality frameworks; design observability and alerting to ensure data integrity and SLA compliance at scale.
• Drive pipeline performance optimization and cloud cost management; lead adoption of CI/CD and infrastructure-as-code practices across the team.
• Mentor data engineers at all levels, conduct design and code reviews, and evaluate emerging technologies to ensure the team's technical decisions align with platform strategy, security, and compliance requirements.

WHAT YOU WILL NEED

• Extensive data engineering experience operating production systems at scale in global engineering organizations.
• Expert-level proficiency in Python and SQL for large-scale data processing and transformation.
• Deep experience with distributed data processing frameworks (Apache Spark, Apache Flink, or equivalent) and streaming architectures (Kafka, Spark Structured Streaming) for both batch and real-time workloads at terabyte scale.
• Proven experience building data infrastructure for LLM and generative AI workloads — including training data preparation, embedding generation, and vector storage.
• Proven ability to provide technical clarity in ambiguous environments — translating loosely defined product goals into actionable architecture decisions and driving alignment across engineering, ML, and product stakeholders.
• Strong cloud platform experience on GCP, AWS, or Azure with hands-on infrastructure-as-code (Terraform or CDK) and DevOps practices.
• Deep understanding of data warehousing, data mesh principles, and open table format standards (Apache Iceberg, Delta Lake, or Apache Hudi).
• Ownership mindset with end-to-end accountability for architecture, implementation, and production operations.

NICE TO HAVE, BUT NOT A DEALBREAKER

• Experience with managed lakehouse platforms (Databricks or equivalent) and their ecosystem tooling.
• Knowledge of media data formats, content metadata standards, or media processing pipelines.
• Experience with observability and monitoring systems (Datadog, Grafana, or OpenTelemetry).
• Experience leading data platform migrations or large-scale data infrastructure initiatives.
• Contributions to open-source data engineering projects or active participation in the data engineering community.
• Curiosity and enthusiasm for multimodal AI, generative AI, and LLM-powered applications.