Senior Data Engineer

Production-level Python: OOP design patterns, async processing, unit/integration testing, GCP SDK usage.

Demonstrated experience designing CI/CD pipelines for data products.

You operate at the frontier of modern data engineering. You understand that AI is not a future consideration - it is a present-day design constraint. You build data infrastructure that is AI-ready by default: pipelines that serve feature stores, architectures that can support RAG and LLM applications, and platforms capable of integrating AI-assisted tooling at every stage of the engineering lifecycle.

In a global team spanning Europe, the US, and India, you are a connector - bridging technical depth with business context, and aligning local delivery with global standards.

• Technical Architecture champion

IaC as a non-negotiable standard.

Design and implement CI/CD pipelines for all data solutions - with automated testing, linting, and deployment gates.

• AI-Era Responsibilities

AI-ready architecture: Design every data platform component to be downstream-AI-compatible - appropriate partitioning, feature store integration, and schema design for ML consumption.

GenAI data infrastructure: Architect data pipelines for LLM-based applications, including embedding generation pipelines, vector store population, and RAG data retrieval layers.

Feature store engineering: Build and maintain centralised feature stores on Vertex AI, ensuring reproducibility and low-latency serving for ML models.

AI-assisted development leadership: Champion GitHub Copilot, Gemini Code Assist, and Cursor as engineering productivity tools - set standards for how the team uses them responsibly.

AI-powered data quality: Design ML-based anomaly detection into pipeline monitoring - moving beyond threshold alerts to intelligent pattern recognition.

LLMOps data layer: Build the data infrastructure that underpins model evaluation, fine-tuning dataset curation, and prompt tracking pipelines.

• Leadership

Hold the bar for quality, testability, and maintainability.

Define and document reusable engineering patterns - pipeline templates, transformation standards, naming conventions.

Actively mentor junior engineers through pairing, structured feedback, and technical design sessions.

Work closely with global Data Engineering counterparts to align on platform standards.

Engage directly with senior business stakeholders to translate complex requirements into technical solutions.

Contribute to hiring: review take-home tasks, conduct technical interviews, calibrate assessments.

Define and execute testing strategies for regulated workloads, including parallel-run validation against legacy systems.

• Operational Excellence

Own pipeline reliability: define SLAs, implement alerting, lead incident resolution.

Drive DataOps practices: automated testing, data contracts, observability-first design.

Monitor and optimise GCP costs; propose and implement efficiency improvements.

Ensure compliance with data security, encryption, and governance standards in all solutions built.

• Essential - Technical

Proven data engineering expertise in production, cloud-native environments.

Advanced SQL proficiency, including BigQuery-specific development, query profiling, partitioning and clustering optimisation, and complex analytical query design.

Strong hands-on experience with Google Cloud Platform (GCP), including architecture design, implementation, and delivery of scalable production solutions.

Deep, hands-on expertise across: BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, Dataplex, Cloud Storage, Terraform, Cloud Build.

Mastery of data modelling methodologies: Dimensional/Kimball, 3NF, Data Vault - with real-world application of each.

Track record of leading legacy-to-cloud migrations.

• Essential - Leadership

Demonstrated professional experience considered in lieu for internal applicants.

• Desired

GCP Professional Data Engineer certification.

Experience designing AI/ML data pipelines - feature stores, training data pipelines, Vertex AI integration.

Hands-on experience with vector databases or embedding pipeline design.

Active use of AI-assisted development tools (Copilot, Gemini, Cursor) in production delivery.

Experience with dbt Core / Dataform in a production, team setting.

Data engineering experience in a regulated financial environment (banking, insurance, credit).

Experience designing event-driven architectures with Pub/Sub and Dataflow.

• What You Can Expect

A defining role on a high-priority, high-visibility data platform programme.

Genuine technical leadership - your architecture decisions will stand in production.

Direct exposure to GenAI infrastructure, ML platforms, and AI-era data engineering.

Collaboration with global engineering teams across three continents.

Support and funding for GCP Professional Data Engineer certification and advanced training.

A clear pathway to Lead Engineer for the right candidate.

Hybrid working from a modern campus environment.

The Company is committed to diversity and equality of opportunity for all and is opposed to any form of less favourable treatment or harassment on the grounds of race, religion or belief, sex, marriage and civil partnership, pregnancy and maternity, age, sexual orientation, gender reassignment or disability.

This position is based in Dunton, and it is expected the successful candidate will be able to attend the Dunton Campus for typically 4 days a week and remain flexible on the days they are required to attend the office according to business requirements.

As part of our pre-employment checks process, successful candidates will be required to undergo a criminal record check. This will be conducted in line with the Rehabilitation of Offenders Act 1974 and applied only to unspent convictions.