Data Engineer - Databricks / AI in London

At Jacobs, we're challenging today to reinvent tomorrow by solving the world's most critical problems for thriving cities, resilient environments, mission‑critical outcomes, operational advancement, scientific discovery and cutting‑edge manufacturing, turning abstract ideas into realities that transform the world for good.

Your impact

Enjoy designing elegant data systems, shipping production‑grade code, and seeing your work make a measurable difference.

Responsibilities

• Design and build robust data platforms and pipelines on Azure and Databricks (batch + streaming) using Python/SQL, Spark, Delta Lake, and Data Lakehouse patterns.
• Develop AI‑enabling foundations: feature stores, ML‑ready datasets, and automated model‑serving pathways (MLflow, model registries, CI/CD).
• Own quality & reliability testing (dbx/pytest), observability (metrics, logging, lineage), and cost/performance optimisation.
• Harden for enterprise security‑by‑design, access patterns with Unity Catalog, data governance, and reproducible environments.
• Automate the boring stuff: Ia C (Terraform/Bicep), CI/CD (Azure Dev Ops/Git Hub Actions), and templated project scaffolding.
• Partner with clients to translate business problems into technical plans, run workshops, and present trade‑offs with clarity.
• Ship value continuously: iterate, review, and release frequently; measure outcomes, not just outputs.

Qualifications

• Utilising SQL and Python for building reliable data pipelines.
• Hands‑on with Spark (preferably Databricks) and modern data modelling (Kimball, Inmon, Data Vault, lakehouse).
• Experience running on a cloud data platform (ideally Azure) with sound software delivery practices: Git, CI/CD, testing, Agile ways of working.
• Streaming/event‑driven designs (Event Hubs, Kafka, Structured Streaming).
• MPP/Data Warehouses (Synapse, Snowflake, Redshift) and No SQL (Cosmos DB).
• ML enablement: feature engineering at scale, MLflow, basic model lifecycle know‑how.
• Infrastructure‑as‑code (Terraform/Bicep) and platform hardening.

Benefits & Working Environment

Joining Jacobs gives you access to flexibility, safety, and global opportunities.

We support flexible working arrangements, well‑being benefits, our global giving and volunteering program, and innovative ways to help our clients make the world a better place.

We value inclusion, belonging, and positive mental health and have initiatives such as Jacobs Employee Networks and Communities of Practice.

We encourage hybrid working with the option to split your week between office and remote.

Jacobs partners with VERCIDA to help attract and retain talent from a wide range of backgrounds.

For greater online accessibility, visit https://www. vercida. com/uk/employers/jacobs.

Find out more about life at Jacobs and our employee networks.

Equal Opportunity Employer

Jacobs is a disability‑confident employer and welcomes applications from candidates who may not meet all specified requirements.

We support reasonable adjustments for the recruitment process upon request.

We empower employees with a hybrid working policy, allowing them to split their work week between Jacobs offices/projects and remote locations.

Your application experience is important to us, and we’re keen to adapt to make every interaction even better.

#J-18808-Ljbffr