Databricks Data Engineer in London

We are seeking a Databricks Data Engineer to join a specialist consultancy focused on transforming enterprises through modern cloud data platforms, advanced analytics and next‑generation engineering capabilities.

This role requires strong experience working within the Lloyd’s of London or wider London Market insurance environment, with the ability to deliver data solutions that meet strict regulatory, audit and operational standards.

You will take ownership of designing, building and optimising large‑scale Azure data solutions, working closely with business and technical stakeholders to translate complex data challenges into secure, scalable and production‑ready platforms. You will help shape cloud data strategy, define engineering best practice and guide delivery across cross‑functional teams.

Responsibilities

• Build and optimise Azure data platforms using ADF, Data Lake, Azure Functions and Databricks
• Deliver reliable, high‑quality data pipelines
• Work with stakeholders to ensure solutions meet Lloyd’s of London regulatory, audit and data governance requirements
• Promote best practices in engineering, automation and cloud design.
• Lead and mentor engineers, setting technical standards.
• Improve platform performance, resilience and cost efficiency.

Skills and Experience

• Extensive engineering experience across Azure Data Services and Databricks in enterprise environments.
• Proven delivery experience within the Lloyd’s of London or London Market insurance sector
• Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling.
• Strong Python, PySpark and Spark SQL capability
• Solid understanding of data governance
• Proven track record of ownership, delivery and operating effectively in fast‑paced environments.

What’s on Offer

• Salary of up to £95,000
• Performance related bonus scheme
• Hybrid working – Up to 3 days working from home

This is just a brief overview of the opportunity. To learn more, simply apply with your CV and we’ll be in touch to discuss the role in more detail.

#J-18808-Ljbffr