Salary: £130,000 - 149,500 per year
Requirements:
- At least two years of recent, hands-on Databricks experience
- Evidence of delivering Databricks solutions into production environments
- An active Databricks Data Engineering, Machine Learning or Generative AI certification
- Strong experience implementing Medallion Architecture across Bronze, Silver and Gold layers
- Hands-on Unity Catalog experience covering catalog design, access controls, lineage and secure data sharing
- Experience developing Declarative Pipelines with Expectations, including failure handling and observability
- Knowledge of Auto Loader, Lakeflow Connect and batch, streaming or incremental ingestion patterns
- Production-level PySpark and SQL development skills
- Strong knowledge of Delta Lake, including MERGE, schema evolution, OPTIMIZE, ZORDER and VACUUM
- Experience with Databricks Workflows, orchestration and CI/CD
- Understanding of monitoring, alerting, replay, backfill and operational support strategies
- Ability to mentor engineers and act as the technical escalation point
- Experience with RBAC, data masking, Photon, cluster optimisation or cost management would be beneficial
- Exposure to Mosaic AI, Vector Search, model serving, RAG or Lakebase would be advantageous
Responsibilities:
- Lead the delivery of a modern enterprise data platform for a leading financial services organisation
- Take technical ownership of Databricks lakehouse solutions, covering ingestion through to curated serving layers
- Implement Medallion Architecture, Unity Catalog, Declarative Pipelines and scalable Databricks-native ingestion patterns
- Mentor engineers and drive production-quality delivery
- Shape engineering standards, reusable patterns and delivery practices
Technologies:
- AI
- CI/CD
- Databricks
- Support
- Machine Learning
- Model Serving
- PySpark
- RAG
- RBAC
- SQL
- Unity
- GameDev
More:
We are offering up to £575 per day inside IR35 on an initial 6-month contract with potential extension. The role is hybrid with three days per week on-site in London and an ASAP start. We are providing technical leadership of a strategic Databricks implementation, with the opportunity to shape engineering standards, reusable patterns and delivery practices for a leading financial services organisation.
last updated 38 week of 2026
#J-18808-Ljbffr
Databricks Lead Engineer - South East London employer: ANSON MCCADE
Anson McCade is an exceptional employer that prioritises employee well-being and professional growth, offering a flexible remote working environment in the UK. With a strong focus on security and digital transformation, employees benefit from comprehensive training programmes and clear pathways for career advancement, all while contributing to impactful projects alongside talented teams and senior stakeholders.
