Databricks Lead Engineer ££575 GBP Hybrid WORKING Location: Central London, Greater London - United Kingdom Type: Contract NEW CONTRACT ROLE - DATABRICKS LEAD ENGINEER A new opportunity has just landed for an experienced, hands-on Databricks Lead Engineer to join a major data transformation programme. We're looking for someone with strong recent experience delivering modern, production-grade data platforms on Databricks. This is a hands-on technical leadership role, so we're looking for someone who can lead engineering delivery and design solutions, rather than simply work from an existing blueprint. Location: London - 3 days per week on-site Rate: Up to £575 per day Contract: Inside IR35 Duration: 6 months initial contract with potential extension Start: ASAP Key requirements: • 2+ years of recent, hands-on Databricks experience delivering production solutions • Active/current Databricks certification - Data Engineer, Machine Learning or Generative AI • Strong experience designing and implementing Databricks Lakehouse architectures • Hands-on implementation of Medallion Architecture - Bronze, Silver and Gold layers • Experience with Unity Catalog, including governance, access control, lineage and secure data sharing • Strong experience building Declarative Pipelines / DLT, including Expectations, data quality, failure handling and observability • Experience with Databricks-native ingestion approaches such as AutoLoader and/or Lakeflow Connect • Strong PySpark and SQL skills • Delta Lake experience, including MERGE, schema evolution, optimisation and performance tuning • Experience with Databricks Workflows and production pipeline orchestration • Strong understanding of CI/CD, testing, monitoring, alerting, incident triage and operational best practice • Ability to mentor engineers and act as the technical escalation point within the delivery team Nice-to-have experience includes: • Mosaic AI, Vector Search, model serving or RAG pipelines • Lakeflow ingestion connectors • Row-level security, column masking and sensitive data handling through Unity Catalog • Databricks performance optimisation, including Photon, cluster sizing, partitioning and shuffle/skew mitigation • Cost optimisation and understanding of Databricks compute and DBU drivers This is a great opportunity for a technically strong Databricks specialist who wants to take ownership of engineering delivery and help shape a modern enterprise data platform. If you're interested, or know someone who could be a good fit, please get in touch. #Databricks #DataEngineering #DataEngineer #Lakehouse #PySpark #DeltaLake #UnityCatalog #DataJobs #ContractJobs #LondonTech #InsideIR35 Reference: AMC-AQU-DLEN Postcode: EC2V 6AA #adqu TPBN1_UKTJ
Databricks Lead Engineer employer: ANSON MCCADE
Anson McCade is an exceptional employer that prioritises employee well-being and professional growth, offering a flexible remote working environment in the UK. With a strong focus on security and digital transformation, employees benefit from comprehensive training programmes and clear pathways for career advancement, all while contributing to impactful projects alongside talented teams and senior stakeholders.
