At a Glance
- Tasks: Architect and build enterprise-grade data solutions using Databricks while driving engineering excellence.
- Company: Join a leading organisation transforming data architecture in a collaborative remote environment.
- Benefits: Competitive daily rate, flexible remote work, and opportunities for long-term contract extensions.
- Other info: Be part of a high-impact programme integrating Data Engineering, AI, and Enterprise Architecture.
- Why this job: Shape the future of data platforms and make a significant impact from day one.
- Qualifications: 8+ years in software engineering with strong Databricks and Delta Lake expertise.
The predicted salary is between 50400 - 50400 £ per year.
We're seeking an exceptional Senior Databricks Architect with a strong hands-on Data Engineering background to join a major long-term transformation programme. This is not a pure architecture role. You'll be a Databricks specialist who remains close to the technology, designing and building enterprise-scale data solutions while driving engineering excellence across critical data platforms.
You'll take ownership of delivering robust, scalable, and production-grade data solutions within a modern Databricks ecosystem, ensuring platforms are governed, secure, maintainable, and optimised for long-term growth.
Key Responsibilities- Architect, build and optimise enterprise-grade data solutions within Databricks
- Design and implement scalable Delta Lake data architectures following Medallion principles
- Lead data engineering delivery across multiple strategic workstreams
- Develop highly resilient, production-ready pipelines with strong operational controls
- Drive data governance, security, lineage and compliance best practices
- Collaborate with product, engineering and business teams to deliver trusted data assets
- Mentor internal engineering teams and establish engineering standards
- Product Data Ingestion
- Design and maintain large-scale ingestion pipelines
- Integrate assessment, content and product data into shared enterprise data platforms
- Ensure data quality, consistency and operational reliability
- AI & GenAI Feature Enablement
- Build and maintain data foundations powering LLM and GenAI capabilities
- Design schemas and pipelines supporting AI-driven products and document processing
- Handle complex parsed, normalised and sensitive datasets at scale
- Support emerging AI use cases including evaluation processing, bulk uploads and intelligent workflows
- IEP & Medicaid Data Integration
- Build robust integrations across IEP, Medicaid and associated service platforms
- Deliver reconciliation, data quality and cross-system consistency
- Support enterprise reporting and operational analytics requirements
- Deep expertise in Databricks and Delta Lake architecture
- Proven experience implementing and operating Medallion Architecture (Bronze, Silver, Gold)
- Advanced pipeline engineering experience including:
- Orchestration
- Idempotent processing
- Error handling
- Monitoring
- Recovery strategies
- Data reconciliation
- Strong Unity Catalog experience including:
- Data governance
- Lineage
- Fine-grained permissions
- Row-level and column-level security
- Extensive ETL and data integration expertise across complex enterprise systems
- Strong understanding of:
- Data quality frameworks
- Data normalisation
- Backfilling strategies
- Data integrity controls
- Experience handling sensitive and regulated data environments
- Knowledge of PII protection techniques including:
- Masking
- Tokenisation
- Anonymisation
- Familiarity with GDPR and broader data privacy standards
- Performance and cost optimisation expertise including:
- Z-Ordering
- Liquid Clustering
- Data Skipping
- Query tuning
- Shuffle optimisation
- Cloud cost control
- 8+ years software engineering experience
- 3+ years operating at Senior/Lead Data Engineer or Databricks Architect level
- Databricks AI capabilities
- Vector Search
- Model Serving
- LLM Pipelines
- Retrieval-Augmented Generation (RAG)
- EdTech domain experience
- Experience with industry standards such as:
- Ed-Fi
- OneRoster
- CEDS
- 1EdTech
High-impact programme combining Data Engineering, AI and Enterprise Architecture. Opportunity to shape the future data platform of a growing organisation. Significant ownership and influence from day one.
Senior Databricks Architect/Lead Databricks Engineer employer: Careerwise
Join a forward-thinking organisation as a Senior Databricks Architect, where you'll play a pivotal role in a transformative programme that values innovation and engineering excellence. Enjoy the flexibility of remote work within the UK, competitive daily rates, and the opportunity to mentor and lead teams while shaping enterprise-scale data solutions. With a strong focus on employee growth and collaboration, this role offers a unique chance to influence the future of data architecture in a supportive and dynamic environment.
StudySmarter Expert Advice🤫
We think this is how you could land Senior Databricks Architect/Lead Databricks Engineer
✨Tip Number 1
Network like a pro! Reach out to your connections in the data engineering and Databricks space. Attend meetups, webinars, or even just grab a coffee with someone in the industry. You never know who might have the inside scoop on job openings!
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your best Databricks projects. Whether it's a GitHub repo or a personal website, having tangible examples of your work can really set you apart from the competition.
✨Tip Number 3
Prepare for those interviews! Brush up on your technical knowledge, especially around Delta Lake and Medallion Architecture. Practice common interview questions and be ready to discuss how you've tackled challenges in past projects.
✨Tip Number 4
Apply through our website! We’ve got loads of opportunities that might be perfect for you. Plus, applying directly can sometimes give you a better chance of getting noticed by hiring managers.
We think you need these skills to ace Senior Databricks Architect/Lead Databricks Engineer
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience with Databricks and Delta Lake architecture. We want to see how your skills align with the key responsibilities mentioned in the job description, so don’t hold back on showcasing your relevant projects!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you’re the perfect fit for this role. Mention your hands-on data engineering background and how you’ve driven engineering excellence in past projects. Let us know what excites you about working with us at StudySmarter!
Showcase Your Problem-Solving Skills:In your application, highlight specific examples where you've tackled complex data challenges. We love seeing how you’ve built resilient pipelines or implemented data governance best practices. This will help us understand your approach to problem-solving in a real-world context.
Apply Through Our Website:We encourage you to apply directly through our website. It’s the easiest way for us to receive your application and ensures you’re considered for the role. Plus, it gives you a chance to explore more about our culture and values!
How to prepare for a job interview at Careerwise
✨Know Your Databricks Inside Out
Make sure you brush up on your Databricks and Delta Lake architecture knowledge. Be ready to discuss specific projects where you've implemented Medallion Architecture and how you've tackled challenges in pipeline engineering. This role demands a hands-on approach, so be prepared to showcase your technical expertise.
✨Showcase Your Problem-Solving Skills
Prepare to share examples of how you've built resilient data pipelines and handled error management. Think about times when you had to optimise performance or ensure data quality in complex environments. Highlighting your problem-solving skills will demonstrate your capability to drive engineering excellence.
✨Understand the Business Context
Familiarise yourself with the company's goals and how your role as a Senior Databricks Architect fits into their long-term transformation programme. Be ready to discuss how you can collaborate with product and engineering teams to deliver trusted data assets that align with business objectives.
✨Prepare Questions About Governance and Compliance
Since data governance and compliance are crucial for this role, come prepared with questions about their current practices. Ask about their approach to data lineage, security, and GDPR compliance. This shows your commitment to maintaining high standards and your understanding of the importance of these aspects in data architecture.
