Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse
Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse

Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse

Full-Time 70000 - 90000 £ / year (est.) Home office (partial)
Nexere Consulting Limited

At a Glance

  • Tasks: Design and build secure, scalable data platforms using cutting-edge technologies.
  • Company: Join a forward-thinking firm focused on data innovation and compliance.
  • Benefits: Competitive salary, flexible working options, and opportunities for professional growth.
  • Why this job: Make a real impact by developing robust data solutions that drive business success.
  • Qualifications: Degree in Computer Science or related field; experience with Azure and Databricks.
  • Other info: Dynamic team environment with excellent career advancement opportunities.

The predicted salary is between 70000 - 90000 £ per year.

The Data Platform Engineer designs, develops, automates, and maintains secure, scalable, and compliant data platforms that enable the firm to efficiently manage, analyse, and utilise data. The role ensures that data solutions are robust and reliable while meeting regulatory obligations and safeguarding client confidentiality.

Key Responsibilities

  • Design and architect scalable, secure, and compliant data platforms and solutions, producing technical documentation and securing approvals through governance bodies such as Architecture Review Boards.
  • Build and deliver robust data solutions using Databricks, PySpark, Spark SQL, Azure Data Factory, and Azure services.
  • Develop APIs and write efficient Python, PySpark, and SQL code to support data integration, processing, and automation.
  • Implement and manage CI/CD pipelines and automated deployments using Azure DevOps to enable reliable releases across environments.
  • Develop and maintain infrastructure-as-code (eg, Terraform, ARM) to provision and manage cloud resources, including ADF pipelines, Databricks assets, and Unity Catalog components.
  • Monitor, troubleshoot, and optimise data platform performance, reliability, and costs, identifying bottlenecks and recommending improvements.
  • Create dashboards and observability tools to report on platform performance, usage, incidents, and operational KPIs.

Knowledge, Skills & Experience

  • Degree in Computer Science, Data Engineering, or a related field.
  • Proven experience designing and building cloud-based data platforms, ideally within Azure.
  • Strong hands-on expertise with Databricks, PySpark, Spark SQL, and Azure Data Factory.
  • Solid understanding of Data Lakehouse architecture and modern data platform design.
  • Proficiency in Python for data engineering, automation, and data processing.
  • Experience developing and integrating REST APIs for data services.
  • Strong DevOps experience, including CI/CD, automated testing, and release management for data platforms.
  • Experience with Infrastructure as Code tools such as Terraform or ARM templates.
  • Knowledge of data modelling, ETL/ELT pipelines, and data warehousing concepts.
  • Familiarity with monitoring, logging, and alerting tools (eg, Azure Monitor).

Desirable

  • Experience with additional Azure services (eg, Fabric, Azure Functions, Logic Apps).
  • Knowledge of cloud cost optimisation for data platforms.
  • Understanding of data governance and regulatory compliance (eg, GDPR).
  • Experience working in regulated or professional services environments.

Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse employer: Nexere Consulting Limited

As a Lead Data Platform Engineer, you will thrive in a dynamic and innovative work environment that prioritises employee growth and collaboration. Our company offers competitive benefits, a strong focus on professional development, and the opportunity to work with cutting-edge technologies in a supportive culture that values creativity and teamwork. Located in a vibrant area, we provide unique advantages such as flexible working arrangements and access to industry-leading resources, making us an excellent employer for those seeking meaningful and rewarding careers.
Nexere Consulting Limited

Contact Detail:

Nexere Consulting Limited Recruiting Team

StudySmarter Expert Advice 🤫

We think this is how you could land Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse

✨Tip Number 1

Network like a pro! Reach out to folks in your industry on LinkedIn or at meetups. A friendly chat can lead to opportunities that aren’t even advertised yet.

✨Tip Number 2

Show off your skills! Create a portfolio showcasing your projects, especially those involving Databricks and Azure. This gives potential employers a taste of what you can do.

✨Tip Number 3

Prepare for interviews by practising common questions related to data platforms and cloud services. We recommend doing mock interviews with friends or using online platforms to boost your confidence.

✨Tip Number 4

Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love hearing from passionate candidates like you!

We think you need these skills to ace Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse

Databricks
PySpark
Spark SQL
Azure Data Factory
Python
CI/CD
Terraform
Infrastructure as Code
Data Lakehouse architecture
REST APIs
Data modelling
ETL/ELT pipelines
Azure services
Monitoring tools
Data governance

Some tips for your application 🫡

Tailor Your CV: Make sure your CV speaks directly to the job description. Highlight your experience with Databricks, Azure Data Factory, and Terraform. We want to see how your skills align with what we're looking for!

Craft a Compelling Cover Letter: Your cover letter is your chance to shine! Use it to tell us why you're passionate about data platforms and how your background makes you the perfect fit for this role. Keep it engaging and relevant!

Showcase Your Projects: If you've worked on any cool projects involving data engineering or cloud platforms, make sure to mention them! We love seeing real-world applications of your skills, especially with Azure services.

Apply Through Our Website: We encourage you to apply through our website for a smoother process. It helps us keep track of your application and ensures you don’t miss out on any important updates from us!

How to prepare for a job interview at Nexere Consulting Limited

✨Know Your Tech Inside Out

Make sure you’re well-versed in Databricks, PySpark, and Azure Data Factory. Brush up on your knowledge of Data Lakehouse architecture and be ready to discuss how you've used these technologies in past projects.

✨Showcase Your Problem-Solving Skills

Prepare to share specific examples of how you've tackled challenges in data platform performance or reliability. Think about bottlenecks you've identified and the improvements you've implemented—this will demonstrate your hands-on experience.

✨Understand Infrastructure as Code

Be ready to talk about your experience with Terraform or ARM templates. Highlight any projects where you’ve successfully provisioned cloud resources and managed deployments, as this is crucial for the role.

✨Familiarise Yourself with Compliance

Since the role involves regulatory obligations, brush up on data governance and compliance standards like GDPR. Be prepared to discuss how you’ve ensured client confidentiality and compliance in your previous roles.

Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse
Nexere Consulting Limited

Land your dream job quicker with Premium

You’re marked as a top applicant with our partner companies
Individual CV and cover letter feedback including tailoring to specific job roles
Be among the first applications for new jobs with our AI application
1:1 support and career advice from our career coaches
Go Premium

Money-back if you don't land a job in 6-months

>