At a Glance
- Tasks: Design and build secure, scalable data platforms using cutting-edge tools like Databricks and Azure.
- Company: Join a forward-thinking tech firm focused on data innovation.
- Benefits: Competitive salary, flexible work options, and opportunities for professional growth.
- Other info: Dynamic team environment with excellent career advancement potential.
- Why this job: Make a real impact by developing robust data solutions that drive business success.
- Qualifications: Degree in Computer Science or related field; experience with cloud-based data platforms.
The predicted salary is between 70000 - 90000 £ per year.
The Data Platform Engineer designs, develops, automates, and maintains secure, scalable, and compliant data platforms that enable the firm to efficiently manage, analyse, and utilise data. The role ensures that data solutions are robust and reliable while meeting regulatory obligations and safeguarding client confidentiality.
Key Responsibilities
- Design and architect scalable, secure, and compliant data platforms and solutions, producing technical documentation and securing approvals through governance bodies such as Architecture Review Boards.
- Build and deliver robust data solutions using Databricks, PySpark, Spark SQL, Azure Data Factory, and Azure services.
- Develop APIs and write efficient Python, PySpark, and SQL code to support data integration, processing, and automation.
- Implement and manage CI/CD pipelines and automated deployments using Azure DevOps to enable reliable releases across environments.
- Develop and maintain infrastructure-as-code (eg, Terraform, ARM) to provision and manage cloud resources, including ADF pipelines, Databricks assets, and Unity Catalog components.
- Monitor, troubleshoot, and optimise data platform performance, reliability, and costs, identifying bottlenecks and recommending improvements.
- Create dashboards and observability tools to report on platform performance, usage, incidents, and operational KPIs.
Knowledge, Skills & Experience
- Degree in Computer Science, Data Engineering, or a related field.
- Proven experience designing and building cloud-based data platforms, ideally within Azure.
- Strong hands‑on expertise with Databricks, PySpark, Spark SQL, and Azure Data Factory.
- Solid understanding of Data Lakehouse architecture and modern data platform design.
- Proficiency in Python for data engineering, automation, and data processing.
- Experience developing and integrating REST APIs for data services.
- Strong DevOps experience, including CI/CD, automated testing, and release management for data platforms.
- Experience with Infrastructure as Code tools such as Terraform or ARM templates.
- Knowledge of data modelling, ETL/ELT pipelines, and data warehousing concepts.
- Familiarity with monitoring, logging, and alerting tools (eg, Azure Monitor).
Desirable
- Experience with additional Azure services (eg, Fabric, Azure Functions, Logic Apps).
- Knowledge of cloud cost optimisation for data platforms.
- Understanding of data governance and regulatory compliance (eg, GDPR).
- Experience working in regulated or professional services environments.
Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehous[...] in London employer: Energy Jobline ZR
As a Lead Data Platform Engineer, you will thrive in a dynamic and innovative environment that prioritises employee growth and collaboration. Our company offers competitive benefits, a supportive work culture, and opportunities to work with cutting-edge technologies in the heart of a vibrant tech hub, ensuring that your contributions are both meaningful and impactful.
StudySmarter Expert Advice🤫
We think this is how you could land Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehous[...] in London
✨Tip Number 1
Network like a pro! Reach out to folks in your industry on LinkedIn or at meetups. We all know that sometimes it’s not just what you know, but who you know that can get you in the door.
✨Tip Number 2
Prepare for those interviews by practising common questions and scenarios related to data platforms. We recommend doing mock interviews with friends or using online platforms to boost your confidence.
✨Tip Number 3
Showcase your skills! Create a portfolio of projects that highlight your experience with Databricks, Azure Data Factory, and Terraform. We love seeing real-world applications of your expertise!
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we’re always on the lookout for talented individuals like you!
We think you need these skills to ace Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehous[...] in London
Some tips for your application 🫡
Tailor Your CV:Make sure your CV reflects the skills and experiences that match the job description. Highlight your expertise in Databricks, Azure Data Factory, and Terraform to show us you’re the right fit for the Lead Data Platform Engineer role.
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to tell us why you're passionate about data platforms and how your background aligns with our needs. Don’t forget to mention any relevant projects or achievements!
Showcase Your Technical Skills:We want to see your technical prowess! Include specific examples of your work with Python, PySpark, and CI/CD pipelines. This will help us understand how you can contribute to building robust data solutions.
Apply Through Our Website:To make sure your application gets the attention it deserves, apply directly through our website. It’s the best way for us to keep track of your application and get back to you quickly!
How to prepare for a job interview at Energy Jobline ZR
✨Know Your Tech Inside Out
Make sure you’re well-versed in Databricks, PySpark, and Azure Data Factory. Brush up on your knowledge of data lakehouse architecture and be ready to discuss how you've used these technologies in past projects.
✨Showcase Your Problem-Solving Skills
Prepare to talk about specific challenges you've faced in building data platforms. Think of examples where you optimised performance or resolved bottlenecks, and be ready to explain your thought process.
✨Demonstrate Your CI/CD Knowledge
Since the role involves managing CI/CD pipelines, be prepared to discuss your experience with Azure DevOps. Share examples of how you've implemented automated deployments and the benefits they brought to your projects.
✨Ask Insightful Questions
Interviews are a two-way street! Prepare thoughtful questions about the company’s data governance practices or their approach to cloud cost optimisation. This shows your genuine interest and helps you assess if it’s the right fit for you.
