At a Glance
- Tasks: Design and build secure, scalable data platforms using cutting-edge technologies.
- Company: Join a forward-thinking firm focused on data innovation and compliance.
- Benefits: Competitive salary, flexible working options, and opportunities for professional growth.
- Why this job: Make a real impact by developing robust data solutions that drive business success.
- Qualifications: Degree in Computer Science or related field; experience with Azure and Databricks.
- Other info: Dynamic team environment with excellent career advancement opportunities.
The predicted salary is between 70000 - 90000 £ per year.
The Data Platform Engineer designs, develops, automates, and maintains secure, scalable, and compliant data platforms that enable the firm to efficiently manage, analyse, and utilise data. The role ensures that data solutions are robust and reliable while meeting regulatory obligations and safeguarding client confidentiality.
Key Responsibilities
- Design and architect scalable, secure, and compliant data platforms and solutions, producing technical documentation and securing approvals through governance bodies such as Architecture Review Boards.
- Build and deliver robust data solutions using Databricks, PySpark, Spark SQL, Azure Data Factory, and Azure services.
- Develop APIs and write efficient Python, PySpark, and SQL code to support data integration, processing, and automation.
- Implement and manage CI/CD pipelines and automated deployments using Azure DevOps to enable reliable releases across environments.
- Develop and maintain infrastructure-as-code (eg, Terraform, ARM) to provision and manage cloud resources, including ADF pipelines, Databricks assets, and Unity Catalog components.
- Monitor, troubleshoot, and optimise data platform performance, reliability, and costs, identifying bottlenecks and recommending improvements.
- Create dashboards and observability tools to report on platform performance, usage, incidents, and operational KPIs.
Knowledge, Skills & Experience
- Degree in Computer Science, Data Engineering, or a related field.
- Proven experience designing and building cloud-based data platforms, ideally within Azure.
- Strong hands-on expertise with Databricks, PySpark, Spark SQL, and Azure Data Factory.
- Solid understanding of Data Lakehouse architecture and modern data platform design.
- Proficiency in Python for data engineering, automation, and data processing.
- Experience developing and integrating REST APIs for data services.
- Strong DevOps experience, including CI/CD, automated testing, and release management for data platforms.
- Experience with Infrastructure as Code tools such as Terraform or ARM templates.
- Knowledge of data modelling, ETL/ELT pipelines, and data warehousing concepts.
- Familiarity with monitoring, logging, and alerting tools (eg, Azure Monitor).
Desirable
- Experience with additional Azure services (eg, Fabric, Azure Functions, Logic Apps).
- Knowledge of cloud cost optimisation for data platforms.
- Understanding of data governance and regulatory compliance (eg, GDPR).
- Experience working in regulated or professional services environments.
Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse employer: Nexere Consulting Limited
Contact Detail:
Nexere Consulting Limited Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse
✨Tip Number 1
Network like a pro! Reach out to folks in your industry on LinkedIn or at meetups. A friendly chat can lead to opportunities that aren’t even advertised yet.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving Databricks and Azure. This gives potential employers a taste of what you can do.
✨Tip Number 3
Prepare for interviews by practising common questions related to data platforms and cloud services. We recommend doing mock interviews with friends or using online platforms to boost your confidence.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen by the right people. Plus, we love hearing from passionate candidates like you!
We think you need these skills to ace Lead Data Platform Engineer - Databricks - IAC - Terraform - Azure Data Factory - Data Lakehouse
Some tips for your application 🫡
Tailor Your CV: Make sure your CV speaks directly to the job description. Highlight your experience with Databricks, Azure Data Factory, and Terraform. We want to see how your skills align with what we're looking for!
Craft a Compelling Cover Letter: Your cover letter is your chance to shine! Use it to tell us why you're passionate about data platforms and how your background makes you the perfect fit for this role. Keep it engaging and relevant!
Showcase Your Projects: If you've worked on any cool projects involving data engineering or cloud platforms, make sure to mention them! We love seeing real-world applications of your skills, especially with Azure services.
Apply Through Our Website: We encourage you to apply through our website for a smoother process. It helps us keep track of your application and ensures you don’t miss out on any important updates from us!
How to prepare for a job interview at Nexere Consulting Limited
✨Know Your Tech Inside Out
Make sure you’re well-versed in Databricks, PySpark, and Azure Data Factory. Brush up on your knowledge of Data Lakehouse architecture and be ready to discuss how you've used these technologies in past projects.
✨Showcase Your Problem-Solving Skills
Prepare to share specific examples of how you've tackled challenges in data platform performance or reliability. Think about bottlenecks you've identified and the improvements you've implemented—this will demonstrate your hands-on experience.
✨Understand Infrastructure as Code
Be ready to talk about your experience with Terraform or ARM templates. Highlight any projects where you’ve successfully provisioned cloud resources and managed deployments, as this is crucial for the role.
✨Familiarise Yourself with Compliance
Since the role involves regulatory obligations, brush up on data governance and compliance standards like GDPR. Be prepared to discuss how you’ve ensured client confidentiality and compliance in your previous roles.