At a Glance
- Tasks: Build and maintain scalable data pipelines using Databricks and Spark.
- Company: Join a forward-thinking company expanding its data engineering team.
- Benefits: Hybrid work model, competitive pay, and opportunities for professional growth.
- Why this job: Make an impact by leading innovative data projects and mentoring future engineers.
- Qualifications: Experience with Databricks, Spark, and strong leadership skills required.
- Other info: Dynamic role with a focus on cutting-edge technology and team collaboration.
The predicted salary is between 54000 - 84000 £ per year.
Senior Data Engineer – Hybrid – Contract – Outside IR35
Our client is expanding their core data engineering function and hiring two Senior Data Engineers to strengthen a modern, scalable data platform. Both will be Databricks specialists, with one role focused on deep technical delivery and the other combining hands-on engineering with team leadership and upskilling responsibilities.
Key Responsibilities
- Build and maintain Databricks Delta Live Tables (DLT) pipelines across Bronze → Silver → Gold layers, ensuring quality, scalability, and reliability.
- Develop and optimise Spark (PySpark) jobs for large-scale distributed processing.
- Design and implement streaming data pipelines with Kafka/MSK, applying best practices for late event handling and throughput.
- Use Terraform and CI/CD pipelines (GitHub Actions or similar) to manage infrastructure and automate deployments.
- (For leadership-focused role) Mentor and upskill engineers, define coding standards, and embed engineering excellence across the team.
What’s Expected
- Proven experience delivering end-to-end data pipelines in Databricks and Spark environments.
- Strong understanding of data modelling, schema evolution, and data contract management.
- Hands-on experience with Kafka, streaming architectures, and real-time processing principles.
- Proficiency with Docker, Terraform, and cloud platforms (AWS, GCP, or Azure) for scalable data infrastructure.
- Demonstrated ability to lead, coach, and elevate team capability, fostering best practices and knowledge sharing.
Senior Data Engineer employer: Develop
Contact Detail:
Develop Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Senior Data Engineer
✨Tip Number 1
Network like a pro! Reach out to your connections in the data engineering field, especially those who work with Databricks or Spark. A friendly chat can lead to insider info about job openings that aren't even advertised yet.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving Databricks Delta Live Tables and Spark jobs. This gives potential employers a taste of what you can do and sets you apart from the crowd.
✨Tip Number 3
Prepare for technical interviews by brushing up on your knowledge of streaming data pipelines and Kafka. Practise coding challenges related to data processing to demonstrate your hands-on experience during interviews.
✨Tip Number 4
Don’t forget to apply through our website! We’re always on the lookout for talented Senior Data Engineers. Your application will get the attention it deserves, and we’d love to see how you can contribute to our team!
We think you need these skills to ace Senior Data Engineer
Some tips for your application 🫡
Tailor Your CV: Make sure your CV highlights your experience with Databricks and Spark. We want to see how you've built and maintained data pipelines, so don’t hold back on those details!
Showcase Your Projects: Include specific examples of projects where you’ve implemented streaming data pipelines or optimised Spark jobs. We love seeing real-world applications of your skills!
Be Clear and Concise: When writing your cover letter, keep it straightforward. We appreciate clarity, so get to the point about why you’re a great fit for the role and how you can contribute to our team.
Apply Through Our Website: Don’t forget to submit your application through our website! It’s the best way for us to receive your details and ensure they’re reviewed promptly.
How to prepare for a job interview at Develop
✨Know Your Tech Inside Out
Make sure you’re well-versed in Databricks, Spark, and the other technologies mentioned in the job description. Brush up on your knowledge of Delta Live Tables, streaming data pipelines, and CI/CD practices. Being able to discuss specific projects where you've used these tools will really impress the interviewers.
✨Showcase Your Problem-Solving Skills
Prepare to discuss challenges you've faced in previous roles, especially around data pipeline delivery and optimisation. Use the STAR method (Situation, Task, Action, Result) to structure your answers. This will help you clearly demonstrate your thought process and how you tackle complex problems.
✨Demonstrate Leadership Qualities
If you're applying for the leadership-focused role, be ready to talk about your mentoring experiences. Share examples of how you've upskilled team members or defined coding standards. Highlighting your ability to foster a culture of engineering excellence will resonate well with the interviewers.
✨Ask Insightful Questions
Prepare thoughtful questions about the company’s data strategy, team dynamics, and future projects. This shows your genuine interest in the role and helps you assess if the company is the right fit for you. Plus, it gives you a chance to engage in a meaningful conversation with your interviewers.