We're partnering with a leading online retail company that's transforming the way data and real-time intelligence shape customer experiences. Their mission is to harness cutting-edge data and streaming technologies to drive smarter decisions, improve efficiency, and create personalised journeys for millions of shoppers worldwide.
The Role
As a Senior Data Engineer, you'll play a key role in developing and optimising the backbone of the company's data platform. You'll be responsible for building and maintaining large-scale, real-time data pipelines that power analytics, machine learning, and operational systems across the business.
You'll collaborate with software engineers, data scientists, and analytics teams to ensure the platform delivers reliable, high-quality, and compliant data at scale. This is a hands-on engineering role that blends software craftsmanship with data architecture expertise.
Key responsibilities:
- Design and implement high-throughput data streaming solutions using Kafka, Flink, or Confluent.
- Build and maintain scalable backend systems in Python or Scala, following clean code and testing principles.
- Develop tools and frameworks for data governance, privacy, and quality monitoring, ensuring full compliance with data protection standards.
- Create resilient data workflows and automation within Airflow, Databricks, and other modern big data ecosystems.
- Implement and manage data observability and cataloguing tools (e.g., Monte Carlo, Atlan, DataHub) to enhance visibility and reliability.
- Partner with ML engineers, analysts, and analytics engineers to understand their data needs and enable advanced data use cases.
- Contribute to an engineering culture that values testing, peer reviews, and automation-first principles.
What You'll Bring
- Strong experience in streaming technologies such as Kafka, Flink, or Confluent.
- Advanced proficiency in Python or Scala, with a solid grasp of software engineering fundamentals.
- Proven ability to design, deploy, and scale production-grade data platforms and backend systems.
- Familiarity with data governance frameworks, privacy compliance, and automated data quality checks.
- Hands-on experience with big data tools (Airflow, Databricks) and data observability platforms.
- Collaborative mindset and experience working with cross-functional teams including ML and analytics specialists.
- Curiosity and enthusiasm for continuous learning - you stay up to date with the latest tools and trends in data engineering and love sharing knowledge with others.
Databricks Data Engineer in London employer: Harnham
Join a dynamic and forward-thinking B2B company that prioritises data-driven decision making and offers a fully remote work environment across the UK. With a strong focus on employee growth, you will have the opportunity to lead a talented team while shaping the marketing analytics strategy in a rapidly expanding organisation. Enjoy a collaborative work culture that values innovation and provides significant investment in your professional development.
We think you need these skills to ace Databricks Data Engineer in London
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Databricks Data Engineer role. Highlight your experience with streaming technologies like Kafka and Flink, and showcase your skills in Python or Scala. We want to see how your background aligns with our mission!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you're passionate about data engineering and how you can contribute to transforming customer experiences. Keep it engaging and relevant to the role we’re offering.
Showcase Your Projects:If you've worked on any relevant projects, don’t hold back! Include links or descriptions of your work with data pipelines, Airflow, or Databricks. We love seeing practical examples of your skills in action.
Apply Through Our Website:We encourage you to apply through our website for the best chance of getting noticed. It helps us keep track of applications and ensures you’re considered for this exciting opportunity. Don’t miss out!