At a Glance
- Tasks: Design and optimise scalable data pipelines using Databricks and AWS technologies.
- Company: Join a leading organisation focused on modern, data-driven platforms.
- Benefits: Enjoy a competitive salary, bonus scheme, shares option, and generous holiday allowance.
- Other info: Collaborative office culture with social events and clear progression opportunities.
- Why this job: Make a real impact by delivering high-quality data insights across various teams.
- Qualifications: Experience with Databricks, Apache Spark, and strong Python and SQL skills required.
The predicted salary is between 60000 - 60000 £ per year.
Salary up to £60,000 + discretionary bonus
Hybrid working model with collaborative office environment
Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS
About the client
Due to continued growth, we’re seeking a skilled Data Engineer to join an established organisation at the forefront of building modern, data-driven platforms. The business is investing heavily in its data ecosystem and is focused on delivering real-time insights and trusted analytics that support teams across product, finance, marketing, and compliance. This is an excellent opportunity to work with modern cloud data technologies and play a key role in shaping a scalable, secure, and high-performance data platform.
What will you be doing?
As a Data Engineer, you will design, build, and optimise scalable data pipelines within a Databricks Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering teams, you will help deliver high-quality, governed data that supports analytics, reporting, and operational decision‑making. You’ll also play a key role in enabling business intelligence through high-performance data models and integrations with reporting tools. You’ll monitor and optimise pipeline performance, implement CI/CD processes for data workflows, and maintain strong governance and security standards across the data platform. Documentation, observability, and cost optimisation will also form part of your responsibilities as you contribute to the ongoing development of the organisation’s data ecosystem.
Our benefits
- Competitive basic salary
- Discretionary bonus scheme
- Company shares option plan
- Contributory pension scheme
- Life insurance (4x basic salary)
- Health cash plan
- Generous holiday allowance including bank holidays
- Study support and clear progression opportunities
- Employee assistance programme available 24/7
- Collaborative office environment with regular social and charity events
Data Engineer – Essential Skills
- Strong experience with Databricks and Apache Spark (PySpark or Scala)
- Experience building streaming and batch pipelines using Spark Structured Streaming
- Hands‑on experience with Kafka (MSK) and real‑time data ingestion
- Strong understanding of Delta Lake, Delta Live Tables, and Medallion Architecture
- Solid AWS experience including services such as S3, Glue, Lambda, Batch, and IAM
- Proficiency in Python and SQL for data engineering and analytics
- Experience implementing CI/CD pipelines (GitHub Actions, Jenkins, or Azure DevOps)
- Strong Git and version control practices
- Experience tuning and optimising Spark workloads
To be considered
Please either apply by clicking online or emailing me directly chelsea.hackett@searchability.com.
Data Engineer - Real-Time Pipelines, Databricks & AWS employer: Searchability
Join a forward-thinking organisation that prioritises innovation and collaboration in the heart of a vibrant city. As a Data Engineer, you'll benefit from a competitive salary, generous holiday allowance, and a supportive work culture that fosters professional growth through study support and clear progression opportunities. With a focus on modern data technologies and a commitment to employee well-being, this company offers a unique environment where your contributions will directly impact the development of cutting-edge data solutions.
StudySmarter Expert Advice🤫
We think this is how you could land Data Engineer - Real-Time Pipelines, Databricks & AWS
✨Tip Number 1
Network like a pro! Reach out to people in the industry, attend meetups, and connect with fellow data enthusiasts. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving Databricks, Spark, and AWS. This gives potential employers a taste of what you can do and sets you apart from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on your technical knowledge and problem-solving skills. Practice common data engineering scenarios and be ready to discuss how you've tackled challenges in past projects.
✨Tip Number 4
Don't forget to apply through our website! It’s the best way to ensure your application gets noticed. Plus, it shows you're serious about joining our team and contributing to our exciting data ecosystem.
We think you need these skills to ace Data Engineer - Real-Time Pipelines, Databricks & AWS
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience with Databricks, Spark, and AWS. We want to see how your skills match the job description, so don’t be shy about showcasing relevant projects or achievements!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you’re excited about this role and how your background makes you a perfect fit. We love seeing passion and personality, so let that come through!
Showcase Your Technical Skills:Since this role is all about data engineering, make sure to highlight your technical skills clearly. Mention your experience with streaming and batch pipelines, CI/CD processes, and any tools like Kafka and Delta Lake that you’ve worked with.
Apply Through Our Website:We encourage you to apply directly through our website for a smoother process. It helps us keep track of applications better and ensures you don’t miss out on any important updates from us!
How to prepare for a job interview at Searchability
✨Know Your Tech Stack
Make sure you’re well-versed in Databricks, Apache Spark, and Kafka. Brush up on your knowledge of real-time data ingestion and how these technologies work together. Being able to discuss specific projects where you've used these tools will show your practical experience.
✨Showcase Your Problem-Solving Skills
Prepare to discuss challenges you've faced in previous roles, especially related to building and optimising data pipelines. Use the STAR method (Situation, Task, Action, Result) to structure your answers and highlight your analytical thinking.
✨Understand CI/CD Processes
Since implementing CI/CD pipelines is crucial for this role, be ready to explain your experience with tools like GitHub Actions or Jenkins. Share examples of how you’ve improved workflow efficiency through automation and version control.
✨Ask Insightful Questions
Prepare thoughtful questions about the company’s data ecosystem and future projects. This shows your genuine interest in the role and helps you assess if the company aligns with your career goals. Think about asking how they measure success in their data initiatives.
