At a Glance
- Tasks: Design and build innovative Databricks solutions while collaborating with backend teams.
- Company: Join the AA, a diverse and inclusive company that values your unique background.
- Benefits: Enjoy 25 days annual leave, free breakdown membership, and discounts on various services.
- Other info: Dynamic workplace with excellent learning opportunities and support for your development.
- Why this job: Make a real impact in a cutting-edge data environment and grow your career.
- Qualifications: Hands-on experience with Azure Databricks and strong Python skills required.
The predicted salary is between 55000 - 65000 £ per year.
We are an equal opportunities employer and welcome applications from everyone. The AA values diversity and actively seeks people from diverse backgrounds to join us and become part of an inclusive company where you can be yourself, be empowered to be your best and feel like you truly belong.
Responsibilities
- Designing, building and operating production-grade Databricks Lakehouse solutions, including structured streaming pipelines using Python and PySpark
- Owning and evolving Unity Catalog‑based governance, ensuring secure, discoverable and well‑managed data assets
- Developing and maintaining event‑driven data pipelines, integrating closely with backend engineering teams
- Implementing and supporting CI/CD pipelines in Azure DevOps to enable reliable, automated Databricks deployments
- Creating high‑quality, analytics‑ready datasets that deliver actionable business insight at scale
- Proactively improving performance, reliability, automation and observability across the Databricks data platform
Qualifications
- Significant, hands‑on experience with Azure Databricks in a production, enterprise‑scale environment
- Proven experience building Spark/Databricks pipelines using Python and PySpark
- Strong experience with structured streaming and event‑driven architectures
- Practical experience implementing and operating Unity Catalog
- Solid understanding of Lakehouse design principles, including dimensional and analytical modelling
- Experience building and maintaining CI/CD pipelines, ideally using Azure DevOps
- Confidence working with large‑scale data, performance tuning, and troubleshooting complex pipelines
- Strong debugging and problem‑solving skills, including the ability to diagnose and resolve issues in Spark/Databricks pipelines; hands‑on experience configuring and managing Databricks environments, including cluster setup, dependency management and package installation
- Designed and built Databricks pipelines end‑to‑end
- Made architectural decisions within Databricks environments
- Worked with Spark internals, optimisation techniques and cluster configuration
- Operated Databricks solutions in live, business‑critical contexts
Benefits
- 25 days annual leave plus bank holidays and holiday buying scheme
- Worksave pension scheme with up to 7% employer contribution
- Free AA breakdown membership from Day 1 plus 50% discount for family and friends
- Discounts on AA products, including car and home insurance
- Employee discount scheme that gives you access to a car salary sacrifice scheme and other discounts on healthcare, shopping, holidays and more
- Company‑funded life assurance
- Diverse learning and development opportunities to support you to progress in your career
- Dedicated Employee Assistance Programme and a 24/7 remote GP service for you and your family
The AA is a well‑loved brand with a wide range of driver services and a growing data infrastructure that is modernising on a cutting‑edge cloud platform. If you’re passionate about data and eager to make a significant impact, we invite you to apply for this experienced Databricks Data Engineer role.
Data Engineer (Databricks) in Hampshire employer: The Automobile Association
The AA is an exceptional employer that champions diversity and inclusivity, ensuring every employee feels empowered and valued. With a strong focus on professional growth, employees benefit from extensive learning opportunities, a supportive work culture, and a comprehensive benefits package including generous leave, pension contributions, and health discounts. Located in a dynamic environment, the AA offers the chance to work with cutting-edge technology while making a meaningful impact in the data landscape.
Contact Details:
The Automobile Association Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land Data Engineer (Databricks) in Hampshire
✨Tip Number 1
Network like a pro! Reach out to folks in the industry, attend meetups, and connect with people on LinkedIn. You never know who might have the inside scoop on job openings or can refer you directly.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your Databricks projects, especially those involving Python and PySpark. This gives potential employers a taste of what you can do and sets you apart from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on your technical knowledge and problem-solving skills. Practice common data engineering scenarios and be ready to discuss your experience with CI/CD pipelines and event-driven architectures.
✨Tip Number 4
Don’t forget to apply through our website! It’s the best way to ensure your application gets seen. Plus, we love seeing candidates who are genuinely interested in joining our diverse and inclusive team.
We think you need these skills to ace Data Engineer (Databricks) in Hampshire
Some tips for your application 🫡
Tailor Your CV:Make sure your CV reflects the skills and experiences that match the Data Engineer role. Highlight your hands-on experience with Azure Databricks, Python, and PySpark to show us you’re the right fit!
Craft a Compelling Cover Letter:Use your cover letter to tell us why you’re passionate about data engineering and how your background aligns with our values of diversity and inclusion. This is your chance to let your personality shine through!
Showcase Your Projects:If you've worked on any relevant projects, especially those involving Databricks or CI/CD pipelines, make sure to mention them. We love seeing real examples of your work and how you’ve tackled challenges in the past.
Apply Through Our Website:We encourage you to apply directly through our website for the best chance of getting noticed. It’s super easy, and we can’t wait to see your application come through!
How to prepare for a job interview at The Automobile Association
✨Know Your Databricks Inside Out
Make sure you brush up on your Azure Databricks knowledge. Be ready to discuss your hands-on experience with building Spark/Databricks pipelines using Python and PySpark. Prepare examples of how you've designed and operated production-grade solutions, as this will show your practical understanding of the platform.
✨Showcase Your Problem-Solving Skills
Be prepared to tackle some technical questions or scenarios during the interview. Think about complex issues you've resolved in Spark/Databricks pipelines and how you approached debugging and performance tuning. Sharing specific examples will demonstrate your strong problem-solving abilities.
✨Understand the Importance of CI/CD
Since the role involves implementing CI/CD pipelines in Azure DevOps, make sure you can explain your experience with this process. Discuss how you've enabled reliable, automated deployments and any challenges you've faced along the way. This will highlight your ability to contribute to a smooth development workflow.
✨Emphasise Your Collaborative Spirit
The job requires close integration with backend engineering teams, so be ready to talk about your teamwork experiences. Share how you've collaborated with others to develop event-driven data pipelines and how you ensure that data assets are secure and well-managed. This will show that you're not just a tech whiz but also a great team player.
