At a Glance
- Tasks: Lead the design and development of a cutting-edge Databricks data platform.
- Company: Join a forward-thinking retail insight team with a focus on innovation.
- Benefits: Enjoy a competitive salary, remote work flexibility, and career growth opportunities.
- Other info: Mentor a small team and collaborate with diverse product teams.
- Why this job: Make a real impact by shaping data architecture for analytics and AI.
- Qualifications: Strong experience in data engineering, Databricks, Python, and SQL required.
The predicted salary is between 65000 - 65000 £ per year.
Salary: £65,000
Type: Full time, permanent
Hours: 37.5 per week
Location: Remote or Hybrid UK, with occasional Richmond office attendance (a couple of times per month)
Overview
We are looking for a Lead Data Engineer to join the Retail Insight Data and Engineering team and own the architecture and development of our Databricks environment. This is a senior, hands-on role where you will design and build scalable data infrastructure that underpins our products, while guiding a small team of engineers and defining platform standards, data architecture and engineering best practices supporting analytics, machine learning and AI across the business.
Key responsibilities
- Lead the design, development and optimisation of a Databricks based data platform, ensuring reliability, performance and scalability.
- Design enterprise scale data architectures on Databricks and cloud platforms such as Azure, AWS or GCP.
- Define lakehouse standards, data modelling approaches and governance frameworks.
- Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies.
- Establish security, access control, data quality and compliance standards.
- Optimise performance, scalability and cloud cost management.
- Lead and mentor a small team of data engineers, setting technical standards and reviewing code.
- Partner with data science, analytics and product teams to ensure the platform supports their work.
- Own data quality, governance and observability across pipelines.
- Support AI and ML enablement including feature stores and model deployment patterns.
- Translate business requirements into technical solutions.
- Contribute to platform roadmap, tooling selection and continuous improvement.
Required experience
- Strong experience as a senior or lead data engineer building and scaling production data platforms.
- Deep hands-on expertise in Databricks including Spark, Delta Lake and Databricks Workflows.
- Python and SQL proficiency.
- Infrastructure as code experience using Terraform, Bicep or CloudFormation.
- Cloud data platform experience, ideally Azure including Azure Data Factory, ADLS Gen2 and Azure DevOps.
- Strong understanding of data modelling, ETL and ELT patterns, distributed processing.
- Experience with security, governance and data platform operations.
- Experience leading or mentoring engineers, with strong stakeholder management.
- Rigorous approach to data quality and testing.
- Databricks certification.
Desirable
- Unity Catalog, MLflow and Databricks Asset Bundles.
- dbt or similar transformation frameworks.
- Retail or CPG data such as POS, inventory, supply chain.
- Streaming technologies such as Kafka or Event Hubs.
Education
Degree in computer science, data engineering or related field, or equivalent practical experience.
Lead Data Engineer in Kingston upon Thames employer: FBI &TMT
Join a forward-thinking company that values innovation and collaboration, offering a dynamic work culture where your contributions as a Lead Data Engineer will directly impact our retail insights. With the flexibility of remote or hybrid working arrangements and opportunities for professional growth, you will thrive in an environment that encourages mentorship and continuous improvement while working with cutting-edge technologies like Databricks. Enjoy the unique advantage of occasional office interactions in Richmond, fostering team spirit and collaboration without compromising on work-life balance.
StudySmarter Expert Advice🤫
We think this is how you could land Lead Data Engineer in Kingston upon Thames
✨Get Involved in Data Science Meetups
Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like FBI &TMT!
✨Show Off Your Projects
Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Lead Data Engineer at FBI &TMT.
✨Leverage Professional Networks
Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like FBI &TMT.
✨Apply Directly through Our Website
When you find a suitable opening like Lead Data Engineer at FBI &TMT, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!
We think you need these skills to ace Lead Data Engineer in Kingston upon Thames
Some tips for your application 🫡
Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!
Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!
Craft a Tailored Cover Letter:For a full-time role at FBI &TMT, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.
Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at FBI &TMT. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!
How to prepare for a job interview at FBI &TMT
✨Brush Up on Your Statistics
For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!
✨Showcase Your Projects
Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!
✨Get Comfortable with Python and R
Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at FBI &TMT!
✨Prepare for Case Studies
Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.
