At a Glance
- Tasks: Lead data engineering projects using Databricks and Microsoft Fabric to deliver impactful solutions.
- Company: Join a Microsoft Solutions Partner recognised for positive change and innovation.
- Benefits: Competitive salary, professional development, and a collaborative work environment.
- Other info: Dynamic team culture with opportunities for growth and creativity.
- Why this job: Shape the future of data engineering while mentoring junior talent and driving real business value.
- Qualifications: Experience in Azure data solutions and strong communication skills are essential.
The predicted salary is between 63000 - 77000 £ per year.
Simpson Associates transforms raw data into actionable insights that drive positive change.
Our Microsoft data expertise, our specialist sector knowledge, plus our innovative and trusted advice and guidance are just some of the reasons clients choose to work with us.
Our mission is to help purpose-led organisations from within the public and private sectors to harness data as a lever for change and enable them to realise business value more quickly.
We provide the full range of services to support organisations on their data transformation journey.
From advisory support and data strategy, to developing Data & AI solutions, right through to providing a range of managed services.
We are a Microsoft Solutions Partner for Cloud & AI Platforms and Security, holding Specialisations in AI Platform on Microsoft Azure, Build AI Apps on Microsoft Azure, Analytics on Microsoft Azure and Data Warehouse Migration to Microsoft Azure.
Previously, Cloud & AI platforms were individual pillars for Data & AI, Digital & App innovation and Infrastructure.
But it's not just about the badges.
We are proud to be recognised as the winner of the 2024 Microsoft Community Response Partner of the Year award, reflecting our dedication to using technology for positive change.
We are also a Databricks partner, and an IBM Gold Partner, specialising in Cognos Analytics and Planning Analytics.
With offices in York and Sheffield, and a team based throughout the UK – we champion creativity, innovation and collaboration in the workplace.
The Role
The Lead Data Engineer plays a pivotal role within the Delivery capabilities delivering high-quality solutions using Databricks, Microsoft Fabric and Azure-native technologies.
The role may involve running projects independently or leading workstreams within larger and more complex engagements under the direction of a Principal Consultant.
The Lead Data Engineer is the primary point of contact for customers during delivery and is expected to combine strong technical capability with excellent consultancy, communication and leadership skills.
You will be responsible for shaping solutions, guiding delivery teams, and ensuring projects are delivered to time, cost and quality expectations, while supporting the development of junior consultants.
Key Responsibilities
- Lead the end-to-end delivery of data engineering projects, either independently or as part of larger multi-disciplinary programmes.
- Act as the primary service delivery contact for customers, building trusted relationships and providing clear technical leadership.
- Demonstrate deep technical expertise in data engineering, understanding how solutions fit within wider customer architectures and business requirements.
- Work with customers to identify, challenge and prioritise business and technical requirements, including running design and discovery workshops.
- Translate functional and non-functional requirements into robust, scalable data platform architectures and technical designs.
- Design and implement data engineering solutions using Microsoft Fabric and Databricks, following best-practice approaches.
- Produce clear technical documentation and communicate designs using appropriate visualisations (e. g. architecture diagrams, data flows).
- Accurately estimate delivery effort, contributing to proposals and addressing variances between pre-sales estimates and detailed requirements.
- Convert estimates into delivery plans and manage these throughout the project lifecycle, working with Project Management and Scheduling as required.
- Provide hands-on technical delivery alongside other consultants when appropriate.
- Take overall responsibility for solution and delivery quality, including code reviews, testing strategies and customer acceptance.
- Identify, manage and mitigate delivery risks and issues early, drawing on experience to pre-empt likely challenges.
- Ensure appropriate deployment, handover and transition into support or managed services.
- Mentor and support junior consultants, contributing to capability development within the Data Engineering team.
- Contribute to intellectual property, reusable assets, marketing content, case studies and blogs.
Skills and Attributes Required
- Technical Skills
- Strong experience designing and delivering modern data engineering solutions on Azure.
- Hands-on expertise with Microsoft Fabric, including Lakehouse, Data Engineering and orchestration capabilities.
- Strong experience with Databricks, including Spark-based data processing and optimisation.
- Solid understanding of data modelling, data integration patterns, and ELT/ETL pipelines.
- Experience working with Azure data services such as Azure Data Lake, Azure SQL and related platform components.
- Knowledge of Dev Ops and deployment practices for data platforms (e. g. CI/CD, version control, automated testing).
- Consulting & Delivery Skills
- Proven experience delivering projects within agreed time, cost and quality constraints.
- Strong ability to communicate complex technical concepts clearly to both technical and non-technical stakeholders.
- Experience leading delivery in both waterfall and agile environments and adapting approach to suit the customer and engagement.
- Commercial awareness and consultative mindset, with the ability to identify additional opportunities and add value for customers.
- Personal Attributes
- Confident, proactive problem-solver who remains calm and reassuring in customer-facing situations.
- Strong organisational and time management skills, with the ability to manage multiple priorities simultaneously.
- A proven self-starter, comfortable working independently while contributing positively to team delivery.
- Commitment to continuous professional development and maintaining up-to-date technical and consultancy skills.
- Willingness to mentor others and contribute to a collaborative, knowledge-sharing culture.
Due to the nature of our work, successful applicants will need to successfully complete the relevant security clearance.
Simpson Associates reserves the right to close the recruitment process at any time.
#J-18808-Ljbffr
Lead Databricks Engineer employer: Simpson Associates
Simpson Associates is an exceptional employer, offering a dynamic work culture that fosters collaboration and innovation in the healthcare sector. With a strong focus on employee growth, you will have access to professional development opportunities while working alongside industry leaders to drive impactful change within NHS England and other key stakeholders. Located in the UK, our team enjoys a supportive environment that values creativity and strategic thinking, making it an ideal place for those looking to make a meaningful contribution in healthcare data and AI.
StudySmarter Expert Advice🤫
We think this is how you could land Lead Databricks Engineer
✨Get Involved in Data Science Meetups
Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like Simpson Associates!
✨Show Off Your Projects
Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Lead Databricks Engineer at Simpson Associates.
✨Leverage Professional Networks
Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like Simpson Associates.
✨Apply Directly through Our Website
When you find a suitable opening like Lead Databricks Engineer at Simpson Associates, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!
We think you need these skills to ace Lead Databricks Engineer
Some tips for your application 🫡
Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!
Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!
Craft a Tailored Cover Letter:For a full-time role at Simpson Associates, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.
Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at Simpson Associates. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!
How to prepare for a job interview at Simpson Associates
✨Brush Up on Your Statistics
For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!
✨Showcase Your Projects
Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!
✨Get Comfortable with Python and R
Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at Simpson Associates!
✨Prepare for Case Studies
Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.
