At a Glance
- Tasks: Run and maintain production data pipelines using Databricks and AWS.
- Company: Join Mastek, a global leader in AI and digital services.
- Benefits: Enjoy competitive salary, hybrid work, and generous leave.
- Other info: Be part of a diverse team with clear career progression.
- Why this job: Make a real impact on large-scale digital transformation projects.
- Qualifications: Experience in Python, PySpark, SQL, and data pipeline support.
The predicted salary is between 63000 - 77000 £ per year.
- Data Engineer
- Support and Maintenance
- Databricks
- AWS
- Python
- Py Spark
- SQL
- Work Pattern: Hybrid
- Travel Requirements: Mostly Remote - Travel to customer and Mastek offices as required
- Security Clearance: Not Mandatory - Good to Have
- Immediate Joiners
- About Mastek
Mastek Limited is a global provider of enterprise AI, digital, and cloud services, enabling clients to achieve measurable and sustainable returns on their technology investments.
With a presence in over 40 countries and a workforce of nearly 5,000 professionals, Mastek delivers innovative technology solutions that help organisations accelerate transformation and unlock business value.
Through our Lead with AI approach, we integrate intelligence across our solutions and operations, enabling ethical, scalable, and domain-driven AI adoption.
We partner with industry leaders including Oracle, Salesforce, Microsoft, AWS, Snowflake, and Databricks, delivering outcomes across sectors such as Healthcare, Financial Services, Retail, Manufacturing, Higher Education, and the Public Sector.
Guided by our core values of Trust, Value, and Velocity, we foster a culture of collaboration, innovation, continuous learning, and career growth.
About the role
We are looking for a hands-on Data Engineer to run and maintain production data pipelines on Databricks and AWS.
You will keep scheduled processing reliable, resolve incidents, deliver permanent fixes and improve the performance of existing data services.
The role combines L2 and L3 production support with data engineering.
You will work with service management, platform teams, data owners and downstream users to keep data accurate and available within agreed service levels and delivery deadlines.
Key responsibilities
- Monitor scheduled jobs, ingestion processes and downstream loads.
Identify failures, delays and unusual data volumes, assess the business impact and prioritise action against delivery deadlines.
- Own incidents from investigation through recovery and closure.
Diagnose issues in Python, Py Spark, SQL, Databricks jobs and AWS integrations, and provide clear progress updates and timely escalation.
- Recover failed or incomplete processing through controlled retries, reruns and backfills.
Check dependencies and reconcile outputs to prevent duplicate records, missing data or inconsistent downstream results.
- Investigate data quality issues, schema changes and late or missing source files.
Trace problems through pipeline stages and agree corrective action with source owners and data consumers.
- Maintain and improve existing Python, Py Spark and SQL code.
Deliver defect fixes and small enhancements, using code review and appropriate tests to protect existing processing rules.
- Tune Spark jobs and SQL queries, including joins, partitioning, data skew and memory usage.
Review Databricks compute usage and S3 file layouts to reduce processing time and unnecessary cost.
- Troubleshoot S3 access, IAM permissions, encrypted data access and connectivity with platform and security teams.
Support runtime and dependency upgrades, controlled releases, rollback plans and post-release validation.
- Automate health checks, reconciliation and routine support tasks.
Improve alerts and track job failures, data freshness, processing duration and recurring incidents to guide preventive maintenance.
- Maintain runbooks, dependency maps, root cause analyses and support handovers.
Check that new or changed pipelines have monitoring, support ownership and recovery procedures before accepting them into service.
Essential skills and experience
- Proven experience supporting and maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis and permanent remediation.
- Strong Python, Py Spark and SQL skills, with the ability to understand unfamiliar code, diagnose defects and make maintainable changes.
- Practical Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI and performance troubleshooting.
- Experience with Spark SQL, Delta Lake, Parquet and Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files.
- Hands-on AWS experience, particularly S3, IAM and Cloud Watch, with an understanding of role-based access, KMS encryption and diagnosing data access failures.
- Understanding of batch and incremental processing, job dependencies, restartability, late-arriving data and safe reprocessing without duplicate results.
- Experience implementing data quality checks and source-to-target reconciliation, and investigating missing, duplicate or incorrect records in large datasets.
- Working knowledge of Git, code reviews, CI/CD and controlled production deployments, including testing, rollback and release validation.
- Experience working within incident, problem and change management processes.
Able to prioritise by business impact, maintain clear support records and communicate with technical and non-technical colleagues.
Desirable experience
- AWS Glue, Lambda, Step Functions or other orchestration services; Linux and shell scripting; Terraform; and delivery tools such as Git Lab CI or Jenkins.
- Monitoring and alerting automation, capacity planning, cloud cost optimisation and disaster recovery exercises for data services.
- Supporting healthcare, public sector or other regulated data environments, including access controls, audit trails and secure handling of sensitive information.
- Transitioning pipelines from project delivery into live support, with practical knowledge transfer and documented operational acceptance.
- Working expectations
Take ownership of issues through to resolution, work calmly during service disruption and raise risks early.
Follow agreed access and change controls, and coordinate maintenance and recovery with the teams affected.
Any on-call or out-of-hours support arrangements will be agreed for the role.
Why Join Mastek?
- Work on large-scale digital transformation programmes across public and private sector clients.
- Collaborate with highly skilled and diverse teams across multiple disciplines.
- Gain exposure to modern technologies, frameworks, and delivery methodologies.
- Access ongoing learning, professional development, and certification opportunities.
- Contribute to meaningful projects that have real-world impact.
- Be part of an inclusive culture that values innovation, teamwork, and continuous improvement.
- Enjoy clear career progression opportunities within a global technology organisation.
- Our UK Benefits Include
- Competitive salary and annual bonus opportunities
- Hybrid and flexible working options
- Generous annual leave entitlement plus bank holidays
- Company pension scheme with employer contributions
- Private medical insurance (for self and dependants)
- Life assurance cover
- Employee Assistance Programme (EAP)
- Wellbeing initiatives and gym support
- Learning and development programmes, certifications, and professional training
- Employee recognition and reward schemes
- Cycle to Work scheme
- Employee referral bonus programme
- Enhanced family-friendly policies
- Diversity & Inclusion
At Mastek, we are committed to building a diverse workforce and creating an inclusive environment where everyone can succeed.
We welcome applications from all backgrounds and are proud to be an equal opportunities employer.
We believe diversity drives innovation and are dedicated to fostering a workplace where every employee feels valued, respected, and empowered to make an impact.
Apply today and help shape the future with Mastek.
Lead Data engineer in Leeds employer: Mastek
Mastek is an exceptional employer that fosters a collaborative and innovative work culture, where Senior Data & AI Architects can thrive while leading transformative projects across diverse sectors. With a strong emphasis on employee growth, Mastek offers ample opportunities for professional development and career advancement, all within a dynamic environment that values creativity and strategic thinking. Located in a vibrant area, the company provides unique advantages such as access to cutting-edge technology and a supportive network of industry experts.
StudySmarter Expert Advice🤫
We think this is how you could land Lead Data engineer in Leeds
✨Get Involved in Data Science Meetups
Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like Mastek!
✨Show Off Your Projects
Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Lead Data engineer at Mastek.
✨Leverage Professional Networks
Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like Mastek.
✨Apply Directly through Our Website
When you find a suitable opening like Lead Data engineer at Mastek, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!
We think you need these skills to ace Lead Data engineer in Leeds
Some tips for your application 🫡
Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!
Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!
Craft a Tailored Cover Letter:For a full-time role at Mastek, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.
Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at Mastek. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!
How to prepare for a job interview at Mastek
✨Brush Up on Your Statistics
For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!
✨Showcase Your Projects
Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!
✨Get Comfortable with Python and R
Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at Mastek!
✨Prepare for Case Studies
Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.