At a Glance
- Tasks: Lead the design and optimisation of AWS-based data pipelines and platforms.
- Company: Join a forward-thinking tech company focused on Data & AI solutions.
- Benefits: Competitive salary, flexible working options, and opportunities for professional growth.
- Other info: Collaborative environment with mentorship opportunities and career advancement.
- Why this job: Make a real impact by solving complex data challenges with cutting-edge technology.
- Qualifications: Expertise in AWS, Python/PySpark, and strong data engineering skills required.
The predicted salary is between 70000 - 90000 £ per year.
The successful candidate will bring deep expertise in data engineering, distributed data processing, and cloud-native platforms, with a strong focus on AWS-based data ecosystems.
- Proven experience in data engineering and cloud-based platform delivery.
- Strong understanding of distributed data processing and scalable system design.
- Ability to lead delivery while remaining hands-on technically.
- Strong analytical, problem-solving, and communication skills.
- Experience working in client-facing and delivery-focused environments.
- Ability to mentor and develop engineering teams.
- Strong hands-on experience with:
- AWS cloud services, especially AWS Glue.
- Python / PySpark for large-scale data processing.
- SQL for querying, transformation, and validation.
- Configuration-driven development (e.g., YAML).
- Experience building and operating:
- Data pipelines.
- ETL/ELT workflows.
- Cloud-native data platforms.
- Familiarity with:
- Data lakes and Lakehouse concepts.
- Distributed processing frameworks (e.g., Apache Spark).
- Strong understanding of:
- ETL vs ELT patterns.
- Performance tuning and optimisation.
- Experience with:
- Version control (Git).
- CI/CD and DevOps practices.
We are seeking an accomplished and detail-oriented Lead Data Engineer – AWS to join our Data & AI practice. This role is critical in designing, building, and optimising end-to-end data pipelines and platforms, enabling scalable data processing, advanced analytics, and AI-driven solutions.
You will play a key role in ensuring data quality, integrity, performance, and reliability, supported by strong engineering and testing practices. As a senior practitioner, you will collaborate with architects, engineers, and analysts to deliver secure, scalable, and high-performing data solutions, leveraging technologies such as AWS Glue, Python/PySpark, SQL, and configuration-driven frameworks (e.g., YAML).
You will thrive in a collaborative, client-facing environment, with a passion for solving complex technical challenges, ensuring delivery excellence, and driving modernisation through cloud-native engineering practices.
- Act as a senior engineer within data engineering and cloud platform initiatives, supporting delivery across complex transformation programmes.
- Collaborate with architects and stakeholders to define and implement scalable AWS-based data solutions.
- Contribute to solution design, estimation, and delivery planning.
- Lead engineering workstreams and ensure high-quality technical delivery.
- Design, build, and optimise scalable data pipelines and data processing frameworks on AWS.
- Develop and maintain ETL/ELT pipelines using:
- AWS Glue.
- Python / PySpark.
- SQL.
- Configuration-driven frameworks (e.g., YAML).
- Implement robust data ingestion, transformation, and processing patterns.
- Build reusable data services, components, and frameworks.
- Define and implement testing strategies for data pipelines, ensuring reliability and accuracy.
- Validate data processing workflows using:
- Python / PySpark transformations.
- SQL-based validation logic.
- Configuration-driven orchestration.
- Develop automated testing, monitoring, and alerting solutions.
- Ensure:
- Data completeness.
- Data accuracy.
- Consistent transformation behaviour.
- Drive improvements in observability and pipeline resilience.
- Lead development on AWS services including:
- S3-based data lakes.
- Supporting services within the AWS data ecosystem.
- Support implementation of modern data architectures, including data lakes and Lakehouse-style platforms.
- Optimise pipelines and jobs for performance, scalability, and cost efficiency.
Data Transformation & Modelling:
- Define and implement data transformation logic aligned to business requirements.
- Support data modelling approaches for analytics and platform use cases.
- Ensure consistency, usability, and quality across data assets and pipelines.
- Collaborate with:
- Solution Architects.
- Data Engineers.
- Analysts and ML engineers.
- Provide technical leadership and mentoring to engineers within the team.
- Promote engineering best practices, automation, and reusable solutions.
- Contribute to engineering standards, documentation, and knowledge sharing.
- Ensure data quality, integrity, and reliability across data platforms.
- Implement and enforce secure coding and data handling practices.
- Support compliance with:
- GDPR.
- Regulated environment standards (where applicable).
- Contribute to monitoring, auditing, and operational processes.
Lead Data Engineer (AWS Data) in London employer: NTT DATA
As a Lead Data Engineer at our innovative firm, you will thrive in a dynamic and collaborative work culture that prioritises employee growth and development. We offer competitive benefits, including flexible working arrangements and opportunities for continuous learning, all while being part of a forward-thinking team dedicated to delivering cutting-edge AWS-based data solutions. Join us in a location that fosters creativity and technical excellence, where your contributions will directly impact our clients and the future of data engineering.
StudySmarter Expert Advice🤫
We think this is how you could land Lead Data Engineer (AWS Data) in London
✨Tip Number 1
Network like a pro! Get out there and connect with folks in the data engineering space. Attend meetups, webinars, or even just grab a coffee with someone who’s already in the field. You never know when a casual chat could lead to your next big opportunity!
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving AWS Glue, Python, and SQL. Having tangible examples of your work can really set you apart during interviews and discussions.
✨Tip Number 3
Prepare for technical interviews by brushing up on your problem-solving skills. Practice coding challenges related to data processing and distributed systems. Being hands-on and ready to tackle real-world scenarios will impress potential employers.
✨Tip Number 4
Don’t forget to apply through our website! We’re always on the lookout for talented individuals like you. Plus, it’s a great way to ensure your application gets the attention it deserves from our hiring team.
We think you need these skills to ace Lead Data Engineer (AWS Data) in London
Some tips for your application 🫡
Tailor Your CV:Make sure your CV highlights your experience with AWS, Python, and data engineering. We want to see how your skills match the job description, so don’t be shy about showcasing your relevant projects!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you’re passionate about data engineering and how your background makes you a perfect fit for our team. Keep it engaging and personal!
Showcase Your Problem-Solving Skills:In your application, give examples of how you've tackled complex data challenges in the past. We love seeing analytical thinking and creativity in action, so share those success stories!
Apply Through Our Website:We encourage you to apply directly through our website. It’s the best way for us to receive your application and ensures you’re considered for the role. Plus, it’s super easy!
How to prepare for a job interview at NTT DATA
✨Know Your AWS Inside Out
Make sure you brush up on your knowledge of AWS services, especially AWS Glue. Be ready to discuss how you've used these tools in past projects and how they can be applied to build scalable data pipelines.
✨Showcase Your Problem-Solving Skills
Prepare to share specific examples of complex technical challenges you've faced and how you solved them. This role requires strong analytical skills, so demonstrating your thought process will impress the interviewers.
✨Be Hands-On with Python and SQL
Since this position involves a lot of hands-on work with Python/PySpark and SQL, be prepared to discuss your experience with these languages. You might even be asked to solve a coding problem on the spot, so practice beforehand!
✨Emphasise Team Collaboration
This role is client-facing and requires collaboration with various stakeholders. Share examples of how you've successfully worked in teams, mentored others, and contributed to a positive team environment.
