Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Lead the Data Engineering team to build and maintain automated data pipelines using cutting-edge cloud technologies.
- Company: Join a forward-thinking company focused on delivering world-class data solutions in a collaborative environment.
- Benefits: Enjoy flexible work arrangements, competitive pay, and opportunities for professional growth.
- Why this job: Be at the forefront of data innovation, working with modern tools and making a real impact.
- Qualifications: Senior experience in Data Engineering, proficiency in Python and SQL, and expertise in AWS and Databricks required.
- Other info: This is a contract position based in Wimbledon, UK, ideal for tech-savvy leaders.
The predicted salary is between 54000 - 84000 £ per year.
Job Type: Contract
Job Location: Wimbledon , UK
Job Description :
For this role, senior experience of Data Engineering and building automated data pipelines on IBM Datastage & DB2, AWS and Databricks from source to operational databases through to curation layer is expected using the latest cloud modern technologies where experience of delivering complex pipelines will be significantly valuable to how to maintain and deliver world class data pipelines.
Knowledge in the following areas essential:
- Databricks: Expertise in managing and scaling Databricks environments for ETL, data science, and analytics use cases.
- AWS Cloud: Extensive experience with AWS services such as S3, Glue, Lambda, RDS, and IAM.
- IBM Skills: DB2, Datastage, Tivoli Workload Scheduler, Urban Code
- Programming Languages: Proficiency in Python, SQL.
- Data Warehousing & ETL: Experience with modern ETL frameworks and data warehousing techniques.
- DevOps & CI/CD: Familiarity with DevOps practices for data engineering, including infrastructure-as-code (e.g., Terraform, CloudFormation), CI/CD pipelines, and monitoring (e.g., CloudWatch, Datadog).
- Familiarity with big data technologies like Apache Spark, Hadoop, or similar.
- ETL/ELT tools and creating common data sets across on-prem (IBMDatastage ETL) and cloud data stores
- Leadership & Strategy: Lead Data Engineering team(s) in designing, developing, and maintaining highly scalable and performant data infrastructures.
- Customer Data Platform Development: Architect and manage our data platforms using IBM (legacy platform) & Databricks on AWS technologies (e.g., S3, Lambda, Glacier, Glue, EventBridge, RDS) to support real-time and batch data processing needs.
- Data Governance & Best Practices: Implement best practices for data governance, security, and data quality across our data platform. Ensure data is well-documented, accessible, and meets compliance standards.
- Pipeline Automation & Optimisation: Drive the automation of data pipelines and workflows to improve efficiency and reliability.
- Team Management: Mentor and grow a team of data engineers, ensuring alignment with business goals, delivery timelines, and technical standards.
- Cross Company Collaboration: Work closely with all levels of business stakeholder including data scientists, finance analysts, MI and cross-functional teams to ensure seamless data access and integration with various tools and systems.
- Cloud Management: Lead efforts to integrate and scale cloud data services on AWS, optimising costs and ensuring the resilience of the platform.
- Performance Monitoring: Establish monitoring and alerting solutions to ensure the high performance and availability of data pipelines and systems to ensure no impact to downstream consumers.
#J-18808-Ljbffr
Big Data Lead employer: Hirewand
Contact Detail:
Hirewand Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Big Data Lead
✨Tip Number 1
Make sure to showcase your hands-on experience with Databricks and AWS in your conversations. Highlight specific projects where you've managed and scaled environments, as this will resonate well with the hiring team.
✨Tip Number 2
Prepare to discuss your approach to data governance and best practices. Be ready to share examples of how you've implemented security measures and ensured data quality in previous roles.
✨Tip Number 3
Demonstrate your leadership skills by discussing how you've mentored teams in the past. Share specific instances where you aligned team goals with business objectives to show your strategic thinking.
✨Tip Number 4
Familiarize yourself with the latest trends in cloud management and performance monitoring. Being able to discuss tools like CloudWatch or Datadog will show that you're up-to-date and ready to tackle the challenges of the role.
We think you need these skills to ace Big Data Lead
Some tips for your application 🫡
Highlight Relevant Experience: Make sure to emphasize your senior experience in Data Engineering, particularly with IBM Datastage, DB2, AWS, and Databricks. Use specific examples of complex data pipelines you've delivered to showcase your expertise.
Showcase Technical Skills: Clearly list your proficiency in Python, SQL, and any other relevant programming languages. Mention your familiarity with big data technologies like Apache Spark and Hadoop, as well as your experience with modern ETL frameworks.
Demonstrate Leadership Abilities: If you have experience leading teams, be sure to highlight this. Discuss how you've mentored data engineers and aligned team goals with business objectives, as this is crucial for the role.
Focus on Cloud Management: Detail your experience with AWS services such as S3, Glue, and Lambda. Explain how you've integrated and scaled cloud data services, optimized costs, and ensured platform resilience in previous roles.
How to prepare for a job interview at Hirewand
✨Showcase Your Technical Expertise
Be prepared to discuss your experience with IBM Datastage, DB2, AWS, and Databricks in detail. Highlight specific projects where you built automated data pipelines and how you tackled challenges using these technologies.
✨Demonstrate Leadership Skills
Since this role involves leading a team, share examples of how you've successfully managed data engineering teams in the past. Discuss your approach to mentoring and aligning team goals with business objectives.
✨Discuss Data Governance Practices
Prepare to talk about your understanding of data governance, security, and quality. Provide examples of how you've implemented best practices in previous roles to ensure compliance and accessibility of data.
✨Emphasize Collaboration Experience
This position requires working closely with various stakeholders. Be ready to share experiences where you collaborated with data scientists, finance analysts, or cross-functional teams to enhance data access and integration.
