Find a job 
Featured Companies 
Featured Universities 
For Companies At a Glance
- Tasks: Lead a team to build and optimise automated data pipelines using cutting-edge cloud technologies.
- Company: Join a forward-thinking company in Wimbledon, focused on innovative data solutions.
- Benefits: Enjoy flexible working options and opportunities for professional growth.
- Why this job: Be at the forefront of data engineering, making a real impact with your skills.
- Qualifications: Senior experience in Data Engineering, with expertise in Databricks, AWS, and programming languages like Python and SQL.
- Other info: Collaborate with diverse teams and drive data governance best practices.
The predicted salary is between 54000 - 84000 £ per year.
Job Type: Contract
Job Location: Wimbledon , UK
Job Description :
For this role, senior experience of Data Engineering and building automated data pipelines on IBM Datastage & DB2, AWS and Databricks from source to operational databases through to curation layer is expected using the latest cloud modern technologies where experience of delivering complex pipelines will be significantly valuable to how to maintain and deliver world class data pipelines.
Knowledge in the following areas essential:
- Databricks: Expertise in managing and scaling Databricks environments for ETL, data science, and analytics use cases.
- AWS Cloud: Extensive experience with AWS services such as S3, Glue, Lambda, RDS, and IAM.
- IBM Skills: DB2, Datastage, Tivoli Workload Scheduler, Urban Code
- Programming Languages: Proficiency in Python, SQL.
- Data Warehousing & ETL: Experience with modern ETL frameworks and data warehousing techniques.
- DevOps & CI/CD: Familiarity with DevOps practices for data engineering, including infrastructure-as-code (e.g., Terraform, CloudFormation), CI/CD pipelines, and monitoring (e.g., CloudWatch, Datadog).
- Familiarity with big data technologies like Apache Spark, Hadoop, or similar.
- ETL/ELT tools and creating common data sets across on-prem (IBMDatastage ETL) and cloud data stores
- Leadership & Strategy: Lead Data Engineering team(s) in designing, developing, and maintaining highly scalable and performant data infrastructures.
- Customer Data Platform Development: Architect and manage our data platforms using IBM (legacy platform) & Databricks on AWS technologies (e.g., S3, Lambda, Glacier, Glue, EventBridge, RDS) to support real-time and batch data processing needs.
- Data Governance & Best Practices: Implement best practices for data governance, security, and data quality across our data platform. Ensure data is well-documented, accessible, and meets compliance standards.
- Pipeline Automation & Optimisation: Drive the automation of data pipelines and workflows to improve efficiency and reliability.
- Team Management: Mentor and grow a team of data engineers, ensuring alignment with business goals, delivery timelines, and technical standards.
- Cross Company Collaboration: Work closely with all levels of business stakeholder including data scientists, finance analysts, MI and cross-functional teams to ensure seamless data access and integration with various tools and systems.
- Cloud Management: Lead efforts to integrate and scale cloud data services on AWS, optimising costs and ensuring the resilience of the platform.
- Performance Monitoring: Establish monitoring and alerting solutions to ensure the high performance and availability of data pipelines and systems to ensure no impact to downstream consumers.
#J-18808-Ljbffr
Contact Detail:
Hirewand Recruiting Team
StudySmarter Expert Advice 🤫
We think this is how you could land Big Data Lead
✨Tip Number 1
Familiarise yourself with the specific tools and technologies mentioned in the job description, such as IBM Datastage, DB2, AWS, and Databricks. Having hands-on experience or relevant projects to discuss can significantly boost your chances during interviews.
✨Tip Number 2
Showcase your leadership skills by preparing examples of how you've successfully led data engineering teams in the past. Be ready to discuss your approach to mentoring team members and aligning projects with business goals.
✨Tip Number 3
Network with professionals in the big data field, especially those who have experience with AWS and Databricks. Engaging in relevant online communities or attending industry events can help you gain insights and potentially get referrals.
✨Tip Number 4
Prepare to discuss your experience with data governance and best practices. Being able to articulate how you've implemented data quality measures and compliance standards will demonstrate your capability to maintain high standards in data management.
We think you need these skills to ace Big Data Lead
Some tips for your application 🫡
Tailor Your CV: Make sure your CV highlights your senior experience in Data Engineering, particularly with IBM Datastage, DB2, AWS, and Databricks. Use specific examples of complex pipelines you've delivered to demonstrate your expertise.
Craft a Strong Cover Letter: In your cover letter, emphasise your leadership skills and experience in managing data engineering teams. Discuss how you have implemented best practices for data governance and pipeline automation in previous roles.
Showcase Relevant Skills: Clearly list your proficiency in programming languages like Python and SQL, as well as your familiarity with big data technologies such as Apache Spark and Hadoop. This will help the hiring team see your fit for the role.
Highlight Collaboration Experience: Mention any past experiences where you worked closely with cross-functional teams, including data scientists and finance analysts. This shows your ability to collaborate effectively and ensure seamless data access across the organisation.
How to prepare for a job interview at Hirewand
✨Showcase Your Technical Expertise
Be prepared to discuss your experience with IBM Datastage, DB2, AWS, and Databricks in detail. Highlight specific projects where you built automated data pipelines and the challenges you overcame.
✨Demonstrate Leadership Skills
Since this role involves leading a team, be ready to share examples of how you've successfully managed teams in the past. Discuss your approach to mentoring and aligning team goals with business objectives.
✨Discuss Data Governance Practices
Familiarise yourself with best practices for data governance and security. Be prepared to explain how you ensure data quality and compliance in your previous roles.
✨Prepare for Scenario-Based Questions
Expect scenario-based questions that assess your problem-solving skills. Think about how you would handle specific challenges related to pipeline automation, performance monitoring, or cross-company collaboration.
