Lead Data Engineer in London

As a Lead Data Engineer or architect at Made Tech, you will play a pivotal role in helping public sector organisations become truly data-led, by equipping them with robust data platforms. You will also join a data team on its mission to get data knowledge and skills out of silos and embedded into delivery teams. You will help implement efficient data pipelines and storage.

Key responsibilities

• Define, shape and perfect data strategies in central and local government.
• Help public sector teams understand the value of their data, and make the most of it.
• Establish yourself as a trusted advisor in data-driven approaches using public cloud services like AWS, Azure and GCP.
• Contribute to recruitment efforts and take on line management responsibilities.

Skills, knowledge and expertise

• Proficiency in Git (including Github Actions) and able to explain the benefits of different branch strategies.
• Strong experience in IaC and able to guide how one could deploy infrastructure into different environments.
• Knowledge of handling and transforming various data types (JSON, CSV, etc.) with Apache Spark, Databricks or Hadoop.
• Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes).
• Ability to create data pipelines on a cloud environment and integrate error handling within these pipelines.
• Understand how to create reusable libraries to encourage uniformity across multiple data pipelines.
• Able to document and present end-to-end diagrams to explain a data processing system on a cloud environment.
• Some knowledge of how to present diagrams (C4, UML, etc.).
• Enthusiasm for learning and self-development.
• Experience of working on agile delivery-led projects and can apply agile practices such as Scrum, XP, Kanban.
• Can own the cloud infrastructure underpinning data systems through a DevOps approach.
• Design and implement efficient data transformation processes at scale, both in batch and streaming use cases.
• Skilled Data Engineer who has delivered data platforms.
• Knowledge of SOLID, DRY and TDD principles and how to practically implement these into a project.
• Demonstrate a commercial mindset when on projects to grow accounts organically with senior stakeholders.
• Experience to improve resilience into a project by checking for software vulnerabilities and implementing appropriate testing strategies (unit integration, data quality, etc.).
• Skilled at offering guidance on how to implement a robust DevOps approach in a data project.
• Comfortably talk about tools needed such as DataOps in areas such as orchestration, data integration and data analytics.

Desirable experience

• Working at a technology consultancy.
• Working with Docker and virtual environments as part of the development and CI/CD process.
• Working with senior stakeholders to gather requirements and keep them engaged.
• Experience in working with a team of engineers using techniques such as pair programming or mob programming.
• Working with data scientists to productionise advanced data deliverables, such as machine learning models.
• Working knowledge of statistics.
• Working with multidisciplinary digital and technology teams.
• Working within the public sector.

Job benefits

Life at Made Tech: We are committed to building a happy, inclusive and diverse workforce. You can get a sense of what it’s like working here from our blog, where we talk about mental health, communities of practice and neurodiversity.

Benefits: We are always listening to our growing teams and evolving the benefits available to our people. Here are some of our most popular benefits:

• 30 days Holiday - we offer 30 days of paid annual leave.
• Flexible Parental Leave - we offer flexible parental leave options.
• Remote Working - we offer part-time remote working for all our staff.
• Paid counselling - we offer paid counselling as well as financial and legal advice.
• Pension - 7% employer matched.

An increasing number of our customers are specifying a minimum of SC (security check) clearance in order to work on their projects. As a result, we are looking for all successful candidates for this role to have eligibility. Eligibility for SC requires 5 years’ UK residency and 5 year employment history (or back to full-time education). Ideally, no more than 30 consecutive days spent outside the UK within the last five years.