At a Glance
- Tasks: Lead the design and development of innovative data solutions and pipelines.
- Company: Join 1st Central, a leading insurance company leveraging smart data and technology.
- Benefits: Enjoy competitive salary, flexible working options, and opportunities for professional growth.
- Other info: Collaborative environment with a focus on mentorship and career advancement.
- Why this job: Make a real impact by transforming data into actionable insights and solutions.
- Qualifications: 10+ years in data engineering with expertise in Azure and strong coding skills.
The predicted salary is between 80000 - 100000 £ per year.
Overview
We're 1st Central, a market leading insurance company utilising smart data and technology at pace.
We are recruiting a Principal Data Engineer to serve as a key bridge between business opportunities and technical solutions.
Responsibilities
- Provide technical leadership on data solutions across both project delivery and ongoing data products, defining the technical direction of the data platform and solution design.
- Collaborate closely with architects to ensure alignment with overall platform patterns and standards.
- Set and enforce coding, low level design, and ingestion standards for the data platform.
- Design and build high complexity, secure, governed, and high performance data pipelines from on premise, off premise, internal, and external data sources.
- Define data quality, observability, and self testing patterns to proactively identify processing issues.
- Transform data into lake storage, database models, reporting structures, and analytics applications.
- Develop physical data models that optimise storage and meet business needs.
- Perform unit testing of own code and peer testing of others' code to ensure quality and integrity.
- Document data platform components to provide transparency and understandability.
- Coach and mentor senior and junior data engineers.
- Create high complexity BI solutions, including data marts, semantic layers, and visualisation solutions in tools such as Power BI.
Qualifications
- 10+ years of experience designing and building end to end data solutions.
- Extensive experience with Microsoft Azure data platforms, including Azure Data Lake Storage Gen2, Azure Synapse Analytics, and Databricks.
- Strong knowledge of Delta Lake lakehouse architectures, Medallion patterns, and scalable batch and streaming workloads.
- Proficiency in Python, SQL, and experience with PySpark, Spark SQL, and other data engineering languages.
- Hands on experience with Git based workflows, CI/CD pipelines (YAML), and infrastructure as code tools such as Terraform or Bicep.
- Experience designing data models (Kimball dimensional modelling) and implementing data quality, cleansing, and validation logic.
- Deep understanding of Azure security, network isolation, identity and access management, and cost management.
- Excellent coaching, mentoring, and leadership skills, with a proven ability to guide engineering teams.
- Strong problem solving skills and the ability to simplify complex problems.
- Familiarity with Agile practices such as Scrum or Kanban.
