At a Glance
- Tasks: Lead data acquisition and design Azure-based data pipelines for impactful solutions.
- Company: Dynamic London-based company focused on data governance and engineering excellence.
- Benefits: Competitive salary, career growth opportunities, and a collaborative work environment.
- Other info: Onsite work three days a week with a focus on continuous improvement.
- Why this job: Shape high-value data solutions and make a real impact in various sectors.
- Qualifications: Strong expertise in Azure services, Python, SQL, and proven leadership experience.
The predicted salary is between 9750 - 10400 £ per year.
Salary: £9,750 - 10,400 per year
Requirements
- Strong expertise in Azure data services, including ADLS, ADF, Databricks, Synapse, Event Hub, and Kafka.
- Hands-on experience with Python, SQL, and Spark/PySpark.
- Experience with data modeling, metadata, lineage tooling, performance and cost optimization.
- Proven leadership experience managing engineering teams.
- Experience working in Agile and DevOps environments, including CI/CD and test automation.
- Exposure to energy trading systems such as ETRM or Endur, market data sources such as ICE, Platts, Bloomberg, and SAP.
- A degree in Computer Science, Engineering, or equivalent practical experience.
- Azure Data Engineer or Databricks certifications are desirable.
Responsibilities
- Lead the data acquisition function by defining ingestion strategy and scalable batch and streaming data patterns.
- Design and deliver Azure-based data pipelines and platform solutions.
- Standardise connectors, schemas, controls, CI/CD, infrastructure as code, and automated testing.
- Lead onboarding of enterprise and market data sources, ensuring service levels and observability are in place.
- Partner with Product Owners, Architects, and business stakeholders on data design and governance.
- Act as the domain Data Manager for lineage, metadata, quality, access and master data ownership.
- Ensure compliance with data lifecycle, audit, records management and governance standards.
- Drive engineering best practices, operational excellence and continuous improvement.
- Manage capacity planning, skills development and vendor contributions.
- Translate end-user requirements into scalable engineering solutions for Trading, Risk, Finance and Operations.
Technologies
- Azure
- CI/CD
- Databricks
- DevOps
- Kafka
- Marketing
- Python
- PySpark
- SAP
- SQL
- Spark
- Cloud
We are a hiring company based in London offering an initial contract through to the end of the year, with onsite working required three days per week. We are looking for a Data Engineering Lead to help shape and deliver reliable, compliant, and high-value data solutions across Trading, Risk, Finance, and Operations. In this role, you will join a team focused on data governance, engineering excellence, and scalable Azure-based solutions. We also work with businesses across the UK in Finance, Marketing, IT, and Engineering, and we are committed to helping people find roles where they can make an impact and grow their careers.
Data Engineering Lead in London employer: Tatton Recruitment
As a leading employer in London, we pride ourselves on fostering a collaborative and innovative work culture that empowers our employees to excel in their careers. With a strong focus on professional development, we offer numerous growth opportunities and the chance to work with cutting-edge technologies in the data engineering field. Our commitment to operational excellence and continuous improvement ensures that you will be part of a dynamic team dedicated to making a meaningful impact across various sectors, including Trading, Risk, Finance, and Operations.
