This is an exciting role that will allow you to be integral to the evolution of our cloud-based enterprise data platform. As a Data Engineer, you will be working on technical solutions, whilst supporting our existing set of SQL Servers that are predominantly used to feed data into a number of analytics products. This will help the business with data-led decision making to provide essential public services.
Your responsibilities will include:
- Designing efficient and robust data engineering projects within both on-premises and cloud-based data stacks that add business value and comply with architectural and data security requirements.
- Building, maintaining and supporting data engineering pipelines, with resolution of incidents.
- End-to-end delivery of data engineering solutions, including data quality, testing, peer reviewing, documenting and demonstrating of completed solutions to the team.
- Working closely with teams within data and other technical colleagues across the business.
- Remaining aware of new data engineering approaches, and be able to suggest how the latest research, techniques and approaches could be implemented to achieve business benefit
- Undertaking any other duties as required to meet the needs of the business.
- Substantial experience of data pipeline design and management
- Enthusiasm to add customer value whilst improving the effectiveness of the data engineering function
- Proactive problem solving and being able to effectively communicate solutions to both the data team, and to stakeholders
- Good understanding of database administration activities and principles
- Data transformation approaches
- Strong Python and SQL programming capabilities
- Data security approaches - e.g. permission models (AD, role-based etc.) and techniques to protect data in certain circumstances (e.g. encryption, masking)
Desirable Qualifications, Knowledge and Experience
- Familiarity with common agile project management principles, particularly in the context of data and analytical project delivery
- Good understanding of typical ingestion patterns (e.g. ETL, ELT) and their effective implementation with on-premises and cloud-based environments
- Knowledge of typical data modelling approaches (e.g. Kimball)
- Experience using the Microsoft data stack (especially Azure Data Lake, Azure Synapse, Azure Data Factory, Databricks)
- Experience of the full software/reporting lifecycle from planning and design through to deployment and maintenance
- Knowledge of the Microsoft BI Stack, including Analysis Services, Integration Services and Reporting Services, and Power BI
- Understanding of DevOps (or other source control technologies) - particularly code repositories and version control, containerisation methodologies, and application deployment practices (CI/CD pipelines)
Data Engineer employer: Dwr Cymru Cyf
Dwr Cymru Cyf is an exceptional employer that prioritises employee development and environmental responsibility, making it a rewarding place to work as an Apprentice Sewerage Network Technician. With a strong commitment to high standards of service and compliance, employees benefit from a collaborative work culture, opportunities for growth, and the chance to make a meaningful impact in their communities. Located in England, the company fosters a supportive environment where team members can thrive while contributing to sustainable practices.
