Remote Data Engineer in Poole

Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK’s most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services in the past two years alone. This work has not only reached over 50 million citizens but also achieved considerable savings for the taxpayer, amounting to over £25 million. At Scrumconnect, we foster a community of talented consultants who thrive on collaboration, sharing knowledge, and continuous learning to address and solve complex challenges. Our mission is to combine advanced software engineering, human-focused design, and data-driven insights to deliver unparalleled service to our clients.

We are looking to fill the position of Data Engineer to help develop and maintain data products. Data Engineering teams are responsible for the delivery and operational stability of the data products built and provide ongoing support for those products. Data Engineers work within, and contribute to, the overall data development lifecycle process as part of multi-functional Agile delivery teams focused on one or more products.

Role Responsibilities:

• Design, build, and operate simple, repeatable ETL data pipelines within distributed processing environments and cloud platforms.
• Develop and produce prototyped then productionized code that can be deployed across a range of ETL, data validation, and other data production processes.
• Develop understanding of the native tooling of one or more of: GCP, Azure, AWS.
• Intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R).
• Develop code for a range of data products including data matching, rule development, scans, operational outputs.
• Participate in development and maintenance of in-house code libraries.
• Undertake unit testing to support common code development.
• Review business requirements to ensure they are clear and robust, and transform requirements into reusable production-ready code and/or effective data models.
• Understand the key principles of database design and resolve technical problems in databases, data processes, data products, and services.
• Apply correct techniques in normalizing data and building robust relational structures in database products.
• Undertake basic data analysis for data profiling, QA, or problem resolution.
• Support queries from end users on data quality issues affecting data production.
• Undertake source system analysis and data profiling to confirm data quality and ensure accurate metadata.
• Work with experts to develop validation frameworks for both simple and complex data sources.
• Communicate with customers to update on progress or confirm requirements.
• Assist with the development of data pipelines, products, and automation processes.
• Engage with other professional communities to identify emerging and cross-community issues affecting your role.
• Describe technical, data, pipeline, and production issues to colleagues of different specialisms.
• Communicate within the team and across teams to monitor expectations around delivery of data engineering, products, and pipelines.
• Familiar with developing data pipelines and products for very large volume 'big data' series using a range of native engineering tools and practices.
• Experience predicting and advising on technology changes in the engineering toolset(s) and platform(s).
• Work with the team in implementing the transition to modern data platforms including Data Warehouse, Lakehouse.
• Ability to manage relationships with non-technical colleagues and work in a collaborative, inclusive way.
• Able to write clean, efficient and well documented code for data processing tasks to a specification.
• Investigate problems in systems, processes and services.
• Use metadata repositories to complete complex tasks such as data and systems integration impact analysis.
• Show an awareness of opportunities for innovation with new tools and uses of data.

Typical Data Engineering Experience required (5+ yrs):

• Knowledge and experience of Azure/AWS Cloud data solution provision.
• Proficient in SQL.
• Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI.
• Enterprise-scale experience with ETL tools (Informatica and or similar).
• Experience of data modelling and transforming raw data into datasets and extracts.
• Experience of working in a large project/scale complex organisation.
• Experience in Agile.
• Ability to analyse and collect information and evidence, identify problems and opportunities.
• Experience of building team capability through role modelling, mentoring, and coaching.
• Understanding of the principles of data processing in a distributed and/or cloud platform.
• Able to use Git for code version control.
• Experience of one or more programming/coding languages listed: Python/PySpark, SQL, Proc SQL, NoSQL, MySQL, SQLite, Spark SQL, Hive SQL, PostgreSQL, SAS, Scala, RegEx, Java, R.

Desired Qualifications:

• Certifications in AWS, Azure, Databricks, or related technologies.
• Experience with public sector data initiatives and compliance requirements.
• Knowledge of machine learning and artificial intelligence concepts.

Your working time at Scrumconnect will be split between multiple locations, including from our HQ and hub locations, client site or home. Travel requirements vary in frequency.