Senior Data Engineer

Job summary

Ready to take on a career-defining challenge? HDFT are looking for two Senior Data Engineers to play a pivotal role in a ground-up deployment of Microsoft Fabric. This is a rare opportunity to work on a completely greenfield project, shaping architecture and developing robust, scalable data pipelines using best-fit technologies.

What\’s in it for you?

– Immediate exposure to Microsoft Fabric be at the forefront of cutting-edge data engineering.

– Greenfield deployment minimal legacy constraints, just innovation.

– Critical role influence architecture, tooling, and best practices from day one.

– Technology-first environment work with modern cloud and data technologies to build something that lasts.

What you\’ll be doing

– Designing and implementing high-performance data pipelines for real time and batch processing.

– Architecting a future-proof data platform, leveraging Microsoft Fabric and other best-in-class tools.

– Collaborating with cross-functional teams to drive data strategy and governance.- Optimising performance, ensuring scalability, security, and reliability.

What we\’re looking for

– Proven experience in data modelling and data engineering with a strong grasp of modern cloud architectures.

– Expertise in data pipelines, ETL/ELT, and working with structured & unstructured data.

– Strong skills in SQL, Python, Spark, or other relevant technologies.

– Prior exposure to Microsoft Fabric is a huge plus but if you\’re eager tolearn, we\’ll get you up to speed.

Main duties of the job

1. Work closely with colleagues and stakeholders to develop new ideas, solutions and strategies for improving patient care and clinical outcomes.

2. Design, develop, and test robust, scalable data pipelines and architectures to support data ingestion, processing, and storage.

3. Conduct testing of solutions, review requirements and specifications, and define test conditions. Identify issues and risks associated with work. Analyse and report test activities and results.

4. Select and implement the appropriate technologies to deliver resilient, scalable and future-proofed data solutions using a highly developed specialist knowledge of analytical techniques and methods including an excellent understanding of NHS information and associated datasets.

5. Collaborate with Data Analysts, and other stakeholders to understand data requirements and translate them into technical solutions.

6. Develop and implement efficient ETL (Extract, Transform, Load) processes to integrate data from various sources into centralised data repositories.

7. Document data architecture, processes, and workflows for reference and knowledge sharing.

8. Utilise programming languages (e.g., C#, Python, SQL) and technologies (e.g., Apache Spark, SSIS, .NET) to manipulate and analyse data.

9. Participate in code reviews, version control (e.g., using Git), and other software development best practices to ensure code quality and maintainability.

About us

Harrogate and District NHS Foundation Trust (HDFT) cares for the population in Harrogate and the local area, as well as across North Yorkshire, Leeds and parts of the North East.

At HDFT we provide outstanding care to both our patients and our staff and we value Kindness, Integrity, Teamwork and Equality. If you\’re looking for an interesting and rewarding career in the NHS, where you can make a real difference to patients\’ lives, HDFT could be the place for you!

We offer colleagues:

• A supportive culture so colleagues can bring their whole selves to work.
• Staff Recognition – as well as regular appraisals, we recognise achievements with our Making a Difference & Team of the Month Awards, along with annual Colleague Recognition Awards.
• Employee Support and wellbeing – we offer a wide range of staff benefits including an Employee Assistance Programme, counselling service and a fast-track Physiotherapy service.

Please note:

• Due to high levels of interest, posts may close early if we reach a sufficient number of applications
• Visa sponsorship is only available for certain roles. Please ensure the role you are interested in meets the criteria for a visa before submitting your application.
• Please also be aware that the Trust does not offer relocation packages for Agenda for Change roles.
• All job offers are subject to satisfactory pre-employment checks.

Job description

Job responsibilities

Please see attached Job Description and Person Specification. If you have any questions, please contact Joe Ingle jolyon.ingle@nhs.net or 01423 553788

Job description
Job responsibilities

Please see attached Job Description and Person Specification. If you have any questions, please contact Joe Ingle jolyon.ingle@nhs.net or 01423 553788

Person Specification

Qualifications

Essential

• Master\’s degree level or able to demonstrate equivalent experience.
• Additional specialist procedural knowledge of systems development to enable design and development across a range of areas.
• Evidence of continued further professional development.

Desirable

• Current Microsoft certification in Data – Azure, Developer, Data Engineer, Data Analyst – field.

Experience

Essential

• Experience of exporting data from various source data systems, transforming data against required business rules and data models, or data storage and analysis requirements, and loading data in various storage platforms.
• Experience of working within and/or with multi-disciplinary teams, a proven track record of successful transformative and innovative working within health or health related settings.
• Significant experience of managing the complete systems development life cycle.
• Experience of using Git as a version control system to manage code artifacts, track change, and collaborate effectively with team members.
• Significant experience of relational database design as well as schema and query optimisation.

Desirable

• Previous NHS experience.
• Experience of managing staff.

Knowledge

Essential

• Extensive experience in data engineering within the healthcare sector, with a focus on NHS data systems.
• Proven track record of designing, developing, and maintaining large-scale data pipelines and ETL processes.
• In-depth knowledge of data modelling, database design, and data warehousing principles.
• Familiarity with healthcare data standards and compliance regulations.
• Significant experience of extracting data, manipulating, understanding, transforming, wrangling and cleaning NHS datasets.
• Experience with ICD, OPCS, SNOMED codes and the NHS Data Dictionary.
• Ability to write well-designed, testable, efficient SQL code which follows good coding standards.
• Experience in a data integration role with the ability to apply logical thinking using a wide range of data processing tools and techniques to build and integrations solution in a highly complex environment.
• Proven ability to transform data from raw into a usable format for users implementing highly complex rules.
• Experience working with open source technologies.
• Ability to work with large data sets with very complex information.

Desirable

• Good knowledge of Python for data engineering purposes.
• Knowledge of NHS strategies, policies and plans.

Skills and Aptitudes

Essential

• Fluent in SQL-based systems.
• Proficiency in programming languages commonly used in data engineering.
• Excellent problem-solving skills and the ability to work collaboratively in a multidisciplinary team.
• Communicate effectively with technical and non-technical stakeholders to support and host discussions within a multidisciplinary team, with potentially difficult dynamics, and be an advocate for the team externally to can manage differing perspectives.
• Excellent analytical, IT Keyboard and numerical skills – confident with statistics and their presentation.
• Excellent verbal and written communication (including presentation) skills.
• Ability to analyse highly complex issues where material is conflicting and drawn from multiple sources (verbal, written and numerical).

Other requirements

Essential

• Analytical mind-set: Ability to analyse complex data sets and derive meaningful insights to support decision-making.
• Able to make a connection between their work and the benefit to patients and the public.
• Capacity to adapt to evolving technologies, methodologies, and healthcare industry standards.
• Proven ability to manage projects, meet deadlines, and deliver high-quality results within scope and budget constraints.
• Demonstrate the ability to work under pressure. High level of Organisation, self-motivation and flexibility in approach and attitude.
• Values diversity and difference and operates with integrity and openness.
• Strong analytical skills and attention to detail, with a demonstrated ability to analyse business needs.
• Strong business engagement and stakeholder management skills.
• Strong problem solving acumen.
• Good verbal and written communication skills.
• Able to demonstrate a professional approach at all times, particularly when under pressure.
• Ability to adapt personal style in order to influence others and gain support for ideas.
• Good interpersonal skills, able to influence and engender collaboration.
• Well-developed delegation, people, and workload management skills.
• Well organised and able to work on own initiative.
• Present verbal, written and numerical information and issues, explaining complexities, to internal stakeholders.

Desirable

• Enthusiasm for self-development.

Person Specification
Qualifications

Essential

• Master\’s degree level or able to demonstrate equivalent experience.
• Additional specialist procedural knowledge of systems development to enable design and development across a range of areas.
• Evidence of continued further professional development.

Desirable

• Current Microsoft certification in Data – Azure, Developer, Data Engineer, Data Analyst – field.

Experience

Essential

• Experience of exporting data from various source data systems, transforming data against required business rules and data models, or data storage and analysis requirements, and loading data in various storage platforms.
• Experience of working within and/or with multi-disciplinary teams, a proven track record of successful transformative and innovative working within health or health related settings.
• Significant experience of managing the complete systems development life cycle.
• Experience of using Git as a version control system to manage code artifacts, track change, and collaborate effectively with team members.
• Significant experience of relational database design as well as schema and query optimisation.

Desirable

• Previous NHS experience.
• Experience of managing staff.

Knowledge

Essential

• Extensive experience in data engineering within the healthcare sector, with a focus on NHS data systems.
• Proven track record of designing, developing, and maintaining large-scale data pipelines and ETL processes.
• In-depth knowledge of data modelling, database design, and data warehousing principles.
• Familiarity with healthcare data standards and compliance regulations.
• Significant experience of extracting data, manipulating, understanding, transforming, wrangling and cleaning NHS datasets.
• Experience with ICD, OPCS, SNOMED codes and the NHS Data Dictionary.
• Ability to write well-designed, testable, efficient SQL code which follows good coding standards.
• Experience in a data integration role with the ability to apply logical thinking using a wide range of data processing tools and techniques to build and integrations solution in a highly complex environment.
• Proven ability to transform data from raw into a usable format for users implementing highly complex rules.
• Experience working with open source technologies.
• Ability to work with large data sets with very complex information.

Desirable

• Good knowledge of Python for data engineering purposes.
• Knowledge of NHS strategies, policies and plans.

Skills and Aptitudes

Essential

• Fluent in SQL-based systems.
• Proficiency in programming languages commonly used in data engineering.
• Excellent problem-solving skills and the ability to work collaboratively in a multidisciplinary team.
• Communicate effectively with technical and non-technical stakeholders to support and host discussions within a multidisciplinary team, with potentially difficult dynamics, and be an advocate for the team externally to can manage differing perspectives.
• Excellent analytical, IT Keyboard and numerical skills – confident with statistics and their presentation.
• Excellent verbal and written communication (including presentation) skills.
• Ability to analyse highly complex issues where material is conflicting and drawn from multiple sources (verbal, written and numerical).

Other requirements

Essential

• Analytical mind-set: Ability to analyse complex data sets and derive meaningful insights to support decision-making.
• Able to make a connection between their work and the benefit to patients and the public.
• Capacity to adapt to evolving technologies, methodologies, and healthcare industry standards.
• Proven ability to manage projects, meet deadlines, and deliver high-quality results within scope and budget constraints.
• Demonstrate the ability to work under pressure. High level of Organisation, self-motivation and flexibility in approach and attitude.
• Values diversity and difference and operates with integrity and openness.
• Strong analytical skills and attention to detail, with a demonstrated ability to analyse business needs.
• Strong business engagement and stakeholder management skills.
• Strong problem solving acumen.
• Good verbal and written communication skills.
• Able to demonstrate a professional approach at all times, particularly when under pressure.
• Ability to adapt personal style in order to influence others and gain support for ideas.
• Good interpersonal skills, able to influence and engender collaboration.
• Well-developed delegation, people, and workload management skills.
• Well organised and able to work on own initiative.
• Present verbal, written and numerical information and issues, explaining complexities, to internal stakeholders.

Desirable

• Enthusiasm for self-development.

Disclosure and Barring Service Check

This post is subject to the Rehabilitation of Offenders Act (Exceptions Order) 1975 and as such it will be necessary for a submission for Disclosure to be made to the Disclosure and Barring Service (formerly known as CRB) to check for any previous criminal convictions.
Certificate of Sponsorship

Applications from job seekers who require current Skilled worker sponsorship to work in the UK are welcome and will be considered alongside all other applications. For further information visit the .

From 6 April 2017, skilled worker applicants, applying for entry clearance into the UK, have had to present a criminal record certificate from each country they have resided continuously or cumulatively for 12 months or more in the past 10 years. Adult dependants (over 18 years old) are also subject to this requirement. .

Additional information
Disclosure and Barring Service Check

This post is subject to the Rehabilitation of Offenders Act (Exceptions Order) 1975 and as such it will be necessary for a submission for Disclosure to be made to the Disclosure and Barring Service (formerly known as CRB) to check for any previous criminal convictions.
Certificate of Sponsorship

Applications from job seekers who require current Skilled worker sponsorship to work in the UK are welcome and will be considered alongside all other applications. For further information visit the .

From 6 April 2017, skilled worker applicants, applying for entry clearance into the UK, have had to present a criminal record certificate from each country they have resided continuously or cumulatively for 12 months or more in the past 10 years. Adult dependants (over 18 years old) are also subject to this requirement. .

Employer details

Employer name

Harrogate and District NHS Foundation Trust
Address

Harrogate District Hospital

Lancaster Park Road

Harrogate

HG2 7SX

Employer\’s website #J-18808-Ljbffr