Senior Data Engineer - Platforms and Tooling in Sheffield

The Home Office works to build a safe, fair and prosperous UK. We achieve this through our work on counter-terrorism, policing, crime, drugs policy, immigration and passports. The Science, Technology, Research and Analysis (STAR) Group sits at the heart of the Home Office. The group ensures the best evidence and analysis are used to address the strategic issues facing the department. We also provide the science, technology, data, and international insights and collaboration capabilities to drive delivery.

As part of the Home Office Analysis & Insight (HOAI) directorate, the Performance Reporting & Analysis Unit (PRAU) is responsible for delivering comprehensive reporting and performance analysis to support the Migration & Borders Mission and its operational capabilities. The Unit provides management information (MI) to Ministers, the Executive Committee, policy and operational leads, and analytical teams ensuring they have the data required for informed decision-making. Additionally, PRAU produces data for publication and external dissemination. The Unit offers expert advice across the Department on optimising the use of performance information and plays a leading role in the ongoing transformation of MI systems and capabilities.

Within the Migration & Borders Mission and Capabilities, PRAU is spearheading a significant transformation in technology and processes to enhance the delivery of MI services. This initiative focuses on three primary areas for the development of MI services:

• Data Engineering teams responsible for ingesting complex datasets and applying business rules.
• MI teams aligned with the Mission and Capabilities, delivering MI services to those specific domains.
• A central “hub” functioning as a centre of excellence, offering guidance, setting standards, and providing technology and support to both Data Engineering and MI-producing teams.

The new position within this group will supervise data engineering activities necessary for maintaining and advancing the software platforms integral to the MI Service. This includes enabling the integration of new data sources and managing the monitoring of existing data feeds, utilisation, and governance.

Key responsibilities include day-to-day system monitoring and troubleshooting, system administration, and leadership on engineering projects aimed at improving the functionality of existing tools and systems. As part of a growing team, this role presents an opportunity to influence best practices across the unit and department. Flexibility will be critical to effectively support and advocate for this transformation, alongside fulfilling the specific duties outlined below.

In addition, the department intends to transition from legacy tools to Microsoft Fabric. You will contribute to the design, migration, and ongoing management of a major government platform leveraging this evolving suite of technologies.

As a Senior Data Engineer in the Platforms and Tooling team, you will maintain, manage and upgrade the Vantage MI platform with modern cloud technologies such as Microsoft Azure Synapse, SQL, Power BI and Microsoft Fabric. You will be a key team member, responsible for supervising:

• Technical Administration.
• Analytics and MI platform upgrades, efficiency and innovation.
• Day-to-day issue management and resolution.
• Help to inform the Tooling and Data Strategies.

You will establish and detail key support processes which enable the Platforms and Tooling team to scale and automate their system maintenance and support activities, improving the service for our end users and developers. The MI Platforms and Tooling team will collaborate closely with colleagues in PRAU and Home Office Digital, and with teams elsewhere in the Department.

Your main day-to-day responsibilities will be:

• Designing, building and maintaining cloud platforms and tools to enable MI teams to work effectively.
• Working with engineering teams to ensure that MI platforms and tooling are integrated with other systems and that they are compliant with security and regulatory standards.
• Work with the 1st and 2nd line support teams to handle incidents that they pass to your team and proactively monitor common issues with a view to resolving the underlying cause.
• Document key platform and tooling configurations and procedures.
• Provide technical support and deliver training to help other engineers deploy high quality MI services.
• Develop monitoring and observability tools to agreed standards for both technical and non-technical stakeholders and consumers.
• Adhere to an ‘accessible by default’ approach in all delivery and documentation activities, working closely with colleagues to implement and enforce accessibility across Power BI reporting and supporting documentation.
• Supporting colleagues with the broader rollout and adoption of MS Power BI across the organisation.
• Work with the wider organisation to enable the ingestion of data into the Vantage platform from various sources, in a secure and cost-effective way.
• Supporting in the successful delivery of completed data loads for customers, Data Engineers and Data Scientists and assisting in the development of new data load programmes.
• Identifying areas for cost savings and/or greater efficiency.

Working Pattern: PRAU is geographically spread across four primary locations (London, Liverpool, Sheffield and Croydon), with most staff working in line with the Department’s hybrid working arrangements which would see the post holder spend 60% of their week in the office. This role is available on a full-time basis with the option of compressed hours working. This role is also suitable for part-time working hours, with a minimum requirement to work 4 days /30 hours per week due to business requirements.

Travel: Candidates will need to agree a base location of either Croydon, Liverpool, Salford or Sheffield with a requirement for periodic attendance at other Home Office locations.