At a Glance
- Tasks: Design and maintain Azure-based data pipelines for impactful teacher development insights.
- Company: Join the National Institute of Teaching, dedicated to improving education through data.
- Benefits: Enjoy 27 days holiday, flexible working, and lifestyle discounts.
- Other info: Supportive environment with continuous professional development opportunities.
- Why this job: Make a real difference in education while developing your data engineering skills.
- Qualifications: Experience in data engineering, Azure Data Factory, and strong T-SQL skills required.
The predicted salary is between 56667 - 56667 £ per year.
Hybrid, within commuting distance of Birmingham, Bristol, London, Blackburn, Redcar or Doncaster
£56,667 + benefits (+ London weighting, if applicable)
We are seeking a motivated and enthusiastic Senior Data Engineer to join our TED team and play an important role in building and maintaining the technical infrastructure that supports the NIoT and TED.
About Us
The National Institute of Teaching (NIoT) has an unswerving commitment to high-quality, evidence-informed teacher education, and is on a mission to improve teacher and leader development across the education system.
A key part of that mission is our Teacher Education Dataset (TED), an ambitious data and research initiative focused on building better evidence about teacher development and what makes teaching impactful. By bringing together education, research and digital infrastructure, TED will support new insight into how teachers develop, improve and thrive across their careers.
About the Role
We are seeking a motivated and enthusiastic Senior Data Engineer to join our TED team and play an important role in building and maintaining the technical infrastructure that supports the NIoT and TED. Working at the intersection of data engineering, data and research, the postholder will help develop secure, reliable and scalable systems that enable robust analysis and contribute to high-quality, evidence-informed improvement across the sector.
Corporate Responsibilities
- To ensure that the responsibilities of the role are carried out in a way which reflects the mission and the values of the NIoT.
- To be aware of and observe all policies, procedures, working practices and regulations, and in particular to comply with policies relating to Equal Opportunities, Health and Safety, Confidentiality, Data Protection and Financial Regulations, reporting any concerns to an appropriate person.
- To comply with all reasonable management requests.
Key Responsibilities
- Design, build and maintain Azure-based data pipelines to support the ingestion, transformation and curation of TED datasets.
- Develop and maintain Azure SQL databases and research-facing schemas across raw, transformed and curated data layers.
- Write efficient, secure and well-structured T-SQL to support ingestion, validation, transformation and optimisation.
- Contribute to the development and ongoing refinement of a Common Data Model across multiple education data providers.
- Integrate and harmonise data from school MIS and EdTech platforms, including Bromcom, Arbor and Wonde.
- Build and maintain robust validation, logging, auditing and reconciliation processes to improve data quality and trust.
- Support secure data handling, including pseudonymisation, access control and compliance with data-sharing agreements and security requirements.
- Work closely with analysts, researchers, architects and partners to deliver scalable, analytics-ready data solutions.
- Contribute to agile planning, technical design, documentation and operational handover.
Essential Criteria
- Proven experience in data engineering, including building and maintaining production-grade data pipelines.
- Strong experience with Azure Data Factory and Azure SQL Server. Strong T-SQL skills, including schema design, stored procedures and query optimisation.
- Experience implementing validation, monitoring, logging and reconciliation controls within data pipelines.
- Strong understanding of secure data architecture and the handling of sensitive personal data.
- Experience with Azure DevOps, version control and CI/CD practices.
- Strong analytical and problem‑solving skills, with excellent attention to detail.
- Ability to communicate effectively and work successfully in a multidisciplinary team.
Desirable Criteria
- Experience with education datasets, school MIS systems or EdTech platforms.
- Experience with common data models, longitudinal datasets or research-oriented data infrastructure.
- Experience with Power BI, Fabric or related analytics technologies.
- Experience in a public sector, education or high-governance data environment.
- Interest in applying data engineering to improve education and teacher development.
Key Benefits
- Generous Annual Leave – 27 days’ holiday a year (plus 8 bank holidays).
- Flexible Working – we offer flexible start and end working times, with hybrid working in place for all roles.
- Pension – Entry to the Local Government Pension Scheme.
- Salary Sacrifice Schemes – we offer salary sacrifice schemes for bikes, cars and tech!
- Lifestyle Benefits – We offer discounts on gyms, cinema, retail and much more!
- Parenthood Leave – We offer above the statutory minimum for maternity, adoption and paternity leave.
- Working Environment – We have a stimulating, supportive and rewarding working environment with a dedicated team of likeminded professionals.
- Support: Our Employee Assistance Programme (EAP) provides confidential support for employees on personal or work-related matters.
- Development: We fully support the development of our staff and ensure that you have high level of continuous professional development.
To ensure a fair and robust selection process, we expect all application responses to be based on your own original thoughts, skills and experience. Our system includes functionality to identify AI-generated content, and applications where responses appear not to be the candidate’s own work may be disregarded.
This post requires a satisfactory enhanced disclosure from the Disclosure and Barring Service (DBS) with a Children’s Barred List Check and Occupational Health Check is required as a condition of employment.
Senior Data Engineer The National Institute of Teaching (NIoT) in Birmingham employer: Society of Research Software Engineering
The British Library is an exceptional employer, offering a vibrant work culture that fosters innovation and collaboration within the Digital Research Team. With a commitment to inclusivity and diversity, employees benefit from flexible working arrangements, generous holiday allowances, and opportunities for professional growth in a prestigious environment dedicated to advancing digital scholarship and public outreach.
Contact Details:
Society of Research Software Engineering Recruitment Team
StudySmarter Expert Advice🤫
We think this is how you could land Senior Data Engineer The National Institute of Teaching (NIoT) in Birmingham
✨Get Involved in Data Science Meetups
Tap into local data science meetups or workshops to connect with fellow enthusiasts and professionals. These events are goldmines for networking, and sometimes even lead directly to job openings at companies like Society of Research Software Engineering!
✨Show Off Your Projects
Start building a public portfolio showcasing your data science projects on platforms like GitHub or personal websites. Highlight unique analyses or models you've developed. This not only demonstrates your skills but also gets your name out there for roles like Senior Data Engineer The National Institute of Teaching (NIoT) at Society of Research Software Engineering.
✨Leverage Professional Networks
Join professional bodies related to data science, like the Data Science Society or similar organisations. Getting involved can lead to mentorship opportunities and insider knowledge about full-time positions at companies like Society of Research Software Engineering.
✨Apply Directly through Our Website
When you find a suitable opening like Senior Data Engineer The National Institute of Teaching (NIoT) at Society of Research Software Engineering, make sure to apply directly through our website. It gives you an edge and shows you're keen to join our team. Plus, who doesn’t love a direct application? It’s easier than navigating through job boards!
Some tips for your application 🫡
Show Off Your Projects:In the world of data science, your projects can speak volumes about your skills. Make sure to showcase a few key projects in your CV or portfolio, especially those that highlight your ability to work with data sets, build models, or use relevant tools like Python, R, or SQL. Don’t forget to include links to any GitHub repositories if applicable!
Quantify Your Achievements:Employers love numbers! When drafting your CV, highlight your achievements with quantifiable results. For instance, mention how your data analysis led to a certain percentage increase in efficiency or revenue at a previous job or project. These details can really make your application pop!
Craft a Tailored Cover Letter:For a full-time role at Society of Research Software Engineering, your cover letter should reflect your passion for data science and your excitement about the specific projects or values of the company. Dive into why you’re a good fit, how your skills align with their needs, and any unique perspectives you can bring to the team.
Stand Out with Relevant Courses and Certifications:Although experience talks, relevant courses or certifications can be your ticket to impressing hiring managers at Society of Research Software Engineering. Mention any standout courses you've completed that equipped you with essential skills, such as machine learning certifications or data visualisation courses. This shows your commitment to continuously developing your skills in the field!
How to prepare for a job interview at Society of Research Software Engineering
✨Brush Up on Your Statistics
For a data science role, we need to seriously sharpen our statistics skills. Get ready to tackle technical questions on probability distributions, hypothesis testing, and regression analysis. These are often the bread and butter of data science interviews, so don't just skim over them!
✨Showcase Your Projects
Prepare a killer portfolio showcasing your data science projects. We should include details about the datasets used, the tools and techniques applied, and the impact of your findings. If we can walk them through a particularly challenging project or a cool visualisation that had real-world implications, it’ll really make us stand out!
✨Get Comfortable with Python and R
Most data science positions require us to be proficient in programming languages like Python and R. We should practice common libraries like pandas, NumPy, and scikit-learn, and be ready for live coding exercises or algorithm questions. Showing off our coding chops can really impress the interviewers at Society of Research Software Engineering!
✨Prepare for Case Studies
Expect to encounter real-world case studies during the interview. We might be asked how we’d approach a data problem or analyse a dataset to extract insights. It's essential to think out loud and demonstrate our problem-solving process so that the interviewer can see our logical thinking in action.
