Principal Data Engineer in London

Principal Data Engineer in London

London Temporary 60000 - 84000 £ / year (est.) Home office (partial)
O

At a Glance

  • Tasks: Build innovative data pipelines and lead AI projects to transform higher education.
  • Company: Join the Office for Students, the independent regulator of higher education in England.
  • Benefits: Competitive salary, flexible working, and a supportive culture focused on staff development.
  • Why this job: Make a real impact on students' experiences while working with cutting-edge technology.
  • Qualifications: Strong Python skills and experience with Azure Databricks are essential.
  • Other info: Dynamic team environment with opportunities for career growth and innovation.

The predicted salary is between 60000 - 84000 £ per year.

Base pay range up to £72,000. 12 months fixed term contract. Location: Bristol.

About us: We are the independent regulator of higher education in England. We aim to ensure that every student, whatever their background, has a fulfilling experience of higher education that enriches their lives and careers. We care about supporting and developing our staff and want to create a welcoming and engaging working environment which is free from discrimination.

The analytics & data engineering team are a newly formed team in the OfS. The team is there to make sure the analyst team and subsequently the business has the right information to carry out the functions of the organisation.

Why work for the OfS: The OfS's Data function is currently undergoing an exciting and major transformation and we’re looking for a talented data engineer to help lead the change. We’re moving away from a legacy on-premises platform to an Azure Databricks platform, unlocking new possibilities for automation and innovation. This is your opportunity to help develop new robust data products from the ground up, set best practice and introduce AI into the organisation to improve how we support the higher education sector and the needs of students.

Key responsibilities:

  • Building and developing reusable pipelines for analytics and AI projects.
  • Pushing for innovation within the platform to enable great efficiencies and detailed insights and outputs.
  • Leading key relationships between IT and Data to grow the platform and release new capabilities.
  • Deploying production AI models with automated monitoring from the data pipeline to the model runs and outputs.
  • Extracting, Loading & Transforming (ELT) data sets from a variety of data sources across the OfS enterprise technology stack.
  • Monitoring the execution of data workflows, including identifying and mitigating risks, setting service level indicators and configuring alerts.
  • Adopting data governance best practice when processing raw data to develop, test and maintain datasets.
  • Developing coding standards for Python programming to be used across the Data function.
  • Supporting continuous data quality improvements across the organisation.
  • Applying dimensional data modelling concepts and practices across the OfS data.
  • Advocating and supporting an evidence-led culture across the organisation.

Essential experience & skills:

  • Strong Python programming knowledge, ideally PySpark.
  • Strong knowledge of the Azure Databricks platform and associated functionalities.
  • Adaptable, with a willingness to work flexibly as the needs of the organisation evolve.
  • Working well within a team, and able to work closely with internal and external stakeholders.
  • An ability to take a logical and analytical approach, and to take a pragmatic, collaborative approach to solving problems.
  • Adept at communicating technical concepts to a non-technical audience.
  • Awareness of the modern data stack and associated methodologies.

Working for us: The OfS regulates the higher education sector on behalf of all students. We value diversity and the wealth of perspectives, experience and ideas that it brings to our work, and we strive to embed equality of opportunity in everything we do. We recruit based on fair and open competition and welcome applications from candidates regardless of age, disability, race and ethnicity, gender reassignment, marriage and civil partnership, pregnancy and maternity, religion or belief, sex or sexual orientation.

Flexibility is important to the OfS, as we recognise that it can enable people to work more effectively. A set of organisation-wide principles shape our approach to flexible and hybrid working.

Important dates:

  • Closing date for applications: 8th September 2025.
  • Interviews: From 15 September 2025.

Principal Data Engineer in London employer: Office for Students

The Office for Students is an exceptional employer, offering a dynamic and inclusive work culture that prioritises the development and well-being of its staff. Located in Bristol, this role as a Principal Data Engineer provides a unique opportunity to be at the forefront of transforming higher education data practices, with access to innovative technologies like Azure Databricks. Employees benefit from flexible working arrangements, a commitment to diversity, and the chance to contribute meaningfully to the educational experiences of students across England.
O

Contact Detail:

Office for Students Recruiting Team

StudySmarter Expert Advice 🤫

We think this is how you could land Principal Data Engineer in London

✨Tip Number 1

Network like a pro! Reach out to people in the industry, especially those at the Office for Students. A friendly chat can open doors that applications alone can't.

✨Tip Number 2

Prepare for interviews by brushing up on your Python and Azure Databricks skills. Be ready to showcase how you can innovate and improve data processes – they love that!

✨Tip Number 3

Show your passion for higher education! Talk about how your work can impact students' experiences. They want to see that you care about their mission.

✨Tip Number 4

Don’t forget to apply through our website! It’s the best way to ensure your application gets noticed and shows you’re serious about joining the team.

We think you need these skills to ace Principal Data Engineer in London

Python Programming
PySpark
Azure Databricks
Data Pipeline Development
ELT (Extract, Load, Transform)
Data Governance Best Practices
Dimensional Data Modelling
Automated Monitoring
Data Quality Improvement
Analytical Skills
Communication Skills
Stakeholder Management
Adaptability
Problem-Solving Skills

Some tips for your application 🫡

Tailor Your Application: Make sure to customise your CV and cover letter for the Principal Data Engineer role. Highlight your Python programming skills and experience with Azure Databricks, as these are key for us at the OfS.

Showcase Your Experience: Don’t just list your previous jobs; explain how your past experiences relate to the responsibilities mentioned in the job description. We want to see how you can contribute to our data transformation journey!

Be Clear and Concise: Keep your application straightforward and to the point. Use clear language to communicate your technical skills and how they align with our goals. Remember, we appreciate a logical and analytical approach!

Apply Through Our Website: We encourage you to submit your application through our official website. It’s the best way for us to receive your details and ensures you’re considered for this exciting opportunity at the OfS!

How to prepare for a job interview at Office for Students

✨Know Your Tech Inside Out

Make sure you brush up on your Python programming skills, especially with PySpark. Familiarise yourself with Azure Databricks and its functionalities, as you'll likely be asked to demonstrate your understanding of these tools during the interview.

✨Showcase Your Problem-Solving Skills

Prepare to discuss specific examples where you've tackled complex data challenges. Highlight your logical and analytical approach, and be ready to explain how you collaborated with teams to find solutions. This will show that you're not just technically skilled but also a team player.

✨Communicate Clearly

Practice explaining technical concepts in simple terms. You might encounter non-technical stakeholders during the interview, so being able to convey your ideas clearly will set you apart. Think about how you can make your experience relatable to someone without a technical background.

✨Understand the Organisation's Goals

Research the Office for Students and their mission. Be prepared to discuss how your skills and experiences align with their goals, particularly in enhancing the student experience through data. Showing that you care about their mission will demonstrate your commitment to the role.

Principal Data Engineer in London
Office for Students
Location: London

Land your dream job quicker with Premium

You’re marked as a top applicant with our partner companies
Individual CV and cover letter feedback including tailoring to specific job roles
Be among the first applications for new jobs with our AI application
1:1 support and career advice from our career coaches
Go Premium

Money-back if you don't land a job in 6-months

O
Similar positions in other companies
UK’s top job board for Gen Z
discover-jobs-cta
Discover now
>