At a Glance
- Tasks: Develop and maintain a cutting-edge data platform for climate and health research.
- Company: Wellcome Sanger Institute, a leader in genomic and epidemiological data analysis.
- Benefits: Competitive salary, fixed-term contract, and opportunities for professional growth.
- Other info: Collaborate with international partners in a dynamic, interdisciplinary environment.
- Why this job: Make a real impact on global health by integrating diverse datasets.
- Qualifications: Proficiency in Python, SQL, and modern data architectures required.
The predicted salary is between 50053 - 59500 € per year.
We are seeking a Data Engineer at Senior or Principal level to further develop, maintain and operate our data platform within the Parasites and Microbes Programme at the Wellcome Sanger Institute.
About The Role
You will work on a Data Integration and Analysis platform underpinned by a Data Lakehouse (DLH), built on technologies such as object storage, distributed query engines, workflow orchestration, and metadata/catalogue systems. Technologies currently in use include:
- Metadata, governance & security: Hive Metastore, DataHub, Apache Ranger, Keycloak, Vault
- Data access & visualisation: Apache Superset, CloudBeaver
A key facet of the role will be the delivery of a DLH-based data integration and analysis platform for the icddr,b Climate Hub (iCCH), working in collaboration with international partners to enable robust, reproducible analyses linking climate and demographic variables with health outcomes. You will play an important part in enabling interdisciplinary research by ensuring that data is well-structured, discoverable, and reproducible, supporting scientists to generate new insights from integrated datasets. Ingesting and transforming a wide range of data types (including e.g. geospatial and climate data, along with genomic data) is a key aspect of the role. You will work closely with data engineers, bioinformaticians, and scientists to ensure the platform meets scientific needs while remaining scalable, reliable, and maintainable.
About You
You will be an experienced Data Engineer with a willingness to operate in a hands‑on capacity across all of the layers of the data platform stack. You will be comfortable in translating often‑complex scientific and data requirements into robust technical solutions, and be able to communicate effectively with both technical and non‑technical stakeholders.
Essential Technical Skills For both Senior and Principal roles:
- Proficiency in Python, SQL and data transformation practices
- Data modelling and warehousing paradigms (e.g. ELT, Star schemas)
- Modern data platform architectures (e.g. data lakes or lakehouses)
- Distributed query or processing engines (e.g. Trino, Spark, Presto)
- Object storage systems (e.g. S3‑compatible systems such as MinIO)
- Workflow orchestration tools (e.g. Prefect, Airflow)
- Containerisation and orchestration (e.g. Docker, Kubernetes)
- CI/CD (e.g. Gitlab CI, Github Actions)
Additional Expectations For Principal-level Appointments:
- Technical leadership, with the ability to define and drive architectural decisions across complex data ecosystems
- Strong ownership and accountability for quality and reliability
- Designing, developing and operating data platforms at scale
- Line management, mentoring and coaching
About Us
Within the Parasites and Microbes Programme, we generate and analyse genomic and epidemiological data to better understand infectious diseases and their impact on human populations. Our work increasingly sits at the intersection of multiple data domains, including genomics, public health surveillance, and environmental and climate science. To support our work, we are developing a modern, scalable Data Lakehouse platform that enables the integration, transformation, and analysis of complex, heterogeneous datasets. This platform is central to a number of strategic initiatives, including a collaboration with the International Centre for Diarrhoeal Disease Research in Bangladesh (icddr,b) to investigate the links between climate change and health outcomes.
Salary Range (Dependent On Skills And Experience):
- Grade 1 Principal Data Engineer £61,511 to £73,000
- Grade 2 Senior Data Engineer £50,053 to £59,500
Contract Type: Fixed Term contract until 29th October 2027
Data Engineer (Senior or Principal) employer: Wellcome Sanger Institute
The Wellcome Sanger Institute is an exceptional employer, offering a collaborative and innovative work culture that empowers Data Engineers to make significant contributions to interdisciplinary research. With a focus on employee growth, the institute provides opportunities for professional development and technical leadership within a cutting-edge data platform environment, all while working alongside leading scientists and international partners in the heart of genomic and epidemiological research.
StudySmarter Expert Advice🤫
We think this is how you could land Data Engineer (Senior or Principal)
✨Tip Number 1
Network like a pro! Reach out to your connections in the data engineering field, attend meetups, and engage in online forums. The more people you know, the better your chances of landing that dream job.
✨Tip Number 2
Show off your skills! Create a portfolio showcasing your projects, especially those involving Python, SQL, and data transformation. This will give potential employers a taste of what you can do and set you apart from the crowd.
✨Tip Number 3
Prepare for interviews by brushing up on your technical knowledge and soft skills. Be ready to discuss your experience with data lakehouses, distributed query engines, and workflow orchestration tools. Confidence is key!
✨Tip Number 4
Don't forget to apply through our website! We love seeing candidates who are genuinely interested in joining our team. Tailor your application to highlight how your skills align with our mission at the Wellcome Sanger Institute.
We think you need these skills to ace Data Engineer (Senior or Principal)
Some tips for your application 🫡
Tailor Your CV:Make sure your CV is tailored to the Data Engineer role. Highlight your experience with Python, SQL, and any relevant data transformation practices. We want to see how your skills align with our needs!
Craft a Compelling Cover Letter:Your cover letter is your chance to shine! Use it to explain why you're passionate about data engineering and how you can contribute to our Data Lakehouse platform. Keep it engaging and relevant to the role.
Showcase Your Projects:If you've worked on any projects that involved data integration or analysis, make sure to mention them! We love seeing real-world applications of your skills, especially if they relate to interdisciplinary research.
Apply Through Our Website:We encourage you to apply through our website for a smoother application process. It helps us keep track of your application and ensures you don’t miss out on any important updates from us!
How to prepare for a job interview at Wellcome Sanger Institute
✨Know Your Tech Stack
Make sure you’re well-versed in the technologies mentioned in the job description, like Python, SQL, and data transformation practices. Brush up on your knowledge of data lakehouses and distributed query engines, as these will likely come up during the interview.
✨Showcase Your Problem-Solving Skills
Prepare to discuss specific examples where you've translated complex scientific requirements into technical solutions. Think about challenges you've faced in previous roles and how you overcame them, especially in data integration and analysis.
✨Communicate Effectively
Since you'll be working with both technical and non-technical stakeholders, practice explaining your past projects in simple terms. This will demonstrate your ability to bridge the gap between different teams and ensure everyone is on the same page.
✨Demonstrate Leadership Potential
If you're applying for the Principal role, be ready to talk about your experience in technical leadership. Share instances where you've made architectural decisions or mentored others, showcasing your ownership and accountability for quality and reliability.
