Data Engineer in London

We are a leading consultancy with a purpose to make an enduring impact on health and healthcare. We work with leaders and frontline teams to improve health, transform healthcare, drive adoption of innovation and create value through investment. Our consultancy serves the entire healthcare sector, from payors and providers of care, to life science companies, health tech and sector suppliers and health investors. We provide end-to-end services, from strategy through implementation, accelerated by data, digital and AI. We shape opinion through evidence-based thought leadership on key issues affecting health. With unmatched ability to access and use health data, our consultants are a driving force for delivering positive and meaningful change.

The Data Engineer sits within the Data Innovation team and works day-to-day on the CF lakehouse, our Databricks platform that holds the routine healthcare data behind our analytical and product work. The role is hands-on and delivery-focused: writing SQL and Python, building and maintaining automated pipelines, and turning raw data into structured, production-ready material that consultants and clients can rely on. CF runs an apprentice model of development. You will learn actively from senior engineers, contribute to the team’s collective knowledge, and take on greater scope as you grow. The work spans data cleansing and validation for client engagements, building and maintaining pipelines, and contributing to CF’s technical products, including through hackathons. This is a good fit for an engineer with a couple of years of experience who wants to build a broad technical foundation in a data-rich healthcare consultancy, work close to real client problems, and see how technical work turns into client value.

Responsibilities

• Build and maintain automated data pipelines using PySpark and Spark, with appropriate monitoring.
• Build data models and pipelines under the guidance of senior colleagues, producing production-ready code or client-ready material.
• Develop data quality, validation and consistency checks, and carry out data cleansing.
• Develop unit, functional and integration tests, and follow the team’s Git and GitHub workflow.
• Participate in agile ways of working: keep user stories and tasks up to date, and contribute to stand-ups, retros and show and tells.
• Flag early when work is deviating from plan, and help identify and deliver mitigations.
• Build an understanding of healthcare data and the CF lakehouse: what is collected, how it is structured, and how new sources are brought in.
• Work with teams to bring analytical insights to client problems.
• Communicate technical solutions to non-technical colleagues, and give and receive feedback well.
• Learn best-practice development processes from senior colleagues, and seek out new tools and techniques to apply.
• Use AI tools to improve code quality and efficiency.
• Support business development and bid writing on technical detail, and contribute to product development through hackathons and thought leadership.

Requirements

• Around 2 to 3 years of experience in a data engineering or comparable technical data role.
• Working SQL, used day-to-day on the lakehouse.
• Intermediate Python for data transformation and automation.
• Experience building automated data pipelines using PySpark or Spark, with appropriate monitoring.
• Ability to write unit, functional and integration tests.
• Comfort working to a Git and GitHub workflow.
• Curiosity about healthcare data and how it relates to care delivery and policy.
• A clear communicator who knows when to ask for support and works well in a team.

Desirable, and not expected on day one:

• Databricks lakehouse experience: notebooks, Workflows and Jobs, Delta Lake Unity Catalog basics (we will train).
• Awareness of cloud basics such as storage and access control, and of medallion structure, bronze to silver to gold.
• Exposure to dbt and basic dimensional modelling.

Flexible working

We follow a hybrid working model that balances in person connections and remote work to drive exceptional client impact. We enjoy working in person together with clients and colleagues and work where clients need us to be. In supporting flexibility and remote working, team members can work from home one day per week as standard. Additionally, we offer 44 remote working days per year which can be used to top up your working from home days and enable you to work from home up to two days per week-subject to client needs. Alternatively, you could use your allowance in blocks to manage school holidays or other commitments. Our core in person working hours are from 10am until 4pm allowing you that extra flexibility to manage your schedule in a way that works for you.

Our commitment to Diversity & Inclusion

We are committed to building an inclusive and supportive culture where diversity thrives, and all our people can excel. We only recruit, promote and reward our people based on their skills and contribution, without regard to gender, race, disability, religion, nationality, ethnicity, sexual orientation, age, marital status, or other characteristics. We are Disability Confident Accredited, and we want you to feel comfortable and able to perform at your best in the recruitment process. If you require any reasonable adjustments for any part of the recruitment process, please let us know.

Benefits

• Holiday entitlement: 25 days/year for staff and 30 days/year for leadership, increasing by 1 day for every year of service up to a maximum of 35 days of holiday per year.
• We contribute 7% of your salary into your pension, while you contribute 3% (or more if you like).
• Access to a flexible benefits programme giving you the chance to increase pension contributions, gain access to a cash plan or benefit from a ClassPass subscription.
• Annual leave purchase: employees with less than 35 days annual leave entitlement are able to purchase additional annual leave days.
• Income protection: in the event of long-term incapacity and a qualifying claim, 75% of salary will be paid.
• Enhanced sick pay benefit beyond Statutory Sick Pay for up to a total 12 weeks in any 12-month period.
• Life insurance covering four times your basic salary in a tax-free lump sum payable to your beneficiaries in the event of your death whilst in service.
• Enhanced family leave policies: additional pay for parents who have a baby or adopt.
• Access to an interest-free loan of up to £10,000.
• Access to an interest-free season ticket loan, repayable by 12 monthly instalments.
• Workplace nursery scheme: access to a scheme to help working parents save tax and NI on the cost of the nursery care.
• Flexible working policy: including the ability to work from home up to two days per week, with 44 remote working days per year.
• An employee assistance and wellness programme: including access to telephone counselling, life coaching, interactive tools online and digital content downloadable from Lifeworks.
• Seasonal flu jabs: provided by Boots annually.
• Eye care tests: vouchers and discounts at Vision Express.
• Ride to work scheme, saving up to 42% on bikes and cycling accessories at Evans Cycles.
• Membership to the Health Service Journal (HSJ).