Senior Data Engineer

Senior Data Engineer

Full-Time 70000 - 90000 £ / year (est.) No working from home possible
Beacon

At a Glance

  • Tasks: Build and scale innovative data products that transform healthcare insights.
  • Company: Beacon Intelligence, a leader in AI-driven pharmaceutical data solutions.
  • Benefits: Competitive salary, inclusive culture, and opportunities for professional growth.
  • Other info: Join a dynamic team focused on innovation and collaboration.
  • Why this job: Make a real impact on patients' lives while working with cutting-edge technology.
  • Qualifications: Strong experience in data platforms, Python, SQL, and cloud environments.

The predicted salary is between 70000 - 90000 £ per year.

Beacon Intelligence delivers data and insights that help R&D scientists develop better pharmaceuticals, faster than ever before.

Our focus is simple: we curate high-accuracy data enriched and transformed with AI.

This is delivered through a range of platforms, increasingly via AI-native products and features that enable customers to extract insights faster than ever before.

This is a rare opportunity to make a meaningful impact on patients’ lives worldwide.

About the Role

As a Senior Data/AI Engineer, you will play a central role in building and scaling our data products.

This is not a traditional back-end data engineering role.

You will work closely with product, commercial and technical colleagues, transforming datasets into commercial, customer-facing solutions.

You will combine deep technical expertise with a strong understanding of how data can be structured, enriched and operationalised through AI systems.

Responsibilities

  • Data Product Development (Core Focus)
  • Design and build scalable data products underpinning Beacon’s commercial offerings
  • Transform raw and third-party data into structured, enriched, product-ready datasets
  • Partner with product and commercial teams to define how data is packaged, accessed and monetised
  • Enable delivery via APIs, internal tools and customer-facing platforms
  • Apply AI and LLM capabilities to enrich and enhance data (e. g. classification, tagging, summarisation, insight generation)
  • Design and build LLM-powered product features
  • Integrate RAG into pipelines to improve data quality and unlock new features
  • Support development of AI-driven products such as recommendation engines, search and insight tools
  • Ensure systems are optimised, well-governed and resilient
  • Support deployment and lifecycle management of data and AI systems
  • Own and evolve the data platform architecture (Databricks, Azure, Airbyte) to support scalability
  • Build and maintain robust data pipelines (batch and streaming) delivering reliable, production-ready datasets
  • Ensure high standards in data quality, testing and observability
  • Improve efficiency through automation, CI/CD and engineering best practice
  • Commercial & Stakeholder Impact
  • Partner with senior stakeholders to identify high-value data product opportunities
  • Translate business needs into practical, scalable data and AI solutions
  • Act as a bridge between technical teams and commercial/product stakeholders
  • Contribute to prioritisation based on business impact
  • Leadership & Growth
  • Shape the evolution of our data product and AI strategy
  • Mentor junior engineers and promote a strong engineering and product mindset
  • Help build a high-performing, commercially aware data and AI engineering team

Qualifications

  • Strong experience with modern data platforms (e. g. Databricks) and cloud environments (Azure preferred)
  • Advanced Python and SQL, focused on production-quality, scalable solutions
  • Experience with Apache Spark (Py Spark) for large-scale data processing
  • Proven experience designing and building data pipelines and models
  • Experience working with APIs and data delivery mechanisms (critical for productisation)
  • Familiarity with CI/CD, Git and modern development practices
  • Required Skills
  • Experience building or supporting data products or customer-facing data solutions
  • Strong understanding of how to structure, expose and scale data for end users or clients
  • Ability to think beyond pipelines and focus on commercial value
  • Experience working closely with product or commercial teams
  • Preferred Skills
  • Familiarity with generative AI / LLMs (Azure Open AI, Lang Chain etc.) and experience applying them to real-world use cases
  • Experience evaluating, monitoring and optimising AI systems
  • Experience working with unstructured data (text, documents, web data)
  • Familiarity with embeddings, vector databases or semantic search
  • Understanding of AI governance and data ethics
  • Experience with Airbyte or modern data ingestion tools
  • Experience with Terraform or infrastructure-as-code
  • Exposure to LLM frameworks (e. g. Lang Chain, Llama Index)
  • Experience with experimentation or A/B testing frameworks
  • Background in building data-driven / AI-enabled products
  • Domain experience in life sciences
  • Pay range and compensation package
  • Equal Opportunity Statement

Beacon Intelligence is committed to diversity and inclusivity in the workplace.

#J-18808-Ljbffr

Senior Data Engineer employer: Beacon

Beacon in Upholland is an excellent employer, offering a supportive work culture where you can make a real difference in patient care. With flexible part-time hours and opportunities for professional growth, you'll thrive in a dynamic environment that values strong communication and teamwork. Join us to enjoy meaningful interactions with patients while developing your administrative skills in a rewarding healthcare setting.

Beacon

Contact Details:

Beacon Recruitment Team

We think you need these skills to ace Senior Data Engineer

SQL
Python
Problem-Solving Skills
Communication Skills
Data Engineering
Automation
Data Pipeline Development