Senior Data Engineer - Multi-Cloud

UBDS Group is seeking an experienced Senior Data Engineer to design, build, and optimise scalable data platforms across complex transformation programmes. This role requires deep expertise in cloud‑based data engineering, architecture, and DevOps practices, with the ability to work across both AWS and Azure environments. You will play a key role in enabling high‑quality data integration, supporting advanced analytics use cases, and delivering robust, reusable data solutions aligned to client strategies.

Key Responsibilities

• Design and develop scalable, automated data pipelines using cloud‑native technologies (e.g. Spark), while integrating with on‑premise data sources where required.
• Implement and manage event‑driven data architectures, optimising pipeline performance, parallelisation, and early risk detection (e.g. fraud use cases).
• Build and maintain data access layers and ETL frameworks, enabling both self‑service and ad‑hoc onboarding of data assets.
• Develop reusable data engineering patterns and tooling to improve efficiency, scalability, and consistency across programmes.
• Apply DevOps and software engineering best practices, including Infrastructure as Code (IaC) and CI/CD pipelines.
• Collaborate with cross‑functional teams to ensure high‑quality data integration, governance, and performance.
• Work across hybrid cloud environments (AWS and Azure), ensuring secure, resilient, and scalable solutions.
• Support deployment and optimisation of data solutions leveraging services such as: AWS: EC2, EMR, S3, Lambda; Azure: Blob Storage and related services.
• Ensure alignment with client data strategies, governance frameworks, and security requirements.
• Contribute to knowledge sharing and capability uplift within both UBDS and client teams.

Essential Skills & Experience

• Strong experience in data engineering and data architecture on complex programmes.
• Proven ability to build and manage automated data pipelines using technologies such as Spark.
• Experience implementing event‑driven architectures and optimising data pipeline performance.
• Strong knowledge of ETL processes and data integration patterns, including self‑service data onboarding.
• Hands‑on experience with AWS and Azure cloud platforms (both required).
• Experience using key services such as: AWS: EC2, EMR, S3, Lambda; Azure: Blob Storage and associated services.
• Understanding of cloud networking concepts within AWS and Azure environments.
• Experience with Terraform (IaC) and GitLab (CI/CD pipelines).
• Strong grounding in DevOps and modern software engineering practices.

Desirable

• Experience working in government or regulated environments.
• Familiarity with large‑scale data platforms and data access layers.
• Experience supporting fraud detection or risk analytics use cases.
• Knowledge of data governance, security, and compliance frameworks.

Security Requirements

This role requires Security Clearance (SC). Candidates must hold active SC clearance or be eligible and willing to undergo clearance.

Engineering Practices

• Strong focus on automation, scalability, and reusability.
• Advocate of DevOps culture, including CI/CD and Infrastructure as Code.
• Experience working in agile, collaborative environments.
• Commitment to building robust, production‑grade data solutions.

Personal Attributes

• Strong problem‑solving and analytical skills.
• Proactive and delivery‑focused mindset.
• Comfortable working in complex, fast‑paced environments.
• Excellent communication and stakeholder collaboration skills.

Benefits

• Training – All team members are offered a number of options in terms of personal development, whether it is technical‑led, business acumen or methodologies.
• Private medical cover for you and your spouse/partner, offered via Vitality.
• Discretionary bonus based on a blend of personal and company performance.
• Holiday – You will receive 25 Days holiday, plus 1 day for Birthday and 1 day for your work anniversary in addition to UK bank holidays.
• Electric Vehicle leasing with salary sacrifice.
• Contributed Pension Scheme.
• Death in service cover.

About UBDS Group

At UBDS Group our mission is to support entrepreneurs who are setting new standards with technology solutions across cloud services, cybersecurity, data and AI, ensuring that every investment advances our commitment to innovation, making a difference, and creating impactful solutions for organisations and society.

Equal Opportunities

We are an equal opportunities employer and do not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, colour, nationality, ethnic or national origin, religion or belief, disability or age.