Data Engineer (National Security R&D) in London

You will be joining one of the UK’s most respected engineering organisations operating at the forefront of National Security technology. For more than six decades they have been solving complex technical challenges across cyber, communications, AI and high-assurance systems for government and defence.

Their engineering teams combine software engineers, data scientists, systems engineers and security specialists to build mission-critical capabilities that operate in some of the most demanding environments. Rather than operating in narrow roles, engineers regularly move across projects and domains, working with new technologies and disciplines as programmes evolve. The result is an environment that feels closer to a national security R&D lab than a traditional delivery consultancy.

With strong demand across secure programmes and continued investment in their National Security division, the data engineering capability continues to grow across their UK sites.

As a Data Engineer you will help design and build the data pipelines, analytics platforms and processing systems that enable complex data-driven capabilities across secure programmes. You will work across a mixture of streaming, batch and real-time data environments, helping transform raw data into operational insights used by engineering and analytical teams.

Typical responsibilities include:

• Building scalable data pipelines and ingestion frameworks using Python-based ETL processing
• Designing and implementing data integration and transformation pipelines across batch, streaming and real-time data sources
• Developing high-performance analytics and processing components using Python or Go
• Deploying analytics and data services into containerised environments using Docker and Kubernetes with Helm
• Working with distributed data technologies such as Apache Spark, Kafka and NiFi
• Designing and managing data models, schemas and database architectures across relational and NoSQL platforms such as PostgreSQL, MySQL, MongoDB and Cassandra
• Developing cloud-native data platforms using AWS services including EC2, S3 and EKS
• Supporting real-time message streaming and event-based architectures using technologies such as Kafka or NATS
• Writing clean, testable and reusable code following secure and test-driven development practices
• Monitoring and maintaining deployed data platforms and analytics services to ensure reliability and performance

Projects often involve complex data processing chains that ingest, transform and route large volumes of structured and unstructured data to support downstream analysis and operational decision making. This role suits engineers who enjoy working across data engineering, analytics and platform infrastructure, rather than specialising in a single narrow technology. The teams value engineers who are curious, adaptable and comfortable collaborating with stakeholders across multiple technical disciplines.

Due to the nature of the work, candidates must meet the criteria required to obtain the highest level of UK security clearance. This typically requires sole UK nationality and long-term UK residency.