Senior Data Engineer

The Company Lab 1 is a fast-growing cybersecurity startup, which identifies and interprets exposed data on behalf of our clients. Lab 1's AI SaaS platform analyses the full corpus of exposed corporate data in near real time, providing customers with the assurance that they'll know the unknown and reducing the risk, cost and anxiety associated with exposed data across complex supply chains.

The Role

Using the power of AI and Machine Learning, we deliver compromised data intelligence on any company within any supply chain. The data that powers those insights, a vast, messy, constantly changing corpus of exposed and compromised information, is the heart of our product. We are hiring a Senior Data Engineer to own and evolve the pipelines and data platform that turn raw exposed data into reliable, queryable intelligence. You will design, build and operate the batch and streaming systems that ingest, transform, model and serve data at scale, feeding our analytics, our ML models and our customer-facing platform. You will be the go-to expert for how data moves through Lab 1, working closely with founders, platform and software engineers in a small, friendly, international team.

We are looking for a versatile and proactive engineer who cares deeply about data quality, correctness and scale. Your primary focus will be owning our data pipelines end to end, while also working alongside our ML efforts on the early stages of the training pipeline (data cleaning and feature engineering) to make sure our models are built on trustworthy, well-structured data. As a core member of the engineering team, your mission will be to make our data trustworthy, timely and easy to build on. You will spend your time understanding how the team and our models consume data, architecting robust pipelines, and delivering clean, well-modelled datasets that let us ship high-quality intelligence faster and more safely. Your work will be the foundation that enables us to solve some of our most interesting, urgent, and taxing problems. You will report to the Chief Data Officer.

Required demonstrable experience in:

• Designing, building and operating production data pipelines end to end, across both batch and streaming
• Strong Python for data engineering, including hands-on PySpark at scale (AWS EMR)
• Strong SQL modelling, transforming and querying large datasets
• Streaming data with Kafka
• Ingesting large, heterogeneous, unclean data from many sources and making it reliable
• Lakehouse and data warehousing patterns: Apache Iceberg, Athena, S3, columnar formats (ORC)
• Workflow orchestration with Airflow
• Search and indexing with OpenSearch
• Infrastructure as Code, specifically Terraform
• Managing core AWS services
• Building and maintaining CI/CD pipelines, ideally with GitHub Actions
• Data quality, observability, lineage and cost awareness through effective monitoring
• Supporting ML/AI workflows through data cleaning and feature engineering for training pipelines

We primarily work with:

• Python (Poetry, migrating to uv)
• PySpark on AWS EMR
• SQL
• Kafka
• Athena + S3 / Apache Iceberg (ORC)
• OpenSearch
• Airflow
• AWS
• Terraform
• GitHub Actions
• Docker

We are interested in candidates who value:

• Team working
• Delivering measurable high quality software
• Keeping sight of the big picture
• Friendly technical discussions and whiteboarding

This is a great opportunity to join a fast growing company at an early stage and make a significant contribution. The role will be remunerated through a salary and, after a qualifying period, you will be granted a meaningful share in the company through an options grant. Candidates must have the right to work in the United Kingdom. The role is remote first with occasional in-person meetings in London (or Norwich).

Benefits:

• Company pension
• Company options scheme