We're partnering with an innovative UK technology company developing a next-generation AI platform for critical defence and healthcare applications. As they continue to scale, they're looking for a Principal Data & AI Engineer to provide technical leadership across the design, architecture, and delivery of enterprise-grade AI and data platforms.
This is a rare opportunity to take ownership of the data and AI engineering function within a rapidly growing business, shaping the technical direction of products that solve complex, real-world problems. You'll work alongside AI researchers, software engineers, and product teams to build highly secure, scalable data platforms that power advanced AI capabilities in mission-critical environments.
If you're passionate about modern data engineering, large-scale AI systems, and influencing technical strategy while remaining hands-on, this role offers genuine autonomy and long-term career progression.
What You'll Be Doing
- Lead the architecture and development of scalable AI and data platforms supporting advanced machine learning and LLM workloads.
- Design and build high-performance data pipelines capable of processing structured and unstructured data from multiple secure sources.
- Architect robust data ingestion, transformation and orchestration frameworks using modern cloud-native tooling.
- Build and optimise Retrieval-Augmented Generation (RAG) pipelines and enterprise knowledge retrieval platforms.
- Develop and optimise vector search capabilities using technologies such as Weaviate, Pinecone, Milvus or Elasticsearch.
- Engineer highly available APIs and backend services using Python (FastAPI/Flask) and modern software engineering practices.
- Design secure, containerised platforms using Docker and Kubernetes for deployment into restricted environments.
- Drive best practice around data modelling, governance, observability, testing and platform reliability.
- Mentor and provide technical leadership to engineers across the wider software and AI teams.
- Work closely with stakeholders to translate complex business challenges into scalable technical solutions.
What We're Looking For
- Significant commercial experience as a Senior, Lead or Principal Data Engineer, AI Engineer or Platform Engineer.
- Strong expertise in Python and SQL, alongside modern software engineering principles.
- Deep understanding of distributed data processing and orchestration technologies (e.g. Spark, Kafka, Airflow, Dagster or similar).
- Experience building AI-enabled platforms using Large Language Models (LLMs), embeddings, vector databases and RAG architectures.
- Hands-on experience with vector databases such as Pinecone, Weaviate, Milvus or Elasticsearch.
- Strong understanding of cloud-native architecture, Docker and Kubernetes.
- Experience designing resilient microservice architectures and RESTful APIs.
- Knowledge of data governance, security, metadata management and production data operations.
- Comfortable leading technical direction, mentoring engineers and influencing architecture decisions.
- Excellent communication skills with the ability to engage technical and non-technical stakeholders.
- Degree in Computer Science, Software Engineering, Data Engineering or a related discipline (or equivalent commercial experience).
- Experience deploying AI platforms into secure or highly regulated environments.
- Knowledge of MLOps tooling and model lifecycle management.
- Experience with Azure, AWS or private cloud infrastructure.
- Familiarity with Infrastructure as Code (Terraform, Helm, Ansible or similar).
- Experience within Defence, National Security, Healthcare, Government or other regulated industries.
- Exposure to knowledge graphs, semantic search or graph databases.
- Experience leading technical strategy within a fast-growing technology business.
Security Requirements
Due to the nature of the work, applicants must be eligible for SC Security Clearance
#J-18808-Ljbffr
Senior Data Architect in Basingstoke employer: iDPP
Join a forward-thinking company that values innovation and collaboration, where as a Senior Mechanical Engineer, you will have the opportunity to work on cutting-edge products in challenging environments. Our supportive work culture fosters professional growth, offering continuous learning opportunities and the chance to take ownership of your designs from concept to manufacture. Located in a vibrant area, we provide a dynamic workplace that encourages creativity and teamwork, making it an excellent choice for engineers looking to make a meaningful impact.
