Principal AI Systems Architect

Job Title: Principal AI Systems Architect

Location: Flexible / Hybrid – London

Employment Type: Permanent

Overview

We are seeking a Principal AI Systems Architect with deep technical expertise in AI engineering at scale to lead the design and orchestration of secure, high-performance multi-agent systems. This role sits at the intersection of AI, distributed systems, and advanced security engineering – ideal for someone who thrives in ambiguous, high-stakes problem domains and can design from first principles.

What You\’ll Be Solving

This is not your typical AI role. You\’ll be tackling:

• Agent orchestration at scale – thousands of agents working concurrently, requiring sophisticated coordination and communication strategies.
• Trust and security in AI systems – dynamic authentication, zero-trust networking, and malicious output protection.
• State consistency and fault tolerance – navigating trade-offs between performance, reliability, and consistency (e.g., causal consistency, network partitioning).
• Failure modes in LLM-based architectures – understanding injection attacks (e.g., prompt injection), and building intelligent defences at the orchestration layer.
• Scalability bottlenecks – architecting systems that handle massive data flows and compute workloads while maintaining responsiveness.

Key Responsibilities

• Architect large-scale, distributed AI systems with a focus on agent coordination, resilience, and security.
• Design scalable orchestration mechanisms that balance performance with robustness.
• Develop and implement defence mechanisms against LLM-related attack vectors, including output injection and system compromise attempts.
• Own core decision-making around consistency models, network reliability, and failure recovery.
• Collaborate with cross-functional teams to align engineering solutions with client-facing use cases.
• Act as a thought leader across AI architecture, agent system design, and production-readiness of cutting-edge models.

Must-Have Experience

• Deep technical expertise in AI/ML system design – not just model training, but the orchestration and scaling of AI components in production.
• Expert knowledge of distributed systems engineering, including consensus algorithms, conflict resolution, and partition tolerance.
• Proven experience with secure agent-based systems, zero-trust architecture, and dynamic authentication.
• In-depth understanding of LLM failure modes, particularly around prompt injection and adversarial behaviours.
• Strong programming ability in languages such as Python, Go, or Rust.
• Prior ownership of complex AI engineering programmes with real-world performance and security constraints.

Nice to Have

• Experience deploying multi-agent AI systems in real-world environments (e.g., financial services, defence, critical infrastructure).
• Exposure to runtime security monitoring, red teaming AI systems, or automated defences.
• Background in causality, system modelling, or probabilistic programming.

Why This Role Matters

This role is about building AI systems that can be trusted at scale. The challenges you\’ll tackle are emerging and highly complex – securing AI agents from manipulation, ensuring their communication is reliable, and embedding them safely in enterprise infrastructure.

You\’ll work closely with senior leadership to set architectural direction, influence product thinking, and ultimately shape the future of secure, scalable AI deployment.