Data Architect – London

We are seeking a Senior Solution Architect with deep experience designing and delivering enterprise-scale big data, streaming, and real‑time data integration solutions. The role is focused on solution architecture across Cloudera-based big data platforms, Kafka-based streaming applications, and real‑time data integration / analytics solutions, including cloud-hosted data platforms such as Cloudera on AWS or equivalent AWS-native services. This is not a pure data modelling, reporting, analytics, or governance role. The successful candidate must have a strong architecture background and be comfortable operating across both solution design and hands‑on technical delivery.

Hybrid working: The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.

Key Responsibilities:

• Lead end-to-end solution architecture for large-scale distributed data platforms, real-time streaming solutions, and enterprise data integration initiatives.
• Design and architect Kafka-based streaming ecosystems, including event ingestion, topic design, partitioning strategies, consumer groups, offset management, retries, dead-letter queues, replay mechanisms, scalability, and resiliency.
• Drive Spark-based data processing solutions for batch, micro-batch, and real-time workloads, ensuring optimal performance and scalability.
• Architect data platforms on Cloudera CDH/CDP and AWS, including Data Lakes, Operational Data Stores (ODS), and distributed storage systems.
• Design and implement real-time data ingestion frameworks leveraging Kafka, Spark Streaming, Flink, NiFi, and Change Data Capture (CDC) patterns.
• Define architecture patterns for data consumption and APIs using Spring Boot, Spring Kafka, Java, REST APIs, and event-driven integration approaches.
• Provide technical leadership across data ingestion, transformation, storage, streaming integration, API enablement, and downstream consumption layers.
• Review and govern technical designs, ensuring alignment with enterprise architecture standards, security requirements, scalability goals, and operational best practices.
• Identify technical risks, dependencies, and non-functional requirements, including performance, resiliency, observability, security, and supportability.
• Collaborate with engineering teams and stakeholders to translate business requirements into scalable, production-ready architecture solutions.
• Own architecture deliverables, including Architecture Summaries, High-Level Designs (HLDs), solution documentation, and presentations to architecture review boards.
• Promote best practices and reusable architecture patterns across Kafka, Spark, distributed data platforms, cloud services, and event-driven architectures.

Hands-on expertise in Kafka and Spark is mandatory, with proven experience delivering production-grade streaming and data processing solutions at scale.

Required Experience:

• 10+ years of overall IT experience.
• 5+ years in Solution Architecture, Technical Architecture, or Lead Design roles.
• Strong experience with Kafka, Spark, Cloudera CDH/CDP, HBase, Flink, NiFi, Hive, and Impala.
• Experience with AWS data services such as MSK, Kinesis, Event Bridge, and cloud-native data platforms.

We are a Disability Confident Employer: Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Government’s Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who: Declare they have a disability, and Meet the minimum essential criteria for the role. Please opt in during the application process.