Kafka Developer (Sheffield)

Job Role: Kafka Developer with Confluent Migration

Location: Sheffield, UK (Hybrid – 2 days onsite every week)

Employment Type: Contract B2B/Free Lance (Inside IR35)

Duration: Initially 6 Months (Extendable)

Roles & Responsibilities:

Summary:

We are seeking a highly experienced and skilled Senior Kafka Engineer to join our team. The ideal candidate will have a proven track record of designing, implementing, and maintaining large-scale Kafka deployments, with specific experience upgrading Kafka clusters. This role requires a deep understanding of Kafka's architecture, internals, and best practices. The candidate must be comfortable working independently and collaboratively within a fast-paced environment.

Responsibilities:

• Design, implement, and maintain high-throughput, low-latency Kafka clusters.
• Lead the upgrade of existing Kafka clusters from version 3.3.1 to the latest supported version, ensuring minimal downtime and data integrity.
• Experience in confluent migration .
• Develop and implement monitoring and alerting systems for Kafka.
• Troubleshoot and resolve Kafka-related issues.
• Participate in on-call rotation to ensure 24/7 system availability.
• Collaborate with other engineers to integrate Kafka into existing systems.
• Design and implement Kafka security best practices.
• Contribute to the improvement of our Kafka infrastructure.
• Mentor junior engineers and share expertise.
• Stay current with the latest Kafka technologies and best practices.

Qualifications:

• 8+ years of experience working with Apache Kafka.
• Proven experience u pgrading Kafka clusters, specifically from version 3.3.1 to a current supported version . Detail of the specific versions involved is highly desirable.
• Strong understanding of Kafka architecture, including topics, partitions, consumers, and producers.
• Experience with Kafka Connect.
• Experience with Kafka Zookeeper, Kraft.
• Experience working in Confluent Kafka.
• Experience in deploying the Kafka image in Kubernetes.
• Experience in setting up Stretched Kafka Cluster.
• Experience with Kafka Streams or ksqlDB a plus.
• Proficiency in at least one scripting language (e.g., Python, Shell).
• Experience with cloud-based Kafka deployments (AWS, Azure, GCP) is highly preferred.
• Experience with Kafka monitoring and alerting tools.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.