Kafka Architect- Cloudera

Job Title: Kafka Architect- ClouderaLocation: Leeds (2days/week Onsite)Duration: 06+ MonthsJob Summary:We are looking for an experienced Kafka Architect to design and implement scalable, high-throughput data streaming solutions using Apache Kafka on Cloudera (on-premises). The ideal candidate will have a strong background in distributed systems, data pipelines, and real-time data processing.Key Responsibilities:-Design and implement scalable Kafka-based architectures using open-source -Kafka on Cloudera on-premises infrastructure.-Lead the setup, configuration, and optimization of Kafka clusters.-Define standards and best practices for Kafka producers, consumers, and streaming applications.-Integrate Kafka with various data sources, storage systems, and enterprise applications.-Monitor Kafka performance and ensure high availability, fault tolerance, and data security.-Collaborate with DevOps, Data Engineering, and Application teams to support real-time data needs.-Automate deployment and configuration using tools like Ansible, Terraform, or Cloudera Manager.-Provide technical leadership and mentorship to junior team members.Required Skills:-Strong hands-on experience with Apache Kafka (including Kafka Connect, Kafka Streams).-Experience with Cloudera distribution for Kafka on on-premises environments.-Proficiency in designing high-volume, low-latency data pipelines.-Solid knowledge of Kafka internals – topics, partitions, consumer groups, offset management, etc.-Experience with data serialization formats like Avro, JSON, Protobuf.-Proficient in Java, Scala, or Python for Kafka-based development.-Familiarity with monitoring tools (Prometheus, Grafana, Confluent Control Center, etc.).-Understanding of networking, security (SSL/SASL), and data governance.-Experience with CI/CD pipelines and containerization (Docker, Kubernetes) is a plus.Kind Regards–Priyanka SharmaSenior Delivery ConsultantOffice: 02033759240Email: psharma@vallumassociates.com