Kafka Architect- Cloudera

Social network you want to login/join with: We are looking for an experienced Kafka Architect to design and implement scalable, high-throughput data streaming solutions using Apache Kafka on Cloudera (on-premises). The ideal candidate will have a strong background in distributed systems, data pipelines, and real-time data processing. Key Responsibilities: -Design and implement scalable Kafka-based architectures using open-source -Kafka on Cloudera on-premises infrastructure. -Lead the setup, configuration, and optimization of Kafka clusters. -Define standards and best practices for Kafka producers, consumers, and streaming applications. -Integrate Kafka with various data sources, storage systems, and enterprise applications. -Monitor Kafka performance and ensure high availability, fault tolerance, and data security. -Collaborate with DevOps, Data Engineering, and Application teams to support real-time data needs. -Automate deployment and configuration using tools like Ansible, Terraform, or Cloudera Manager. -Provide technical leadership and mentorship to junior team members. Required Skills: -Strong hands-on experience with Apache Kafka (including Kafka Connect, Kafka Streams). -Experience with Cloudera distribution for Kafka on on-premises environments. -Solid knowledge of Kafka internals – topics, partitions, consumer groups, offset management, etc. -Experience with data serialization formats like Avro, JSON, Protobuf. -Proficient in Java, Scala, or Python for Kafka-based development. -Familiarity with monitoring tools (Prometheus, Grafana, Confluent Control Center, etc.). -Understanding of networking, security (SSL/SASL), and data governance. -Experience with CI/CD pipelines and containerization (Docker, Kubernetes) is a plus. #J-18808-Ljbffr