Kafka Migration Engineer (.NET + Kafka)
Location: London. UK
Duration: 4 months
Note: This role requires strong .NET skills in addition to Kafka expertise. All producers and consumers are .NET applications — APIs produce messages, and some services both consume and produce. This is not an infrastructure-only Kafka role; it requires reading, modifying, and validating .NET Kafka client code.
Responsibilities
Migrate all Kafka producers and consumers from DHLParcel.UK.Kafka to Confluent Cloud
Update Confluent.Kafka .NET client configuration (SASL auth, Schema Registry, bootstrap servers)
Validate message flows end-to-end for apps acting as both producer and consumer
Ensure consumer group configurations, offset management, and error handling are correct post-migration
Kafka throughput validation and performance testing
Work with .NET engineers to resolve application-level Kafka integration issues
Required Skills
C# / .NET — strong coding ability (reading and modifying producer/consumer code)
Confluent.Kafka .NET client library (ProducerBuilder, ConsumerBuilder, configuration)
Confluent Cloud — SASL/PLAIN authentication, Schema Registry, ACLs
Kafka internals: topics, partitions, consumer groups, offset management, rebalancing
Message serialization (Avro, JSON Schema) with .NET-specific serializers/deserializers
Troubleshooting Kafka connectivity and message flow issues in .NET applications
Kafka monitoring (consumer lag, throughput, error rates)
Nice to Have
Experience with apps that are both consumers and producers (event-driven architectures)
Schema evolution and compatibility strategies in Schema Registry
Performance tuning (batch size, linger.ms, compression) for .NET producers
Kafka Connect and MirrorMaker for migration scenarios
Kafka Migration Engineer in London employer: American IT Systems
Palantir is an exceptional employer, offering a dynamic work culture that fosters innovation and collaboration among high-complexity teams. With a focus on employee growth, we provide opportunities for professional development in cutting-edge technologies like Kubernetes and Terraform, all while contributing to meaningful projects that support UK Government programs in defense. Our London location not only offers a vibrant city life but also a unique chance to work on secure and scalable infrastructure that makes a real impact.
