How to choose Kafka transaction id for several applications, hosted in Kubernetes? How to choose Kafka transaction id for several applications, hosted in Kubernetes? kubernetes kubernetes

How to choose Kafka transaction id for several applications, hosted in Kubernetes?


If the consumer starts the transaction (read-process-write) then the transaction id prefix must be the same for all instances of the same app (so that zombie fencing works correctly after a rebalance). The actual transaction id used is <prefix><group>.<topic>.<partition>.

If you have multiple apps, they should have unique prefixes (although if they consume from different topics, they will be unique anyway).

For producer-only transactions, the prefix must be unique in each instance (to prevent kafka fencing the producers).