Avro binary format compression benefit

["Rangwani, Sunil, Vodafone Group (External)" <su...@vodafone.com>]

Hi

I would like to know if anyone has seen any benefit in specifying a compression.type for Kafka Producer sending avro binary (not JSON) encoded messages to Kafka cluster. I can appreciate the compression benefits for JSON but is there really any benefit in compressing binary avro data?

Kafka Producer has a user thread(s) and a sender thread and the user thread uses a compressor but I am not clear whether the whole batch is compressed or each message is compressed before adding to the batch. I imagine compressing a batch would provide better compression ratio compared to compressing individual messages? Does kafka compress individual messages or the whole batch of messages?

Many thanks!

S