You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by "joseph george k." <hi...@gmail.com> on 2017/03/09 10:25:28 UTC

Need help in solving log duplication issue when throttling is enabled in Kafka

Hi,

I currently face log duplication issue in my setup after enabling log
throttling in kafka.

*Change applied to enable log throttling :*

quota.producer.default=102400 #100Kbps

*How my setup looks :*

Hekad agents (producers) -> Kafka -> ELK stack (consumers)

This error conditions (duplicate log lines) is observed when there is an
outburst of logs from the producers. The same is working fine when
throttling is disabled.

Please can someone help me to overcome this log duplication issue.

Regards,
Joseph George K