You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Soumyajit Sahu (Jira)" <ji...@apache.org> on 2021/04/23 01:20:00 UTC
[jira] [Commented] (KAFKA-10619) Producer will enable EOS by
default
[ https://issues.apache.org/jira/browse/KAFKA-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17329915#comment-17329915 ]
Soumyajit Sahu commented on KAFKA-10619:
----------------------------------------
Hi Cheng,
I would like to raise an issue with making enable.idempotence=true by default.
Recently, we hit an issue where a producer app of ours was using enable.idempotence=true, but was creating a producer object for each message.
We ended up in having thousands of producer ids for our log segment, and the broker crashed with OutOfMemory exception while trying to take a snapshot.
I am not aware of any way to monitor this or check on such behavior. Hence, setting this to true could break many clusters with a surprise.
It would be ideal to come up with a monitoring or controlling mechanism for this before we set it to true by default.
> Producer will enable EOS by default
> -----------------------------------
>
> Key: KAFKA-10619
> URL: https://issues.apache.org/jira/browse/KAFKA-10619
> Project: Kafka
> Issue Type: Improvement
> Reporter: Cheng Tan
> Assignee: Cheng Tan
> Priority: Major
> Fix For: 3.0.0
>
>
> This is an after-work for KIP-185.
> In the producer config,
> # the default value of `acks` will change to `all`
> # `enable.idempotence` will change to `true`
> [An analysis of the impact of max.in.flight.requests.per.connection and acks on Producer performance|https://cwiki.apache.org/confluence/display/KAFKA/An+analysis+of+the+impact+of+max.in.flight.requests.per.connection+and+acks+on+Producer+performance] indicates that changing `acks` from `1` to `all` won't increase the latency and decrease the throughput in a significant way.
>
> KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default
--
This message was sent by Atlassian Jira
(v8.3.4#803005)