You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jiangjie Qin (JIRA)" <ji...@apache.org> on 2015/05/02 21:18:07 UTC
[jira] [Commented] (KAFKA-1788) producer record can stay in
RecordAccumulator forever if leader is no available
[ https://issues.apache.org/jira/browse/KAFKA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525421#comment-14525421 ]
Jiangjie Qin commented on KAFKA-1788:
-------------------------------------
I took a shot to incorporate the solution to this problem in KAFKA-2142.
The approach I took there is to just use metadata timeout instead of add a new timeout. Because I think this is essentially metadata not available. So we should treat it the same as in send(). This also saves us another timeout configuration.
[~ewencp] My concern about having cap on the buffer for each topic-partition is that what if the traffic of each topic-partition is not balanced. If so we might end up waiting on a busy topic-partition's buffer allocation while we actually have plenty of memory to use. That could hurt the performance a lot.
> producer record can stay in RecordAccumulator forever if leader is no available
> -------------------------------------------------------------------------------
>
> Key: KAFKA-1788
> URL: https://issues.apache.org/jira/browse/KAFKA-1788
> Project: Kafka
> Issue Type: Bug
> Components: core, producer
> Affects Versions: 0.8.2.0
> Reporter: Jun Rao
> Assignee: Parth Brahmbhatt
> Labels: newbie++
> Fix For: 0.8.3
>
> Attachments: KAFKA-1788.patch, KAFKA-1788_2015-01-06_13:42:37.patch, KAFKA-1788_2015-01-06_13:44:41.patch
>
>
> In the new producer, when a partition has no leader for a long time (e.g., all replicas are down), the records for that partition will stay in the RecordAccumulator until the leader is available. This may cause the bufferpool to be full and the callback for the produced message to block for a long time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)