You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/05/23 10:35:04 UTC
[jira] [Commented] (KAFKA-5263) kakfa-clients consume 100% CPU with
manual partition assignment when network connection is lost
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021013#comment-16021013 ]
ASF GitHub Bot commented on KAFKA-5263:
---------------------------------------
GitHub user rajinisivaram opened a pull request:
https://github.com/apache/kafka/pull/3124
KAFKA-5263: Avoid tight polling loop in consumer with no ready nodes
If all brokers are unavailable, consumers using manual assignment poll in a tight loop with no channels to poll from. Producers and consumers using group management avoid this since they wait for metadata which attempt to create new connections with any necessary backoff. This PR does a similar wait for consumers using manual assignment when no nodes are ready.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rajinisivaram/kafka KAFKA-5263
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/3124.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3124
----
commit e4cd05a61c3e5422b2a897004b41d523b5af8c16
Author: Rajini Sivaram <ra...@googlemail.com>
Date: 2017-05-23T10:05:21Z
KAFKA-5263: Avoid tight polling loop in consumer with no ready nodes
----
> kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost
> -----------------------------------------------------------------------------------------------
>
> Key: KAFKA-5263
> URL: https://issues.apache.org/jira/browse/KAFKA-5263
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1
> Reporter: Konstantin Smirnov
> Assignee: Rajini Sivaram
> Attachments: cpu_consuming.log, cpu_consuming_profile.csv
>
>
> Noticed that lose of the connection to Kafka broker leads kafka-clients to consume 100% CPU. The bug only appears when the manual partition assignmet is used. It appears since the version 0.10.1.0. The bug is quite similar to KAFKA-1642.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)