You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Kafka <ka...@126.com> on 2016/04/23 04:05:28 UTC

kafka 0.8.2.1 High CPU load

Hi,we are using kafka_2.10-0.8.2.1,and our vm machine config is:4 core,8G
our cluster is consist of three brokers,and our broker config is default 2 replica,our broker load often very high once in a while,
load is greater than 1.5 on average core。

we have about 70 topics on this cluster
when we use Top util, we can see 280% cpu unilization, then i use JSTACK, I found there are 4 threads use cpu most, which show below:
"kafka-network-thread-9092-0" prio=10 tid=0x00007f46c8709000 nid=0x35dd runnable [0x00007f46b73f2000]
  java.lang.Thread.State: RUNNABLE
"kafka-network-thread-9092-1" prio=10 tid=0x00007f46c873c000 nid=0x35de runnable [0x00007f46b75f4000]
/kafka-network-thread
"kafka-network-thread-9092-2" prio=10 tid=0x00007f46c8756000 nid=0x35df runnable [0x00007f46b7cfb000]
  java.lang.Thread.State: RUNNABLE
"kafka-network-thread-9092-3" prio=10 tid=0x00007f46c876f800 nid=0x35e0 runnable [0x00007f46b5adb000]
  java.lang.Thread.State: RUNNABLE


I found one task:https:// <https://issues.apache.org/jira/browse/KAFKA-493 <https://issues.apache.org/jira/browse/KAFKA-493>>issues.apache.org/jira/browse/KAFKA-493 <http://issues.apache.org/jira/browse/KAFKA-493><http://issues.apache.org/jira/browse/KAFKA-493 <http://issues.apache.org/jira/browse/KAFKA-493>> concerns this, but I think this does’t fit me, because we have other clusters that deployed in 
the same config’s vm machine, and others are not occurred this.

the brokers config is same,but we add:

replica.fetch.wait.max.ms=100
num.replica.fetchers=2


I want to question does is the main reason, or have other reason that leads to high cpu load?