[jira] [Created] (KAFKA-10254) 100% cpu usage by kafkaConsumer poll , when broker can‘t be connect

["xiaotong.wang (Jira)" <ji...@apache.org>]

xiaotong.wang created KAFKA-10254:
-------------------------------------

             Summary: 100% cpu usage by kafkaConsumer poll , when broker can‘t be connect 
                 Key: KAFKA-10254
                 URL: https://issues.apache.org/jira/browse/KAFKA-10254
             Project: Kafka
          Issue Type: Bug
          Components: clients
    Affects Versions: 2.5.0
            Reporter: xiaotong.wang
         Attachments: image-2020-07-09-19-24-20-604.png

steps

1、start kafka broker 

2、start kafka consumer and subscribe some topic with some kafkaConsumer instance and  call  kafkaConsumer.*poll(Duration.ofMillis(pollTimeout))*   and set auto.commit.enabled=false

3、iptables to disable kafka broker  ip  in client vm or shutdown kafka brokers

4、cpu go to 100%

 

*why?*

 

 

left Vserison :2.3.1

right Version:2.5.0

 

for 2.3.1 kafkaConsumer when kafka  brokers go  down,updateAssignmentMetadataIfNeeded will block x ms and return empty records ,

!image-2020-07-09-19-24-20-604.png!

 

for 2.5.0

private Map<TopicPartition, List<ConsumerRecord<K, V>>> pollForFetches(Timer timer) {
 *long pollTimeout = coordinator == null ? timer.remainingMs() :*
 *Math.min(coordinator.timeToNextPoll(timer.currentTimeMs()), timer.remainingMs());*



i check the source of kafka client ,poll timeout will be change to 0 ms ,when heartbeat timeout ，so  it will call poll without any block ,this will cause cpu go to 100%

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)