Kafka Consumer 2.0.0 Can't Receive Some Messages

[刘俊杰 <ju...@gmail.com>]

Recently,I had some problems when using kafka consumer AIPs.(The broker and
replicas number are 3) There is the error log:

10:42:40:446 INFO [FetchSessionHandler] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Node 2 sent an invalid full fetch response
with extra=(mbGW4rH5-0, response=(
10:43:10:499 INFO [FetchSessionHandler] [Consumer clientId=mbGW4rH5,
groupId=group-admin-test] Error sending fetch request
(sessionId=INVALID, epoch=INITIAL) to node 2:
org.apache.kafka.common.errors.TimeoutException: Failed to send
request after 30000 ms..

.and then the client was suspended since then,it couldn't receive any
messages,so I turned up DEBUG of the log.and there are the logs:

10:58:11:200 INFO [FetchSessionHandler] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Error sending fetch request
(sessionId=INVALID, epoch=INITIAL) to node 2:
org.apache.kafka.common.errors.TimeoutException: Failed to send
request after 30000 ms..
10:58:11:200 DEBUG [Fetcher] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Added READ_UNCOMMITTED fetch request for
partition mbGW4rH5-0 at offset 15 to node 17.162.114.1:9294 (id: 2
rack: null)
10:58:11:200 DEBUG [FetchSessionHandler$Builder] [Consumer
clientId=mbGW4rH5, groupId=group-mbGW4rH5] Built full fetch
(sessionId=INVALID, epoch=INITIAL) for node 2 with 1 partition(s).
10:58:11:200 DEBUG [Fetcher] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Sending READ_UNCOMMITTED
FullFetchRequest(mbGW4rH5-0) to broker 17.162.114.1:9294 (id: 2 rack:
null)
10:58:13:161 DEBUG [AbstractCoordinator] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Sending Heartbeat request to coordinator
cloud-access.hanclouds.com:9292 (id: 2147483647 rack: null)
10:58:13:207 DEBUG [AbstractCoordinator$HeartbeatResponseHandler]
[Consumer clientId=mbGW4rH5, groupId=group-mbGW4rH5] Received
successful Heartbeat response
10:58:15:113 DEBUG [ConsumerCoordinator] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Sending asynchronous auto-commit of offsets
{mbGW4rH5-0=OffsetAndMetadata{offset=15, metadata=''}}
10:58:15:159 DEBUG [ConsumerCoordinator$OffsetCommitResponseHandler]
[Consumer clientId=mbGW4rH5, groupId=group-mbGW4rH5] Committed offset
15 for partition mbGW4rH5-0
10:58:15:159 DEBUG [ConsumerCoordinator$4] [Consumer
clientId=mbGW4rH5, groupId=group-mbGW4rH5] Completed asynchronous
auto-commit of offsets {mbGW4rH5-0=OffsetAndMetadata{offset=15,
metadata=''}}
10:58:16:162 DEBUG [AbstractCoordinator] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Sending Heartbeat request to coordinator
17.162.110.1:9292 (id: 2147483647 rack: null)
10:58:16:217 DEBUG [AbstractCoordinator$HeartbeatResponseHandler]
[Consumer clientId=mbGW4rH5, groupId=group-mbGW4rH5] Received
successful Heartbeat response
10:58:19:162 DEBUG [AbstractCoordinator] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Sending Heartbeat request to coordinator
17.162.110.1:9292 (id: 2147483647 rack: null)
10:58:19:211 DEBUG [AbstractCoordinator$HeartbeatResponseHandler]
[Consumer clientId=mbGW4rH5, groupId=group-mbGW4rH5] Received
successful Heartbeat response
10:58:20:114 DEBUG [ConsumerCoordinator] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Sending asynchronous auto-commit of offsets
{mbGW4rH5-0=OffsetAndMetadata{offset=15, metadata=''}}
10:58:20:165 DEBUG [ConsumerCoordinator$OffsetCommitResponseHandler]
[Consumer clientId=mbGW4rH5, groupId=group-mbGW4rH5] Committed offset
15 for partition mbGW4rH5-0
10:58:20:165 DEBUG [ConsumerCoordinator$4] [Consumer
clientId=mbGW4rH5, groupId=group-mbGW4rH5] Completed asynchronous
auto-commit of offsets {mbGW4rH5-0=OffsetAndMetadata{offset=15,
metadata=''}}
10:58:22:163 DEBUG [AbstractCoordinator] [Consumer clientId=mbGW4rH5,
groupId=group-mbGW4rH5] Sending Heartbeat request to coordinator
17.162.110.1:9292 (id: 2147483647 rack: null)

 and I googled it but couldn't find a answer of this. It seemed that the
client couldn't receive the offset of 15,So I changed the group,and set the
offset to the latest and it worked again.So I want to ask why the offset of
15 can't be received? and how to jump over the offset which cant not be
received to avoid the client being hanged? By the way the Kafka version was
2.0.0 and so was the kafka-clients.And the strangest is that if I close the
Kafka SASL-PLAINTEXT,this situation will not happen...
<https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=junrzzliu&uid=junrzzliu%40gmail.com&iconUrl=http%3A%2F%2Fmail-online.nosdn.127.net%2Fsmeec4f0046b25812545b59d563d602b6c.jpg&items=%5B%22junrzzliu%40gmail.com%22%5D>