You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "leibo (JIRA)" <ji...@apache.org> on 2018/11/28 07:32:00 UTC
[jira] [Updated] (KAFKA-7684) kafka consumer SchemaException
occurred: Error reading field 'brokers':
[ https://issues.apache.org/jira/browse/KAFKA-7684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
leibo updated KAFKA-7684:
-------------------------
Description:
kafka consumer occurred org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'brokers': Error reading array of size 65535, only 22 bytes available when the broker is not work due to a l*ong time full GC.*
*Exception Detail:*
org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'brokers': Error reading array of size 65535, only 22 bytes available
at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:75)
at org.apache.kafka.common.protocol.ApiKeys.parseResponse(ApiKeys.java:153)
at org.apache.kafka.clients.NetworkClient.parseStructMaybeUpdateThrottleTimeMetrics(NetworkClient.java:560)
at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:657)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:442)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:208)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:184)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:217)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:203)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:286)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1078)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1043)
at com.zte.ums.zenap.cometd.ReceiveKafkaMessage.run(ReceiveKafkaMessage.java:40)
*After this , the following exception occurred:*
2018-11-21 19:43:52 097 ERROR [org.apache.kafka.clients.consumer.KafkaConsumer][Thread-22] - Failed to close coordinator
java.lang.IllegalStateException: Correlation id for response (3097236) does not match request (3097235), request header: \{api_key=8,api_version=3,correlation_id=3097235,client_id=pm-task_100.100.0.43_8}
at org.apache.kafka.clients.NetworkClient.correlate(NetworkClient.java:752)
at org.apache.kafka.clients.NetworkClient.parseStructMaybeUpdateThrottleTimeMetrics(NetworkClient.java:561)
at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:657)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:442)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:208)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:199)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitPendingRequests(ConsumerNetworkClient.java:284)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.close(AbstractCoordinator.java:706)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.close(ConsumerCoordinator.java:503)
at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1605)
at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1573)
at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1549)
at com.zte.ums.zenap.pm.common.kafka.BasicKafkaConsumerWorker$1.run(BasicKafkaConsumerWorker.java:77)
at java.lang.Thread.run(Thread.java:811)
18-11-21 19:43:52 099 ERROR [com.zte.ums.zenap.dw.ext.FrameWorkThreadGroup][Thread-24] - Thread-24 throw this exception
org.apache.kafka.common.KafkaException: Failed to close kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1623)
at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1573)
at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1549)
at com.zte.ums.zenap.pm.common.kafka.BasicKafkaConsumerWorker$1.run(BasicKafkaConsumerWorker.java:77)
at java.lang.Thread.run(Thread.java:811)
Kafka broker version: *0.11.0.1*
Kafka consumer version: *0.11.0.1*
*More Info:*
In the same time , the kafka broker have a long time(about 16 seconds) Full GC , and lots of consumer rebalance and zookeeper session timeout occurred.
was:
kafka consumer occurred org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'brokers': Error reading array of size 65535, only 22 bytes available when the broker is not work due to a long time full GC.
Exception Detail:
org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'brokers': Error reading array of size 65535, only 22 bytes available
at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:75)
at org.apache.kafka.common.protocol.ApiKeys.parseResponse(ApiKeys.java:153)
at org.apache.kafka.clients.NetworkClient.parseStructMaybeUpdateThrottleTimeMetrics(NetworkClient.java:560)
at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:657)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:442)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:208)
at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:184)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:217)
at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:203)
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:286)
at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1078)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1043)
at com.zte.ums.zenap.cometd.ReceiveKafkaMessage.run(ReceiveKafkaMessage.java:40)
> kafka consumer SchemaException occurred: Error reading field 'brokers':
> ------------------------------------------------------------------------
>
> Key: KAFKA-7684
> URL: https://issues.apache.org/jira/browse/KAFKA-7684
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.11.0.1
> Reporter: leibo
> Priority: Major
>
> kafka consumer occurred org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'brokers': Error reading array of size 65535, only 22 bytes available when the broker is not work due to a l*ong time full GC.*
> *Exception Detail:*
> org.apache.kafka.common.protocol.types.SchemaException: Error reading field 'brokers': Error reading array of size 65535, only 22 bytes available
> at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:75)
> at org.apache.kafka.common.protocol.ApiKeys.parseResponse(ApiKeys.java:153)
> at org.apache.kafka.clients.NetworkClient.parseStructMaybeUpdateThrottleTimeMetrics(NetworkClient.java:560)
> at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:657)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:442)
> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:208)
> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:184)
> at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:217)
> at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:203)
> at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:286)
> at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1078)
> at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1043)
> at com.zte.ums.zenap.cometd.ReceiveKafkaMessage.run(ReceiveKafkaMessage.java:40)
> *After this , the following exception occurred:*
> 2018-11-21 19:43:52 097 ERROR [org.apache.kafka.clients.consumer.KafkaConsumer][Thread-22] - Failed to close coordinator
> java.lang.IllegalStateException: Correlation id for response (3097236) does not match request (3097235), request header: \{api_key=8,api_version=3,correlation_id=3097235,client_id=pm-task_100.100.0.43_8}
> at org.apache.kafka.clients.NetworkClient.correlate(NetworkClient.java:752)
> at org.apache.kafka.clients.NetworkClient.parseStructMaybeUpdateThrottleTimeMetrics(NetworkClient.java:561)
> at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:657)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:442)
> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:208)
> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:199)
> at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitPendingRequests(ConsumerNetworkClient.java:284)
> at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.close(AbstractCoordinator.java:706)
> at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.close(ConsumerCoordinator.java:503)
> at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1605)
> at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1573)
> at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1549)
> at com.zte.ums.zenap.pm.common.kafka.BasicKafkaConsumerWorker$1.run(BasicKafkaConsumerWorker.java:77)
> at java.lang.Thread.run(Thread.java:811)
> 18-11-21 19:43:52 099 ERROR [com.zte.ums.zenap.dw.ext.FrameWorkThreadGroup][Thread-24] - Thread-24 throw this exception
> org.apache.kafka.common.KafkaException: Failed to close kafka consumer
> at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1623)
> at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1573)
> at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1549)
> at com.zte.ums.zenap.pm.common.kafka.BasicKafkaConsumerWorker$1.run(BasicKafkaConsumerWorker.java:77)
> at java.lang.Thread.run(Thread.java:811)
>
> Kafka broker version: *0.11.0.1*
> Kafka consumer version: *0.11.0.1*
> *More Info:*
> In the same time , the kafka broker have a long time(about 16 seconds) Full GC , and lots of consumer rebalance and zookeeper session timeout occurred.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)