You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "allenzhuyi (JIRA)" <ji...@apache.org> on 2017/04/09 09:48:42 UTC
[jira] [Commented] (KAFKA-2729) Cached zkVersion not equal to that
in zookeeper, broker not recovering.
[ https://issues.apache.org/jira/browse/KAFKA-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962085#comment-15962085 ]
allenzhuyi commented on KAFKA-2729:
-----------------------------------
we see the bug in kafka_2.12-0.10.2.0,when there is long lantency ping timeout.We have 3 brokers,2 Brokes find the error below
[
Partition [__consumer_offsets,9] on broker 3: Shrinking ISR for partition [__consumer_offsets,9] from 3,1,2 to 3,2 (kafka.cluster.Paition)
Partition [__consumer_offsets,9] on broker 3: Cached zkVersion [89] not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition)
]
but another one broker does'n realize the bug,the producer continuly report timeout exception and send message fail,Until finally the broker zk session expired and re-registering broker info in ZK for broker. The System is recovered.
It is a serious bug,Please help us to solve it quickly.
Thank you.
> Cached zkVersion not equal to that in zookeeper, broker not recovering.
> -----------------------------------------------------------------------
>
> Key: KAFKA-2729
> URL: https://issues.apache.org/jira/browse/KAFKA-2729
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8.2.1
> Reporter: Danil Serdyuchenko
>
> After a small network wobble where zookeeper nodes couldn't reach each other, we started seeing a large number of undereplicated partitions. The zookeeper cluster recovered, however we continued to see a large number of undereplicated partitions. Two brokers in the kafka cluster were showing this in the logs:
> {code}
> [2015-10-27 11:36:00,888] INFO Partition [__samza_checkpoint_event-creation_1,3] on broker 5: Shrinking ISR for partition [__samza_checkpoint_event-creation_1,3] from 6,5 to 5 (kafka.cluster.Partition)
> [2015-10-27 11:36:00,891] INFO Partition [__samza_checkpoint_event-creation_1,3] on broker 5: Cached zkVersion [66] not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition)
> {code}
> For all of the topics on the effected brokers. Both brokers only recovered after a restart. Our own investigation yielded nothing, I was hoping you could shed some light on this issue. Possibly if it's related to: https://issues.apache.org/jira/browse/KAFKA-1382 , however we're using 0.8.2.1.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)