You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Upendra Yadav <up...@gmail.com> on 2019/12/03 11:58:15 UTC

Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

Hi Team,

I'm getting below exception in one of the kafka broker while restarting.
But server is starting and serving properly.

Why this error is coming?
Is there any severe issue due to this?
And how to resolve it?

ERROR [Group Metadata Manager on Broker 3]: Error in loading offsets from
[__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager)
kafka.common.KafkaException: Unknown group metadata version 1
at
kafka.coordinator.GroupMetadataManager$.schemaForGroup(GroupMetadataManager.scala:764)
at
kafka.coordinator.GroupMetadataManager$.readGroupMessageValue(GroupMetadataManager.scala:934)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:412)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:383)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at kafka.message.MessageSet.foreach(MessageSet.scala:71)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply$mcV$sp(GroupMetadataManager.scala:383)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:239)
at
kafka.coordinator.GroupMetadataManager.kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1(GroupMetadataManager.scala:374)
at
kafka.coordinator.GroupMetadataManager$$anonfun$loadGroupsForPartition$1.apply$mcV$sp(GroupMetadataManager.scala:353)
at
kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Re: Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

Posted by Upendra Yadav <up...@gmail.com>.

Hi Team,

We are also facing slowness in producing messages for some of topics. Kafka
acknowledgement is taking more than 1 seconds sometimes. Normally it takes
1 to 2 milliseconds.

Do you think, this exception leads to slow acknowledge?
Your help is much appreciated, we are facing this issue in our production
setup. Another datacenter with same workload works without any issue.

On Wed, Dec 4, 2019 at 5:46 PM Upendra Yadav <up...@gmail.com> wrote:

>
>
> ---------- Forwarded message ---------
> From: Upendra Yadav <up...@gmail.com>
> Date: Tue, Dec 3, 2019 at 5:28 PM
> Subject: Group Metadata Manager on Broker 3]: Error in loading offsets
> from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager)
> kafka.common.KafkaException: Unknown group metadata version 1
> To: <us...@kafka.apache.org>
>
>
> Hi Team,
>
> I'm getting below exception in one of the kafka broker while restarting.
> But server is starting and serving properly.
>
> Why this error is coming?
> Is there any severe issue due to this?
> And how to resolve it?
>
> ERROR [Group Metadata Manager on Broker 3]: Error in loading offsets from
> [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager)
> kafka.common.KafkaException: Unknown group metadata version 1
> at
> kafka.coordinator.GroupMetadataManager$.schemaForGroup(GroupMetadataManager.scala:764)
> at
> kafka.coordinator.GroupMetadataManager$.readGroupMessageValue(GroupMetadataManager.scala:934)
> at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:412)
> at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:383)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at kafka.message.MessageSet.foreach(MessageSet.scala:71)
> at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply$mcV$sp(GroupMetadataManager.scala:383)
> at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
> at
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
> at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
> at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:239)
> at
> kafka.coordinator.GroupMetadataManager.kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1(GroupMetadataManager.scala:374)
> at
> kafka.coordinator.GroupMetadataManager$$anonfun$loadGroupsForPartition$1.apply$mcV$sp(GroupMetadataManager.scala:353)
> at
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>

Re: Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

Posted by deng ziming <de...@gmail.com>.

I once encountered the same problem when I installed 2 versions of Kafka
with the same config, and startup version A Kafka firstly and shutdown it,
and misoperated to start version B Kafka secondly and write some data to it
and shutdown it, and then startup version A Kafka.

I think it's OK to ignore it, maybe some multiple version altered the same
${log.dirs}

On Wed, Dec 4, 2019 at 11:31 PM Upendra Yadav <up...@gmail.com> wrote:

> ---------- Forwarded message ---------
> From: Upendra Yadav <up...@gmail.com>
> Date: Tue, Dec 3, 2019 at 5:28 PM
> Subject: Group Metadata Manager on Broker 3]: Error in loading offsets from
> [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager)
> kafka.common.KafkaException: Unknown group metadata version 1
> To: <us...@kafka.apache.org>
>
>
> Hi Team,
>
> I'm getting below exception in one of the kafka broker while restarting.
> But server is starting and serving properly.
>
> Why this error is coming?
> Is there any severe issue due to this?
> And how to resolve it?
>
> ERROR [Group Metadata Manager on Broker 3]: Error in loading offsets from
> [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager)
> kafka.common.KafkaException: Unknown group metadata version 1
> at
>
> kafka.coordinator.GroupMetadataManager$.schemaForGroup(GroupMetadataManager.scala:764)
> at
>
> kafka.coordinator.GroupMetadataManager$.readGroupMessageValue(GroupMetadataManager.scala:934)
> at
>
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:412)
> at
>
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:383)
> at scala.collection.Iterator$class.foreach(Iterator.scala:893)
> at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at kafka.message.MessageSet.foreach(MessageSet.scala:71)
> at
>
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply$mcV$sp(GroupMetadataManager.scala:383)
> at
>
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
> at
>
> kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
> at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
> at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:239)
> at
>
> kafka.coordinator.GroupMetadataManager.kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1(GroupMetadataManager.scala:374)
> at
>
> kafka.coordinator.GroupMetadataManager$$anonfun$loadGroupsForPartition$1.apply$mcV$sp(GroupMetadataManager.scala:353)
> at
>
> kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
> at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>

Fwd: Group Metadata Manager on Broker 3]: Error in loading offsets from [__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager) kafka.common.KafkaException: Unknown group metadata version 1

Posted by Upendra Yadav <up...@gmail.com>.

---------- Forwarded message ---------
From: Upendra Yadav <up...@gmail.com>
Date: Tue, Dec 3, 2019 at 5:28 PM
Subject: Group Metadata Manager on Broker 3]: Error in loading offsets from
[__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager)
kafka.common.KafkaException: Unknown group metadata version 1
To: <us...@kafka.apache.org>


Hi Team,

I'm getting below exception in one of the kafka broker while restarting.
But server is starting and serving properly.

Why this error is coming?
Is there any severe issue due to this?
And how to resolve it?

ERROR [Group Metadata Manager on Broker 3]: Error in loading offsets from
[__consumer_offsets,34] (kafka.coordinator.GroupMetadataManager)
kafka.common.KafkaException: Unknown group metadata version 1
at
kafka.coordinator.GroupMetadataManager$.schemaForGroup(GroupMetadataManager.scala:764)
at
kafka.coordinator.GroupMetadataManager$.readGroupMessageValue(GroupMetadataManager.scala:934)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:412)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1$$anonfun$apply$mcV$sp$1.apply(GroupMetadataManager.scala:383)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at kafka.message.MessageSet.foreach(MessageSet.scala:71)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply$mcV$sp(GroupMetadataManager.scala:383)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
at
kafka.coordinator.GroupMetadataManager$$anonfun$kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1$1.apply(GroupMetadataManager.scala:374)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:239)
at
kafka.coordinator.GroupMetadataManager.kafka$coordinator$GroupMetadataManager$$loadGroupsAndOffsets$1(GroupMetadataManager.scala:374)
at
kafka.coordinator.GroupMetadataManager$$anonfun$loadGroupsForPartition$1.apply$mcV$sp(GroupMetadataManager.scala:353)
at
kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:110)
at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:56)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)