You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Yifan Ying <na...@gmail.com> on 2016/07/29 07:04:44 UTC
Mirrormaker between 0.8.2.1 cluster and 0.10 cluster
Hi all,
I am trying to use the mirrormaker on the 0.10 cluster to mirror the
0.8.2.1 cluster into 0.10 cluster. Then I got a bunch of consumer errors as
follows:
Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@f9533ee
(kafka.consumer.ConsumerFetcherThread)
java.nio.BufferUnderflowException
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:145)
at java.nio.ByteBuffer.get(ByteBuffer.java:694)
at kafka.api.ApiUtils$.readShortString(ApiUtils.scala:40)
at kafka.api.TopicData$.readFrom(FetchResponse.scala:96)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
at scala.collection.immutable.Range.foreach(Range.scala:160)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
at
kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
at
kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@5d1339d8
java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at
scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
at
kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
at
kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
Is there any compatibility issue when using the new mirrormaker to mirror a
0.8.2.1 cluster into a 0.10 cluster?
Thanks.
--
Yifan
Re: Mirrormaker between 0.8.2.1 cluster and 0.10 cluster
Posted by Gwen Shapira <gw...@confluent.io>.
You need to use the old mirrormaker (0.8.2.1) to mirror 0.8.2.1 to 0.10.0.0.
This is true in general - always use MirrorMaker from the older release.
Because new Kafka can talk to old clients and not the other way around.
Gwen
On Fri, Jul 29, 2016 at 12:04 AM, Yifan Ying <na...@gmail.com> wrote:
> Hi all,
>
> I am trying to use the mirrormaker on the 0.10 cluster to mirror the
> 0.8.2.1 cluster into 0.10 cluster. Then I got a bunch of consumer errors as
> follows:
>
> Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@f9533ee
> (kafka.consumer.ConsumerFetcherThread)
>
> java.nio.BufferUnderflowException
>
> at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:145)
>
> at java.nio.ByteBuffer.get(ByteBuffer.java:694)
>
> at kafka.api.ApiUtils$.readShortString(ApiUtils.scala:40)
>
> at kafka.api.TopicData$.readFrom(FetchResponse.scala:96)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
>
> at scala.collection.immutable.Range.foreach(Range.scala:160)
>
> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
>
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
>
> at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
>
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
>
> at
> kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
>
> at
> kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
>
> at
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
>
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
>
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
>
>
> Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@5d1339d8
>
> java.lang.IllegalArgumentException
>
> at java.nio.Buffer.limit(Buffer.java:267)
>
> at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
>
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
>
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>
> at scala.collection.immutable.Range.foreach(Range.scala:141)
>
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>
> at scala.collection.AbstractTraversable.map(Traversable.scala:105)
>
> at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
>
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
>
> at
> scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
>
> at scala.collection.immutable.Range.foreach(Range.scala:141)
>
> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
>
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
>
> at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
>
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
>
> at
> kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
>
> at
> kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
>
> at
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
>
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
>
> Is there any compatibility issue when using the new mirrormaker to mirror a
> 0.8.2.1 cluster into a 0.10 cluster?
>
> Thanks.
>
>
> --
> Yifan