You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Ewen Cheslack-Postava (JIRA)" <ji...@apache.org> on 2017/01/03 23:08:58 UTC
[jira] [Updated] (KAFKA-4102) Is kafka mirror maker in 0.10.X don't
support mirror kafka 0.8.X?
[ https://issues.apache.org/jira/browse/KAFKA-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ewen Cheslack-Postava updated KAFKA-4102:
-----------------------------------------
Fix Version/s: (was: 0.10.2.0)
> Is kafka mirror maker in 0.10.X don't support mirror kafka 0.8.X?
> -----------------------------------------------------------------
>
> Key: KAFKA-4102
> URL: https://issues.apache.org/jira/browse/KAFKA-4102
> Project: Kafka
> Issue Type: Bug
> Reporter: yf
>
> I want this feature (https://cwiki.apache.org/confluence/display/KAFKA/KIP-3+-+Mirror+Maker+Enhancement) in kafka new version.
> But it seems can't support kafka 0.8.X ?
> Cmd: ./kafka_2.11-0.10.0.1/bin/kafka-mirror-maker.sh --producer.config svc_run/mirror_kafka_write_online2streaming/producer.properties --consumer.config svc_run/mirror_kafka_write_online2streaming/consumer.properties --whitelist sandbox
> Producer config:
> queue.buffering.max.messages=16384
> bootstrap.servers=XXX
> send.buffer.bytes=131072
> message.send.max.retries=1048576
> consumer config:
> zookeeper.connect=XXXX
> group.id=test_mirror
> consumer.timeout.ms=-1
> zookeeper.connection.timeout.ms=60000
> zookeeper.session.timeout.ms=60000
> socket.receive.buffer.bytes=-1
> auto.commit.interval.ms=5000
> auto.commit.enable=false
> But I got failures like follows:
> FetchRequest@33437902 (kafka.consumer.ConsumerFetcherThread)
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.immutable.Range.foreach(Range.scala:160)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at scala.collection.immutable.Range.foreach(Range.scala:160)
> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
> at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
> at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
> at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)