You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Ewen Cheslack-Postava (JIRA)" <ji...@apache.org> on 2017/01/03 23:08:58 UTC
[jira] [Resolved] (KAFKA-4102) Is kafka mirror maker in 0.10.X
don't support mirror kafka 0.8.X?
[ https://issues.apache.org/jira/browse/KAFKA-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ewen Cheslack-Postava resolved KAFKA-4102.
------------------------------------------
Resolution: Won't Fix
[yf] Currently the Java clients are only compatible with brokers that are the same version or newer. Since MM uses the clients, it has this same limitation.
This won't be fixed for older versions of Kafka, but https://cwiki.apache.org/confluence/display/KAFKA/KIP-97%3A+Improved+Kafka+Client+RPC+Compatibility+Policy will be improving compatibility for 0.10+ brokers & clients.
> Is kafka mirror maker in 0.10.X don't support mirror kafka 0.8.X?
> -----------------------------------------------------------------
>
> Key: KAFKA-4102
> URL: https://issues.apache.org/jira/browse/KAFKA-4102
> Project: Kafka
> Issue Type: Bug
> Reporter: yf
> Fix For: 0.10.2.0
>
>
> I want this feature (https://cwiki.apache.org/confluence/display/KAFKA/KIP-3+-+Mirror+Maker+Enhancement) in kafka new version.
> But it seems can't support kafka 0.8.X ?
> Cmd: ./kafka_2.11-0.10.0.1/bin/kafka-mirror-maker.sh --producer.config svc_run/mirror_kafka_write_online2streaming/producer.properties --consumer.config svc_run/mirror_kafka_write_online2streaming/consumer.properties --whitelist sandbox
> Producer config:
> queue.buffering.max.messages=16384
> bootstrap.servers=XXX
> send.buffer.bytes=131072
> message.send.max.retries=1048576
> consumer config:
> zookeeper.connect=XXXX
> group.id=test_mirror
> consumer.timeout.ms=-1
> zookeeper.connection.timeout.ms=60000
> zookeeper.session.timeout.ms=60000
> socket.receive.buffer.bytes=-1
> auto.commit.interval.ms=5000
> auto.commit.enable=false
> But I got failures like follows:
> FetchRequest@33437902 (kafka.consumer.ConsumerFetcherThread)
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at kafka.api.FetchResponsePartitionData$.readFrom(FetchResponse.scala:38)
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:100)
> at kafka.api.TopicData$$anonfun$1.apply(FetchResponse.scala:98)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> at scala.collection.immutable.Range.foreach(Range.scala:160)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> at kafka.api.TopicData$.readFrom(FetchResponse.scala:98)
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:170)
> at kafka.api.FetchResponse$$anonfun$4.apply(FetchResponse.scala:169)
> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
> at scala.collection.immutable.Range.foreach(Range.scala:160)
> at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
> at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
> at kafka.api.FetchResponse$.readFrom(FetchResponse.scala:169)
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:135)
> at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:108)
> at kafka.consumer.ConsumerFetcherThread.fetch(ConsumerFetcherThread.scala:29)
> at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:107)
> at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:98)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)