You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by saurabh mimani <mi...@gmail.com> on 2017/06/28 05:18:30 UTC
Is Kafka client 0.10.2.1 backward compatible
I am trying to use partitionsFor method of kafka client 2.11: 0.10.2.1
<https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1> for
topics published by Kafka 0.8.2.x, but it gives timeout. Following are
more details:
I have a small Java spark service which uses kafka client 2.11: 0.10.2.1
<https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1>.
Following is the code which works fine when I read topics published from
latest Kafka version:
Properties props = new Properties();
props.put(org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
producerConfig.getBrokerConnectionString());
props.put(org.apache.kafka.clients.producer.ProducerConfig.ACKS_CONFIG,
"all");
props.put(org.apache.kafka.clients.producer.ProducerConfig.RETRIES_CONFIG,
producerConfig.getRetry());
props.put(org.apache.kafka.clients.producer.ProducerConfig.BATCH_SIZE_CONFIG,
producerConfig.getBatchSize());
props.put(org.apache.kafka.clients.producer.ProducerConfig.LINGER_MS_CONFIG,
producerConfig.getLingerTimeInMs());
props.put(org.apache.kafka.clients.producer.ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG,
producerConfig.getRequestTimeout());
props.put(org.apache.kafka.clients.producer.ProducerConfig.MAX_BLOCK_MS_CONFIG,
producerConfig.getMaxBlockMS());
props.put(org.apache.kafka.clients.producer.ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG,
producerConfig.getMaxIdleTime());
props.put(org.apache.kafka.clients.producer.ProducerConfig.BUFFER_MEMORY_CONFIG,
maxBytesInBuffer / producerConfig.getProducersCount());
props.put(org.apache.kafka.clients.producer.ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
"org.apache.kafka.common.serialization.StringSerializer");
props.put(org.apache.kafka.clients.producer.ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
"org.apache.kafka.common.serialization.ByteArraySerializer");
producers = new Producer[1];
producers[0] = new KafkaProducer<>(props);
producers[0].partitionsFor("mYTopic").size();
There is a existing Kafka topic, where kafka version is 0.8.2.x . I wanted
to use the same code for this as well. But this code gives timeout in last
line(partitionsFor) with topic published by Kafka of version 0.8.2.x. Any
help in this regard will be appreciated.
in short: Kafka topic(published by 0.8.2.x) not able to read by 0.10.2.1
client
Best Regards
Saurabh Kumar Mimani
Re: Is Kafka client 0.10.2.1 backward compatible
Posted by Edoardo Comar <EC...@uk.ibm.com>.
Hi Saurabh
the 0.10.2.x client is backward compatible up to the 0.10.0.0 broker, not
earlier.
ciao
Edo
--------------------------------------------------
Edoardo Comar
IBM Message Hub
IBM UK Ltd, Hursley Park, SO21 2JN
Tom Crayford <tc...@heroku.com> wrote on 28/06/2017 15:21:04:
> From: Tom Crayford <tc...@heroku.com>
> To: dev@kafka.apache.org
> Date: 28/06/2017 15:21
> Subject: Re: Is Kafka client 0.10.2.1 backward compatible
>
> Kafka promises one thing and one thing only for backwards compatability,
> which is that brokers with newer versions will always support older
> clients. The inverse: old brokers with new clients is not true.
>
> On Wed, Jun 28, 2017 at 6:18 AM, saurabh mimani
<mi...@gmail.com>
> wrote:
>
> > I am trying to use partitionsFor method of kafka client 2.11:
0.10.2.1
> > <
https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1>
> > for
> > topics published by Kafka 0.8.2.x, but it gives timeout. Following
are
> > more details:
> >
> > I have a small Java spark service which uses kafka client 2.11:
0.10.2.1
> > <
https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1>.
> >
> > Following is the code which works fine when I read topics published
from
> > latest Kafka version:
> >
> > Properties props = new Properties();
> > props.put(org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_
> > SERVERS_CONFIG,
> > producerConfig.getBrokerConnectionString());
> > props.put(org.apache.kafka.clients.producer.
> > ProducerConfig.ACKS_CONFIG,
> > "all");
> > props.put(org.apache.kafka.clients.producer.
> > ProducerConfig.RETRIES_CONFIG,
> > producerConfig.getRetry());
> > props.put(org.apache.kafka.clients.producer.ProducerConfig.BATCH_SIZE_
> > CONFIG,
> > producerConfig.getBatchSize());
> > props.put(org.apache.kafka.clients.producer.ProducerConfig.LINGER_MS_
> > CONFIG,
> > producerConfig.getLingerTimeInMs());
> > props.put(org.apache.kafka.clients.producer.ProducerConfig.REQUEST_
> > TIMEOUT_MS_CONFIG,
> > producerConfig.getRequestTimeout());
> > props.put(org.apache.kafka.clients.producer.
> > ProducerConfig.MAX_BLOCK_MS_CONFIG,
> > producerConfig.getMaxBlockMS());
> > props.put(org.apache.kafka.clients.producer.
> > ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG,
> > producerConfig.getMaxIdleTime());
> > props.put(org.apache.kafka.clients.producer.
> > ProducerConfig.BUFFER_MEMORY_CONFIG,
> > maxBytesInBuffer / producerConfig.getProducersCount());
> > props.put(org.apache.kafka.clients.producer.
> > ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
> > "org.apache.kafka.common.serialization.StringSerializer");
> > props.put(org.apache.kafka.clients.producer.ProducerConfig.VALUE_
> > SERIALIZER_CLASS_CONFIG,
> > "org.apache.kafka.common.serialization.ByteArraySerializer");
> > producers = new Producer[1];
> > producers[0] = new KafkaProducer<>(props);
> > producers[0].partitionsFor("mYTopic").size();
> >
> > There is a existing Kafka topic, where kafka version is 0.8.2.x . I
wanted
> > to use the same code for this as well. But this code gives timeout in
last
> > line(partitionsFor) with topic published by Kafka of version 0.8.2.x.
Any
> > help in this regard will be appreciated.
> >
> > in short: Kafka topic(published by 0.8.2.x) not able to read by
0.10.2.1
> > client
> >
> >
> >
> > Best Regards
> >
> > Saurabh Kumar Mimani
> >
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: Is Kafka client 0.10.2.1 backward compatible
Posted by Tom Crayford <tc...@heroku.com>.
Kafka promises one thing and one thing only for backwards compatability,
which is that brokers with newer versions will always support older
clients. The inverse: old brokers with new clients is not true.
On Wed, Jun 28, 2017 at 6:18 AM, saurabh mimani <mi...@gmail.com>
wrote:
> I am trying to use partitionsFor method of kafka client 2.11: 0.10.2.1
> <https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1>
> for
> topics published by Kafka 0.8.2.x, but it gives timeout. Following are
> more details:
>
> I have a small Java spark service which uses kafka client 2.11: 0.10.2.1
> <https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/0.10.2.1>.
>
> Following is the code which works fine when I read topics published from
> latest Kafka version:
>
> Properties props = new Properties();
> props.put(org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_
> SERVERS_CONFIG,
> producerConfig.getBrokerConnectionString());
> props.put(org.apache.kafka.clients.producer.
> ProducerConfig.ACKS_CONFIG,
> "all");
> props.put(org.apache.kafka.clients.producer.
> ProducerConfig.RETRIES_CONFIG,
> producerConfig.getRetry());
> props.put(org.apache.kafka.clients.producer.ProducerConfig.BATCH_SIZE_
> CONFIG,
> producerConfig.getBatchSize());
> props.put(org.apache.kafka.clients.producer.ProducerConfig.LINGER_MS_
> CONFIG,
> producerConfig.getLingerTimeInMs());
> props.put(org.apache.kafka.clients.producer.ProducerConfig.REQUEST_
> TIMEOUT_MS_CONFIG,
> producerConfig.getRequestTimeout());
> props.put(org.apache.kafka.clients.producer.
> ProducerConfig.MAX_BLOCK_MS_CONFIG,
> producerConfig.getMaxBlockMS());
> props.put(org.apache.kafka.clients.producer.
> ProducerConfig.CONNECTIONS_MAX_IDLE_MS_CONFIG,
> producerConfig.getMaxIdleTime());
> props.put(org.apache.kafka.clients.producer.
> ProducerConfig.BUFFER_MEMORY_CONFIG,
> maxBytesInBuffer / producerConfig.getProducersCount());
> props.put(org.apache.kafka.clients.producer.
> ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
> "org.apache.kafka.common.serialization.StringSerializer");
> props.put(org.apache.kafka.clients.producer.ProducerConfig.VALUE_
> SERIALIZER_CLASS_CONFIG,
> "org.apache.kafka.common.serialization.ByteArraySerializer");
> producers = new Producer[1];
> producers[0] = new KafkaProducer<>(props);
> producers[0].partitionsFor("mYTopic").size();
>
> There is a existing Kafka topic, where kafka version is 0.8.2.x . I wanted
> to use the same code for this as well. But this code gives timeout in last
> line(partitionsFor) with topic published by Kafka of version 0.8.2.x. Any
> help in this regard will be appreciated.
>
> in short: Kafka topic(published by 0.8.2.x) not able to read by 0.10.2.1
> client
>
>
>
> Best Regards
>
> Saurabh Kumar Mimani
>