You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Roger Vandusen <ro...@ticketmaster.com> on 2017/04/01 06:46:46 UTC
Which is True? Kafka site vs Confluent 3.2 site upgrade doc details
contradiction regarding 0.10.2 clients backward compatible to resp. 0.10.0 vs
0.10.1?
Read below and answer: So which is the source of truth ?
Is 0.10.2.0 compatible to 0.10.0 or 0.10.1?
Which site needs correction?
From current Kafka docs.
Statement from Kafka site:
https://kafka.apache.org/documentation/#upgrade
Starting with version 0.10.2, Java clients (producer and consumer) have acquired the ability to communicate with older brokers.
Version 0.10.2 clients can talk to version 0.10.0 or newer brokers.
However, if your brokers are older than 0.10.0, you must upgrade all the brokers in the Kafka cluster before upgrading your clients.
Version 0.10.2 brokers support 0.8.x and newer clients. Before 0.10.2, Kafka is backward compatible,
which means that clients from Kafka 0.8.x releases (CP 1.0.x) will work with brokers from Kafka 0.9.x, 0.10.0, 0.10.1 and 0.10.2
releases (CP 2.0.x, 3.0.x, 3.1.x and 3.2.x), but not vice-versa.
This means you always need to plan upgrades such that all brokers are upgraded before clients.
Hmm...do some more reading and research on Confluent site found this which seems to contradict the above statement:
http://docs.confluent.io/3.2.0/streams/upgrade-guide.html
Upgrading from CP 3.1.x (Kafka 0.10.1.x-cp2) to CP 3.2.0 (Kafka 0.10.2.0-cp1)
Compatibility
Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.0-cp1) are forward and backward compatible with certain Kafka clusters.
Compatibility Matrix:
Kafka Broker (columns)
Streams API (rows)
3.0.x / 0.10.0.x
3.1.x / 0.10.1.x
3.2.0 / 0.10.2.0
3.0.x / 0.10.0.x
compatible
compatible
compatible
3.1.x / 0.10.1.x
compatible
compatible
3.2.0 / 0.10.2.0
compatible
compatible
EMPHASIS ON CONTRADICTION BELOW from site desciption:
Backward-compatible to CP 3.1.x clusters (Kafka 0.10.1.x-cp2):
This is the first release allowing to upgrade your Kafka Streams application without a broker upgrade.
New Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.x-cp1) will work with older Kafka clusters running CP 3.1.x (Kafka 0.10.1.x-cp2).
Kafka clusters running CP 3.0.x (Kafka 0.10.0.x-cp1) are not compatible with new CP 3.2.0 Kafka Streams applications though.
-Roger
Re: Which is True? Kafka site vs Confluent 3.2 site upgrade doc
details contradiction regarding 0.10.2 clients backward compatible to resp.
0.10.0 vs 0.10.1?
Posted by "Matthias J. Sax" <ma...@confluent.io>.
The reason Streams API 0.10.2 is not backward compatible to 0.10.0
broker is not related to Producer/Consumer API. Streams API (as of
0.10.2) uses a new AdminClient (cf. KIP-4) for topic management that is
not supported by 0.10.0 brokers. 0.10.0 broker topics are managed via ZK
client only -- and ZK client got remove form Streams API in 0.10.2.
-Matthias
On 4/1/17 12:27 AM, Hans Jespersen wrote:
> They are both true. The Apache text is talking about the compatibility of the Producer/Consumer API and the Confluent text is talking about the Streams API.
>
> -hans
>
>> On Mar 31, 2017, at 11:46 PM, Roger Vandusen <ro...@ticketmaster.com> wrote:
>>
>> Read below and answer: So which is the source of truth ?
>> Is 0.10.2.0 compatible to 0.10.0 or 0.10.1?
>>
>> Which site needs correction?
>>
>>
>> From current Kafka docs.
>> Statement from Kafka site:
>> https://kafka.apache.org/documentation/#upgrade
>>
>> Starting with version 0.10.2, Java clients (producer and consumer) have acquired the ability to communicate with older brokers.
>> Version 0.10.2 clients can talk to version 0.10.0 or newer brokers.
>> However, if your brokers are older than 0.10.0, you must upgrade all the brokers in the Kafka cluster before upgrading your clients.
>> Version 0.10.2 brokers support 0.8.x and newer clients. Before 0.10.2, Kafka is backward compatible,
>> which means that clients from Kafka 0.8.x releases (CP 1.0.x) will work with brokers from Kafka 0.9.x, 0.10.0, 0.10.1 and 0.10.2
>> releases (CP 2.0.x, 3.0.x, 3.1.x and 3.2.x), but not vice-versa.
>> This means you always need to plan upgrades such that all brokers are upgraded before clients.
>>
>>
>> Hmm...do some more reading and research on Confluent site found this which seems to contradict the above statement:
>> http://docs.confluent.io/3.2.0/streams/upgrade-guide.html
>>
>>
>> Upgrading from CP 3.1.x (Kafka 0.10.1.x-cp2) to CP 3.2.0 (Kafka 0.10.2.0-cp1)
>> Compatibility
>> Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.0-cp1) are forward and backward compatible with certain Kafka clusters.
>>
>> Compatibility Matrix:
>>
>>
>> Kafka Broker (columns)
>>
>> Streams API (rows)
>>
>> 3.0.x / 0.10.0.x
>>
>> 3.1.x / 0.10.1.x
>>
>> 3.2.0 / 0.10.2.0
>>
>> 3.0.x / 0.10.0.x
>>
>> compatible
>>
>> compatible
>>
>> compatible
>>
>> 3.1.x / 0.10.1.x
>>
>>
>>
>> compatible
>>
>> compatible
>>
>> 3.2.0 / 0.10.2.0
>>
>>
>>
>> compatible
>>
>> compatible
>>
>>
>> EMPHASIS ON CONTRADICTION BELOW from site desciption:
>>
>> Backward-compatible to CP 3.1.x clusters (Kafka 0.10.1.x-cp2):
>> This is the first release allowing to upgrade your Kafka Streams application without a broker upgrade.
>> New Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.x-cp1) will work with older Kafka clusters running CP 3.1.x (Kafka 0.10.1.x-cp2).
>> Kafka clusters running CP 3.0.x (Kafka 0.10.0.x-cp1) are not compatible with new CP 3.2.0 Kafka Streams applications though.
>>
>> -Roger
>>
Re: Which is True? Kafka site vs Confluent 3.2 site upgrade doc details contradiction regarding 0.10.2 clients backward compatible to resp. 0.10.0 vs 0.10.1?
Posted by Hans Jespersen <ha...@confluent.io>.
They are both true. The Apache text is talking about the compatibility of the Producer/Consumer API and the Confluent text is talking about the Streams API.
-hans
> On Mar 31, 2017, at 11:46 PM, Roger Vandusen <ro...@ticketmaster.com> wrote:
>
> Read below and answer: So which is the source of truth ?
> Is 0.10.2.0 compatible to 0.10.0 or 0.10.1?
>
> Which site needs correction?
>
>
> From current Kafka docs.
> Statement from Kafka site:
> https://kafka.apache.org/documentation/#upgrade
>
> Starting with version 0.10.2, Java clients (producer and consumer) have acquired the ability to communicate with older brokers.
> Version 0.10.2 clients can talk to version 0.10.0 or newer brokers.
> However, if your brokers are older than 0.10.0, you must upgrade all the brokers in the Kafka cluster before upgrading your clients.
> Version 0.10.2 brokers support 0.8.x and newer clients. Before 0.10.2, Kafka is backward compatible,
> which means that clients from Kafka 0.8.x releases (CP 1.0.x) will work with brokers from Kafka 0.9.x, 0.10.0, 0.10.1 and 0.10.2
> releases (CP 2.0.x, 3.0.x, 3.1.x and 3.2.x), but not vice-versa.
> This means you always need to plan upgrades such that all brokers are upgraded before clients.
>
>
> Hmm...do some more reading and research on Confluent site found this which seems to contradict the above statement:
> http://docs.confluent.io/3.2.0/streams/upgrade-guide.html
>
>
> Upgrading from CP 3.1.x (Kafka 0.10.1.x-cp2) to CP 3.2.0 (Kafka 0.10.2.0-cp1)
> Compatibility
> Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.0-cp1) are forward and backward compatible with certain Kafka clusters.
>
> Compatibility Matrix:
>
>
> Kafka Broker (columns)
>
> Streams API (rows)
>
> 3.0.x / 0.10.0.x
>
> 3.1.x / 0.10.1.x
>
> 3.2.0 / 0.10.2.0
>
> 3.0.x / 0.10.0.x
>
> compatible
>
> compatible
>
> compatible
>
> 3.1.x / 0.10.1.x
>
>
>
> compatible
>
> compatible
>
> 3.2.0 / 0.10.2.0
>
>
>
> compatible
>
> compatible
>
>
> EMPHASIS ON CONTRADICTION BELOW from site desciption:
>
> Backward-compatible to CP 3.1.x clusters (Kafka 0.10.1.x-cp2):
> This is the first release allowing to upgrade your Kafka Streams application without a broker upgrade.
> New Kafka Streams applications built with CP 3.2.0 (Kafka 0.10.2.x-cp1) will work with older Kafka clusters running CP 3.1.x (Kafka 0.10.1.x-cp2).
> Kafka clusters running CP 3.0.x (Kafka 0.10.0.x-cp1) are not compatible with new CP 3.2.0 Kafka Streams applications though.
>
> -Roger
>