You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Marina Popova <pp...@protonmail.com.INVALID> on 2020/08/02 17:03:31 UTC
Re: Kafka compatibility with ZK
Actually, I'm very interested in your experience as well.... I'm about to start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK 3.5.6
I have Kafka and ZK as separate clusters.
My plan is :
1. rolling upgrade the Kafka cluster to 2.4 - using the inter.broker.protocol.version set to 0.11 at first
2. rolling upgrade ZK cluster to 3.5.6
3. set inter.broker.protocol.version=2.4.0 and rolling restart the Kafka cluster again
Anybody sees a problem with this approach?
thanks,
Marina
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <ak...@liftoff.io> wrote:
> Hello,
> We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK needs
> to be upgraded too (we're currently on 3.4.6). The upgrade guide says that
> "kafka has switched to the XXX version of ZK" but never says if switching
> to a newer ZK is mandatory or not. What are the guidelines on keeping Kafka
> and ZK compatible?
>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Andrey Klochkov
Re: Kafka compatibility with ZK
Posted by Peter Bukowinski <pm...@gmail.com>.
That procedure looks safe and sane to me, Marina.
> On Aug 2, 2020, at 10:04 AM, Marina Popova <pp...@protonmail.com.invalid> wrote:
>
>
> Actually, I'm very interested in your experience as well.... I'm about to start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK 3.5.6
>
> I have Kafka and ZK as separate clusters.
>
> My plan is :
> 1. rolling upgrade the Kafka cluster to 2.4 - using the inter.broker.protocol.version set to 0.11 at first
> 2. rolling upgrade ZK cluster to 3.5.6
> 3. set inter.broker.protocol.version=2.4.0 and rolling restart the Kafka cluster again
>
> Anybody sees a problem with this approach?
>
>
> thanks,
> Marina
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <ak...@liftoff.io> wrote:
>>
>> Hello,
>> We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK needs
>> to be upgraded too (we're currently on 3.4.6). The upgrade guide says that
>> "kafka has switched to the XXX version of ZK" but never says if switching
>> to a newer ZK is mandatory or not. What are the guidelines on keeping Kafka
>> and ZK compatible?
>>
>> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> Andrey Klochkov
>
>
Re: Kafka compatibility with ZK
Posted by Ismael Juma <is...@juma.me.uk>.
Answers inline.
On Fri, Aug 14, 2020 at 6:46 AM Marina Popova
<pp...@protonmail.com.invalid> wrote:
> thank you, Ismael!
> a follow up question (or two :) ):
> -- our clients (consumers and producers) are all upgraded to Kafka 2.4.0 /
> ZK 3.5.6 version at this time, and since it is a very involved process to
> upgrade them, I'd want to leave them at that for now. Any issues you can
> think of running clients at these versions and brokers at version 2.5.1/ZK
> 3.5.8 ?
>
Kafka clients don't use ZK, so we don't have to worry about that aspect.
There are no issues running 2.4.0 clients with 2.5.1 brokers. However,
there have been some important client fixes since 2.4.0 was released so
it's a good idea to upgrade when you have a chance. It can be done
independently of the brokers upgrade,
-- the brokers upgrade from 0.11 directly to 2.5.1: any words of wisdom on
> what particular properties I should pay attention to, that might have
> changed default values or semantics in between? I have compared server,
> consumer, producer and zookeeper properties files between those
> distributions and did not se anything critical, but just wanted to make
> sure....
>
I highly recommend reviewing the "notable changes" section for each release
in the following doc:
https://kafka.apache.org/documentation/#upgrade
A couple of things I would mention:
1. Run the cluster with the existing inter-broker protocol version for a
few days and make sure that all is good with the applications before
bumping it to the current version. You will not be able to downgrade the
inter-broker protocol version after you bump it (the details are mentioned
in the documentation above).
2. The request metrics include a version tag and you may have to adjust
your monitoring tools.
Hope it helps.
Ismael
>
> Thank you!
> Marina
>
> Sent with [ProtonMail](https://protonmail.com) Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Tuesday, August 11, 2020 6:02 PM, Ismael Juma <is...@juma.me.uk>
> wrote:
>
> > Note that 2.5.1 has now been released.
> >
> > Ismael
> >
> > On Sun, Aug 2, 2020 at 11:13 AM Ismael Juma <is...@juma.me.uk> wrote:
> >
> >> Hi Marina,
> >>
> >> If you upgrade to ZK 3.5.x, I highly recommend going with 3.5.8. It has
> a few critical fixes when compared to 3.5.6. Also, Kafka 2.5.1 should be
> out in a few days and I'd recommend that over 2.4.1.
> >>
> >> Ismael
> >>
> >> On Sun, Aug 2, 2020 at 10:04 AM Marina Popova <pp...@protonmail.com.invalid>
> wrote:
> >>
> >>> Actually, I'm very interested in your experience as well.... I'm about
> to start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka
> 2.4/ZK 3.5.6
> >>>
> >>> I have Kafka and ZK as separate clusters.
> >>>
> >>> My plan is :
> >>> 1. rolling upgrade the Kafka cluster to 2.4 - using the
> inter.broker.protocol.version set to 0.11 at first
> >>> 2. rolling upgrade ZK cluster to 3.5.6
> >>> 3. set inter.broker.protocol.version=2.4.0 and rolling restart the
> Kafka cluster again
> >>>
> >>> Anybody sees a problem with this approach?
> >>>
> >>> thanks,
> >>> Marina
> >>>
> >>> Sent with ProtonMail Secure Email.
> >>>
> >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> >>> On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <
> aklochkov@liftoff.io> wrote:
> >>>
> >>>> Hello,
> >>>> We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK
> needs
> >>>> to be upgraded too (we're currently on 3.4.6). The upgrade guide says
> that
> >>>> "kafka has switched to the XXX version of ZK" but never says if
> switching
> >>>> to a newer ZK is mandatory or not. What are the guidelines on keeping
> Kafka
> >>>> and ZK compatible?
> >>>>
> >>>>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>>
> >>>> Andrey Klochkov
Re: Kafka compatibility with ZK
Posted by Marina Popova <pp...@protonmail.com.INVALID>.
thank you, Ismael!
a follow up question (or two :) ):
-- our clients (consumers and producers) are all upgraded to Kafka 2.4.0 / ZK 3.5.6 version at this time, and since it is a very involved process to upgrade them, I'd want to leave them at that for now. Any issues you can think of running clients at these versions and brokers at version 2.5.1/ZK 3.5.8 ?
-- the brokers upgrade from 0.11 directly to 2.5.1: any words of wisdom on what particular properties I should pay attention to, that might have changed default values or semantics in between? I have compared server, consumer, producer and zookeeper properties files between those distributions and did not se anything critical, but just wanted to make sure....
Thank you!
Marina
Sent with [ProtonMail](https://protonmail.com) Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, August 11, 2020 6:02 PM, Ismael Juma <is...@juma.me.uk> wrote:
> Note that 2.5.1 has now been released.
>
> Ismael
>
> On Sun, Aug 2, 2020 at 11:13 AM Ismael Juma <is...@juma.me.uk> wrote:
>
>> Hi Marina,
>>
>> If you upgrade to ZK 3.5.x, I highly recommend going with 3.5.8. It has a few critical fixes when compared to 3.5.6. Also, Kafka 2.5.1 should be out in a few days and I'd recommend that over 2.4.1.
>>
>> Ismael
>>
>> On Sun, Aug 2, 2020 at 10:04 AM Marina Popova <pp...@protonmail.com.invalid> wrote:
>>
>>> Actually, I'm very interested in your experience as well.... I'm about to start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK 3.5.6
>>>
>>> I have Kafka and ZK as separate clusters.
>>>
>>> My plan is :
>>> 1. rolling upgrade the Kafka cluster to 2.4 - using the inter.broker.protocol.version set to 0.11 at first
>>> 2. rolling upgrade ZK cluster to 3.5.6
>>> 3. set inter.broker.protocol.version=2.4.0 and rolling restart the Kafka cluster again
>>>
>>> Anybody sees a problem with this approach?
>>>
>>> thanks,
>>> Marina
>>>
>>> Sent with ProtonMail Secure Email.
>>>
>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>> On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <ak...@liftoff.io> wrote:
>>>
>>>> Hello,
>>>> We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK needs
>>>> to be upgraded too (we're currently on 3.4.6). The upgrade guide says that
>>>> "kafka has switched to the XXX version of ZK" but never says if switching
>>>> to a newer ZK is mandatory or not. What are the guidelines on keeping Kafka
>>>> and ZK compatible?
>>>>
>>>> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>
>>>> Andrey Klochkov
Re: Kafka compatibility with ZK
Posted by Ismael Juma <is...@juma.me.uk>.
Note that 2.5.1 has now been released.
Ismael
On Sun, Aug 2, 2020 at 11:13 AM Ismael Juma <is...@juma.me.uk> wrote:
> Hi Marina,
>
> If you upgrade to ZK 3.5.x, I highly recommend going with 3.5.8. It has a
> few critical fixes when compared to 3.5.6. Also, Kafka 2.5.1 should be out
> in a few days and I'd recommend that over 2.4.1.
>
> Ismael
>
> On Sun, Aug 2, 2020 at 10:04 AM Marina Popova
> <pp...@protonmail.com.invalid> wrote:
>
>>
>> Actually, I'm very interested in your experience as well.... I'm about to
>> start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK
>> 3.5.6
>>
>> I have Kafka and ZK as separate clusters.
>>
>> My plan is :
>> 1. rolling upgrade the Kafka cluster to 2.4 - using the
>> inter.broker.protocol.version set to 0.11 at first
>> 2. rolling upgrade ZK cluster to 3.5.6
>> 3. set inter.broker.protocol.version=2.4.0 and rolling restart the Kafka
>> cluster again
>>
>> Anybody sees a problem with this approach?
>>
>>
>> thanks,
>> Marina
>>
>>
>> Sent with ProtonMail Secure Email.
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <ak...@liftoff.io>
>> wrote:
>>
>> > Hello,
>> > We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK
>> needs
>> > to be upgraded too (we're currently on 3.4.6). The upgrade guide says
>> that
>> > "kafka has switched to the XXX version of ZK" but never says if
>> switching
>> > to a newer ZK is mandatory or not. What are the guidelines on keeping
>> Kafka
>> > and ZK compatible?
>> >
>> >
>> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> >
>> > Andrey Klochkov
>>
>>
>>
Re: Kafka compatibility with ZK
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Marina,
If you upgrade to ZK 3.5.x, I highly recommend going with 3.5.8. It has a
few critical fixes when compared to 3.5.6. Also, Kafka 2.5.1 should be out
in a few days and I'd recommend that over 2.4.1.
Ismael
On Sun, Aug 2, 2020 at 10:04 AM Marina Popova
<pp...@protonmail.com.invalid> wrote:
>
> Actually, I'm very interested in your experience as well.... I'm about to
> start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK
> 3.5.6
>
> I have Kafka and ZK as separate clusters.
>
> My plan is :
> 1. rolling upgrade the Kafka cluster to 2.4 - using the
> inter.broker.protocol.version set to 0.11 at first
> 2. rolling upgrade ZK cluster to 3.5.6
> 3. set inter.broker.protocol.version=2.4.0 and rolling restart the Kafka
> cluster again
>
> Anybody sees a problem with this approach?
>
>
> thanks,
> Marina
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <ak...@liftoff.io>
> wrote:
>
> > Hello,
> > We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK
> needs
> > to be upgraded too (we're currently on 3.4.6). The upgrade guide says
> that
> > "kafka has switched to the XXX version of ZK" but never says if switching
> > to a newer ZK is mandatory or not. What are the guidelines on keeping
> Kafka
> > and ZK compatible?
> >
> >
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > Andrey Klochkov
>
>
>
Re: Kafka compatibility with ZK
Posted by Ismael Juma <is...@juma.me.uk>.
Great, thanks for the update Manoj and and Andrey! I recommend the simple
upgrade from 2.5.0 to 2.5.1 to get the fix for
https://issues.apache.org/jira/browse/KAFKA-9835.
Ismael
On Thu, Sep 3, 2020 at 9:01 PM <Ma...@cognizant.com> wrote:
> We also upgraded kafka 2.2.1 to kafka 2.5.0 and kept same zookeeper . no
> issued reported .
> Later we also upgraded zookeeper to 3.5.8 . all good .
>
> On 9/3/20, 8:42 PM, "Andrey Klochkov" <ak...@liftoff.io> wrote:
>
> [External]
>
>
> Hello all,
> FWIW we upgraded to Kafka 2.4.1 and kept ZK at 3.4.6, no issues
> noticed.
>
> On Sun, Aug 2, 2020 at 10:04 AM Marina Popova
> <pp...@protonmail.com.invalid> wrote:
>
> >
> > Actually, I'm very interested in your experience as well.... I'm
> about to
> > start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka
> 2.4/ZK
> > 3.5.6
> >
> > I have Kafka and ZK as separate clusters.
> >
> > My plan is :
> > 1. rolling upgrade the Kafka cluster to 2.4 - using the
> > inter.broker.protocol.version set to 0.11 at first
> > 2. rolling upgrade ZK cluster to 3.5.6
> > 3. set inter.broker.protocol.version=2.4.0 and rolling restart the
> Kafka
> > cluster again
> >
> > Anybody sees a problem with this approach?
> >
> >
> > thanks,
> > Marina
> >
> >
> > Sent with ProtonMail Secure Email.
> >
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <
> aklochkov@liftoff.io>
> > wrote:
> >
> > > Hello,
> > > We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if
> ZK
> > needs
> > > to be upgraded too (we're currently on 3.4.6). The upgrade guide
> says
> > that
> > > "kafka has switched to the XXX version of ZK" but never says if
> switching
> > > to a newer ZK is mandatory or not. What are the guidelines on
> keeping
> > Kafka
> > > and ZK compatible?
> > >
> > >
> >
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >
> > > Andrey Klochkov
> >
> >
> >
>
> --
> Andrey Klochkov
>
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored.
>
Re: Kafka compatibility with ZK
Posted by Ma...@cognizant.com.
We also upgraded kafka 2.2.1 to kafka 2.5.0 and kept same zookeeper . no issued reported .
Later we also upgraded zookeeper to 3.5.8 . all good .
On 9/3/20, 8:42 PM, "Andrey Klochkov" <ak...@liftoff.io> wrote:
[External]
Hello all,
FWIW we upgraded to Kafka 2.4.1 and kept ZK at 3.4.6, no issues noticed.
On Sun, Aug 2, 2020 at 10:04 AM Marina Popova
<pp...@protonmail.com.invalid> wrote:
>
> Actually, I'm very interested in your experience as well.... I'm about to
> start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK
> 3.5.6
>
> I have Kafka and ZK as separate clusters.
>
> My plan is :
> 1. rolling upgrade the Kafka cluster to 2.4 - using the
> inter.broker.protocol.version set to 0.11 at first
> 2. rolling upgrade ZK cluster to 3.5.6
> 3. set inter.broker.protocol.version=2.4.0 and rolling restart the Kafka
> cluster again
>
> Anybody sees a problem with this approach?
>
>
> thanks,
> Marina
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <ak...@liftoff.io>
> wrote:
>
> > Hello,
> > We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK
> needs
> > to be upgraded too (we're currently on 3.4.6). The upgrade guide says
> that
> > "kafka has switched to the XXX version of ZK" but never says if switching
> > to a newer ZK is mandatory or not. What are the guidelines on keeping
> Kafka
> > and ZK compatible?
> >
> >
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > Andrey Klochkov
>
>
>
--
Andrey Klochkov
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and privileged information. If you are not the intended recipient(s), please reply to the sender and destroy all copies of the original message. Any unauthorized review, use, disclosure, dissemination, forwarding, printing or copying of this email, and/or any action taken in reliance on the contents of this e-mail is strictly prohibited and may be unlawful. Where permitted by applicable law, this e-mail and other e-mail communications sent to and from Cognizant e-mail addresses may be monitored.
Re: Kafka compatibility with ZK
Posted by Andrey Klochkov <ak...@liftoff.io>.
Hello all,
FWIW we upgraded to Kafka 2.4.1 and kept ZK at 3.4.6, no issues noticed.
On Sun, Aug 2, 2020 at 10:04 AM Marina Popova
<pp...@protonmail.com.invalid> wrote:
>
> Actually, I'm very interested in your experience as well.... I'm about to
> start the same (similar) upgrade - from Kafka 0.11/ZK3.4.13 to Kafka 2.4/ZK
> 3.5.6
>
> I have Kafka and ZK as separate clusters.
>
> My plan is :
> 1. rolling upgrade the Kafka cluster to 2.4 - using the
> inter.broker.protocol.version set to 0.11 at first
> 2. rolling upgrade ZK cluster to 3.5.6
> 3. set inter.broker.protocol.version=2.4.0 and rolling restart the Kafka
> cluster again
>
> Anybody sees a problem with this approach?
>
>
> thanks,
> Marina
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, July 23, 2020 4:01 PM, Andrey Klochkov <ak...@liftoff.io>
> wrote:
>
> > Hello,
> > We're upgrading our Kafka from 1.1.0 to 2.4.1 and I'm wondering if ZK
> needs
> > to be upgraded too (we're currently on 3.4.6). The upgrade guide says
> that
> > "kafka has switched to the XXX version of ZK" but never says if switching
> > to a newer ZK is mandatory or not. What are the guidelines on keeping
> Kafka
> > and ZK compatible?
> >
> >
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > Andrey Klochkov
>
>
>
--
Andrey Klochkov