You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Gaurav Pande <ga...@gmail.com> on 2023/06/13 06:10:56 UTC
Process to Upgrade Zookeeper from 2.7.0 to 3.4.1
Hello Guys,
Iam new in this space, I was going through the documentation of Upgrading
Kafka brokers to 3.4 from any previous version with Zookeeper mode , but I
couldn't find any Upgrade process for Zookeeper.
Iam using Zookeeper provided by Kafka binary in this case 2.7.0 and not
installed zk externally.
So what's the process of Upgrading Zookeeper? And should I upgrade
Zookeeper first or 3 Kafka brokers?
Note - I have single Zookeeper and 3 Kafka brokers at this point.
Regards,
Gaurav
Re: Process to Upgrade Zookeeper from 2.7.0 to 3.4.1
Posted by Enrico Olivelli <eo...@gmail.com>.
Gaurav,
Il giorno gio 15 giu 2023 alle ore 15:27 Gaurav Pande
<ga...@gmail.com> ha scritto:
>
> Hi Divij,
>
> Thanks a lot for detailed explanation.
>
> One last thing(stupid question) if you don't mind :
>
> Presently I have only a single Zookeeper running in standalone mode, My
> query is if I standup 2 new zk nodes how should I make my ensemble work
> w.r.t Should I configure two new zk nodes and restart them first or should
> I restart the standalone existing Zookeeper by adding two new zk nodes
> entry ?
Switching from standalone mode to 3 nodes is doable but you must test
it in a dev environment.
You risk that your system falls into a split brain situation.
You have to configure the list of servers in the zookeeper
configuration file and also add an "id" configuration file on each
server.
I don't know how Kafka bundles ZooKeeper so I am not going to paste
instructions here
You can ask for help on the users mailing list of ZooKeeper
https://zookeeper.apache.org/lists.html
Cheers
Enrico
>
> Regards,
> Gaurav Pande
>
> On Wed, 14 Jun, 2023, 22:50 Divij Vaidya, <di...@gmail.com> wrote:
>
> > Gaurav
> >
> > You can find the compatibility matrix for Zk here:
> >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784#KIP902:UpgradeZookeeperto3.8.1-Compatibility,Deprecation,andMigrationPlan
> >
> >
> > More specifically, for your use case of migrating from Kafka 2.7 to Kafka
> > 3.4, you will not face any hiccups. Kafka 2.7 uses Zk version 3.5.x and
> > Kafka 3.4 uses Zk version 3.6.x. Quoting from Zookeeper's documentation for
> > 3.6.0 [1]
> > *"This is the first release for 3.6 branch. It comes with lots of new
> > features and improvements around performance and security. It is also
> > introducing new APIS on the client side. ZooKeeper clients from 3.4 and 3.5
> > branch are fully compatible with 3.6 servers. The upgrade from 3.5.7 to
> > 3.6.0 can be executed as usual, no particular additional upgrade procedure
> > is needed."*
> >
> > Hence, you can update your Zk cluster first and upgrade it to 3.6.x. Your
> > existing 2.7 brokers will continue to work since they will be using Zk 3.5
> > clients to connect to the Zk cluster. 3.5 clients are compatible with Zk
> > 3.6 as per above statement. After the Zk cluster upgrade, you can perform a
> > rolling upgrade of your kafka brokers.
> >
> > Hope this helps.
> >
> > [1] https://zookeeper.apache.org/releases.html
> >
> > --
> > Divij Vaidya
> >
> >
> >
> > On Wed, Jun 14, 2023 at 2:41 PM Gaurav Pande <ga...@gmail.com> wrote:
> >
> > > Hi Luke,
> > >
> > > Thanks for helping here , iam using Zookeeper that comes with Apache
> > > kafka itself , is the same ? So not using external zk binary.
> > >
> > > Regards,
> > > Gaurav
> > >
> > > On Wed, 14 Jun, 2023, 17:43 Luke Chen, <sh...@gmail.com> wrote:
> > >
> > > > Hi Gaurav,
> > > >
> > > > Please check Zookeeper's doc for upgrading guide.
> > > >
> > > > Thanks.
> > > > Luke
> > > >
> > > > On Wed, Jun 14, 2023 at 12:03 PM Gaurav Pande <ga...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Guys,
> > > > >
> > > > > Could anyone help on this query?
> > > > >
> > > > > Regards,
> > > > > Gaurav
> > > > >
> > > > > On Tue, 13 Jun, 2023, 11:40 Gaurav Pande, <ga...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hello Guys,
> > > > > >
> > > > > > Iam new in this space, I was going through the documentation of
> > > > Upgrading
> > > > > > Kafka brokers to 3.4 from any previous version with Zookeeper
> > mode ,
> > > > > but I
> > > > > > couldn't find any Upgrade process for Zookeeper.
> > > > > >
> > > > > > Iam using Zookeeper provided by Kafka binary in this case 2.7.0 and
> > > not
> > > > > > installed zk externally.
> > > > > >
> > > > > > So what's the process of Upgrading Zookeeper? And should I upgrade
> > > > > > Zookeeper first or 3 Kafka brokers?
> > > > > >
> > > > > > Note - I have single Zookeeper and 3 Kafka brokers at this point.
> > > > > >
> > > > > > Regards,
> > > > > > Gaurav
> > > > > >
> > > > >
> > > >
> > >
> >
Re: Process to Upgrade Zookeeper from 2.7.0 to 3.4.1
Posted by Gaurav Pande <ga...@gmail.com>.
Hi Divij,
Thanks a lot for detailed explanation.
One last thing(stupid question) if you don't mind :
Presently I have only a single Zookeeper running in standalone mode, My
query is if I standup 2 new zk nodes how should I make my ensemble work
w.r.t Should I configure two new zk nodes and restart them first or should
I restart the standalone existing Zookeeper by adding two new zk nodes
entry ?
Regards,
Gaurav Pande
On Wed, 14 Jun, 2023, 22:50 Divij Vaidya, <di...@gmail.com> wrote:
> Gaurav
>
> You can find the compatibility matrix for Zk here:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784#KIP902:UpgradeZookeeperto3.8.1-Compatibility,Deprecation,andMigrationPlan
>
>
> More specifically, for your use case of migrating from Kafka 2.7 to Kafka
> 3.4, you will not face any hiccups. Kafka 2.7 uses Zk version 3.5.x and
> Kafka 3.4 uses Zk version 3.6.x. Quoting from Zookeeper's documentation for
> 3.6.0 [1]
> *"This is the first release for 3.6 branch. It comes with lots of new
> features and improvements around performance and security. It is also
> introducing new APIS on the client side. ZooKeeper clients from 3.4 and 3.5
> branch are fully compatible with 3.6 servers. The upgrade from 3.5.7 to
> 3.6.0 can be executed as usual, no particular additional upgrade procedure
> is needed."*
>
> Hence, you can update your Zk cluster first and upgrade it to 3.6.x. Your
> existing 2.7 brokers will continue to work since they will be using Zk 3.5
> clients to connect to the Zk cluster. 3.5 clients are compatible with Zk
> 3.6 as per above statement. After the Zk cluster upgrade, you can perform a
> rolling upgrade of your kafka brokers.
>
> Hope this helps.
>
> [1] https://zookeeper.apache.org/releases.html
>
> --
> Divij Vaidya
>
>
>
> On Wed, Jun 14, 2023 at 2:41 PM Gaurav Pande <ga...@gmail.com> wrote:
>
> > Hi Luke,
> >
> > Thanks for helping here , iam using Zookeeper that comes with Apache
> > kafka itself , is the same ? So not using external zk binary.
> >
> > Regards,
> > Gaurav
> >
> > On Wed, 14 Jun, 2023, 17:43 Luke Chen, <sh...@gmail.com> wrote:
> >
> > > Hi Gaurav,
> > >
> > > Please check Zookeeper's doc for upgrading guide.
> > >
> > > Thanks.
> > > Luke
> > >
> > > On Wed, Jun 14, 2023 at 12:03 PM Gaurav Pande <ga...@gmail.com>
> > > wrote:
> > >
> > > > Hi Guys,
> > > >
> > > > Could anyone help on this query?
> > > >
> > > > Regards,
> > > > Gaurav
> > > >
> > > > On Tue, 13 Jun, 2023, 11:40 Gaurav Pande, <ga...@gmail.com>
> > wrote:
> > > >
> > > > > Hello Guys,
> > > > >
> > > > > Iam new in this space, I was going through the documentation of
> > > Upgrading
> > > > > Kafka brokers to 3.4 from any previous version with Zookeeper
> mode ,
> > > > but I
> > > > > couldn't find any Upgrade process for Zookeeper.
> > > > >
> > > > > Iam using Zookeeper provided by Kafka binary in this case 2.7.0 and
> > not
> > > > > installed zk externally.
> > > > >
> > > > > So what's the process of Upgrading Zookeeper? And should I upgrade
> > > > > Zookeeper first or 3 Kafka brokers?
> > > > >
> > > > > Note - I have single Zookeeper and 3 Kafka brokers at this point.
> > > > >
> > > > > Regards,
> > > > > Gaurav
> > > > >
> > > >
> > >
> >
>
Re: Process to Upgrade Zookeeper from 2.7.0 to 3.4.1
Posted by Divij Vaidya <di...@gmail.com>.
Gaurav
You can find the compatibility matrix for Zk here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240882784#KIP902:UpgradeZookeeperto3.8.1-Compatibility,Deprecation,andMigrationPlan
More specifically, for your use case of migrating from Kafka 2.7 to Kafka
3.4, you will not face any hiccups. Kafka 2.7 uses Zk version 3.5.x and
Kafka 3.4 uses Zk version 3.6.x. Quoting from Zookeeper's documentation for
3.6.0 [1]
*"This is the first release for 3.6 branch. It comes with lots of new
features and improvements around performance and security. It is also
introducing new APIS on the client side. ZooKeeper clients from 3.4 and 3.5
branch are fully compatible with 3.6 servers. The upgrade from 3.5.7 to
3.6.0 can be executed as usual, no particular additional upgrade procedure
is needed."*
Hence, you can update your Zk cluster first and upgrade it to 3.6.x. Your
existing 2.7 brokers will continue to work since they will be using Zk 3.5
clients to connect to the Zk cluster. 3.5 clients are compatible with Zk
3.6 as per above statement. After the Zk cluster upgrade, you can perform a
rolling upgrade of your kafka brokers.
Hope this helps.
[1] https://zookeeper.apache.org/releases.html
--
Divij Vaidya
On Wed, Jun 14, 2023 at 2:41 PM Gaurav Pande <ga...@gmail.com> wrote:
> Hi Luke,
>
> Thanks for helping here , iam using Zookeeper that comes with Apache
> kafka itself , is the same ? So not using external zk binary.
>
> Regards,
> Gaurav
>
> On Wed, 14 Jun, 2023, 17:43 Luke Chen, <sh...@gmail.com> wrote:
>
> > Hi Gaurav,
> >
> > Please check Zookeeper's doc for upgrading guide.
> >
> > Thanks.
> > Luke
> >
> > On Wed, Jun 14, 2023 at 12:03 PM Gaurav Pande <ga...@gmail.com>
> > wrote:
> >
> > > Hi Guys,
> > >
> > > Could anyone help on this query?
> > >
> > > Regards,
> > > Gaurav
> > >
> > > On Tue, 13 Jun, 2023, 11:40 Gaurav Pande, <ga...@gmail.com>
> wrote:
> > >
> > > > Hello Guys,
> > > >
> > > > Iam new in this space, I was going through the documentation of
> > Upgrading
> > > > Kafka brokers to 3.4 from any previous version with Zookeeper mode ,
> > > but I
> > > > couldn't find any Upgrade process for Zookeeper.
> > > >
> > > > Iam using Zookeeper provided by Kafka binary in this case 2.7.0 and
> not
> > > > installed zk externally.
> > > >
> > > > So what's the process of Upgrading Zookeeper? And should I upgrade
> > > > Zookeeper first or 3 Kafka brokers?
> > > >
> > > > Note - I have single Zookeeper and 3 Kafka brokers at this point.
> > > >
> > > > Regards,
> > > > Gaurav
> > > >
> > >
> >
>
Re: Process to Upgrade Zookeeper from 2.7.0 to 3.4.1
Posted by Gaurav Pande <ga...@gmail.com>.
Hi Luke,
Thanks for helping here , iam using Zookeeper that comes with Apache
kafka itself , is the same ? So not using external zk binary.
Regards,
Gaurav
On Wed, 14 Jun, 2023, 17:43 Luke Chen, <sh...@gmail.com> wrote:
> Hi Gaurav,
>
> Please check Zookeeper's doc for upgrading guide.
>
> Thanks.
> Luke
>
> On Wed, Jun 14, 2023 at 12:03 PM Gaurav Pande <ga...@gmail.com>
> wrote:
>
> > Hi Guys,
> >
> > Could anyone help on this query?
> >
> > Regards,
> > Gaurav
> >
> > On Tue, 13 Jun, 2023, 11:40 Gaurav Pande, <ga...@gmail.com> wrote:
> >
> > > Hello Guys,
> > >
> > > Iam new in this space, I was going through the documentation of
> Upgrading
> > > Kafka brokers to 3.4 from any previous version with Zookeeper mode ,
> > but I
> > > couldn't find any Upgrade process for Zookeeper.
> > >
> > > Iam using Zookeeper provided by Kafka binary in this case 2.7.0 and not
> > > installed zk externally.
> > >
> > > So what's the process of Upgrading Zookeeper? And should I upgrade
> > > Zookeeper first or 3 Kafka brokers?
> > >
> > > Note - I have single Zookeeper and 3 Kafka brokers at this point.
> > >
> > > Regards,
> > > Gaurav
> > >
> >
>
Re: Process to Upgrade Zookeeper from 2.7.0 to 3.4.1
Posted by Luke Chen <sh...@gmail.com>.
Hi Gaurav,
Please check Zookeeper's doc for upgrading guide.
Thanks.
Luke
On Wed, Jun 14, 2023 at 12:03 PM Gaurav Pande <ga...@gmail.com> wrote:
> Hi Guys,
>
> Could anyone help on this query?
>
> Regards,
> Gaurav
>
> On Tue, 13 Jun, 2023, 11:40 Gaurav Pande, <ga...@gmail.com> wrote:
>
> > Hello Guys,
> >
> > Iam new in this space, I was going through the documentation of Upgrading
> > Kafka brokers to 3.4 from any previous version with Zookeeper mode ,
> but I
> > couldn't find any Upgrade process for Zookeeper.
> >
> > Iam using Zookeeper provided by Kafka binary in this case 2.7.0 and not
> > installed zk externally.
> >
> > So what's the process of Upgrading Zookeeper? And should I upgrade
> > Zookeeper first or 3 Kafka brokers?
> >
> > Note - I have single Zookeeper and 3 Kafka brokers at this point.
> >
> > Regards,
> > Gaurav
> >
>
Re: Process to Upgrade Zookeeper from 2.7.0 to 3.4.1
Posted by Gaurav Pande <ga...@gmail.com>.
Hi Guys,
Could anyone help on this query?
Regards,
Gaurav
On Tue, 13 Jun, 2023, 11:40 Gaurav Pande, <ga...@gmail.com> wrote:
> Hello Guys,
>
> Iam new in this space, I was going through the documentation of Upgrading
> Kafka brokers to 3.4 from any previous version with Zookeeper mode , but I
> couldn't find any Upgrade process for Zookeeper.
>
> Iam using Zookeeper provided by Kafka binary in this case 2.7.0 and not
> installed zk externally.
>
> So what's the process of Upgrading Zookeeper? And should I upgrade
> Zookeeper first or 3 Kafka brokers?
>
> Note - I have single Zookeeper and 3 Kafka brokers at this point.
>
> Regards,
> Gaurav
>