You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Sankalp Bhatia <sa...@gmail.com> on 2020/07/14 20:51:51 UTC
Upgrading existing non-TLS cluster with no downtime
Hi All,
I am trying to follow the section "Upgrading existing non-TLS cluster with
no downtime" in the zookeeper guide :
https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
I have an ensemble of 3 servers. I have a couple of questions:
1) When I set sslQuorum=true and portUnification=true on the first server,
does it go out of the quorum? And when these properties are set in the
second server, a new quorum of first and second server is formed and now
the third server is out of quorum. When the 3rd server follows suit, it is
added back to the quorum.
If this is the case, what is the use of a the port-unification feature
here?
2) The guideline says to check after restarting every broker that the
quorum is healthy, is there any metric to track that?
Thanks,
Sankalp
Re: Upgrading existing non-TLS cluster with no downtime
Posted by Sankalp Bhatia <sa...@gmail.com>.
Thanks Mate for the responses. Brings a lot of clarity to me. I was able to
get it working this time without downtime. Don't know what I did wrong the
last time though.
On Mon, 20 Jul 2020 at 12:57, Szalay-Bekő Máté <sz...@gmail.com>
wrote:
> echo "stat" | nc localhost 2182Hi,
>
> I guess this is the part you are referring:
>
> https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster
> (your link was pointing to the 3.3.2 admin guide where this chapter was
> missing)
>
> > 1) When I set sslQuorum=true and portUnification=true on the first
> server,
> does it go out of the quorum? And when these properties are set in the
> second server, a new quorum of first and second server is formed and now
> the third server is out of quorum. When the 3rd server follows suit, it is
> added back to the quorum.
>
> the "sslQuorum=true and portUnification=true" setting is needed in step 4
> (although the numbering is bad in the markdown...). After step 3 you
> already have a 3 server quorum up with portUnification=true, meaning the
> cluster can handle both TLS/SSL and regular/non-secure connections. So when
> you restart server 1 with sslQuorum=true, then it will be able to re-join
> to the quorum, as server 2 and 3 are capable of handling SSL connections
> (even if they are not using it for connection initiation). So ideally
> between restarting each servers with sslQuorum=true, you always should have
> a 3 node full quorum.
>
> > 2) The guideline says to check after restarting every broker that the
> quorum is healthy, is there any metric to track that?
>
> I send the "stat" command to all nodes to see if everyone is connected to
> the quorum. E.g.: echo "stat" | nc localhost 2181
> I usually use 4-letter-word commands but the REST admin API works as well,
> and actually that is the officially recommended way, as the 4-letter-words
> are / will be deprecated some time.
> For the admin server see:
> https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#sc_adminserver
>
> Kind regards,
> Mate
>
> On Tue, Jul 14, 2020 at 10:52 PM Sankalp Bhatia <sankalpbhatia92@gmail.com
> >
> wrote:
>
> > +users
> >
> > On Tue, 14 Jul 2020 at 21:51, Sankalp Bhatia <sa...@gmail.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > I am trying to follow the section "Upgrading existing non-TLS cluster
> > with
> > > no downtime" in the zookeeper guide :
> > > https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
> > >
> > > I have an ensemble of 3 servers. I have a couple of questions:
> > >
> > > 1) When I set sslQuorum=true and portUnification=true on the first
> > > server, does it go out of the quorum? And when these properties are set
> > > in the second server, a new quorum of first and second server is formed
> > and
> > > now the third server is out of quorum. When the 3rd server follows
> suit,
> > it
> > > is added back to the quorum.
> > >
> > > If this is the case, what is the use of a the port-unification feature
> > > here?
> > >
> > > 2) The guideline says to check after restarting every broker that the
> > > quorum is healthy, is there any metric to track that?
> > >
> > > Thanks,
> > > Sankalp
> > >
> > >
> > >
> > >
> >
>
Re: Upgrading existing non-TLS cluster with no downtime
Posted by Sankalp Bhatia <sa...@gmail.com>.
Thanks Mate for the responses. Brings a lot of clarity to me. I was able to
get it working this time without downtime. Don't know what I did wrong the
last time though.
On Mon, 20 Jul 2020 at 12:57, Szalay-Bekő Máté <sz...@gmail.com>
wrote:
> echo "stat" | nc localhost 2182Hi,
>
> I guess this is the part you are referring:
>
> https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster
> (your link was pointing to the 3.3.2 admin guide where this chapter was
> missing)
>
> > 1) When I set sslQuorum=true and portUnification=true on the first
> server,
> does it go out of the quorum? And when these properties are set in the
> second server, a new quorum of first and second server is formed and now
> the third server is out of quorum. When the 3rd server follows suit, it is
> added back to the quorum.
>
> the "sslQuorum=true and portUnification=true" setting is needed in step 4
> (although the numbering is bad in the markdown...). After step 3 you
> already have a 3 server quorum up with portUnification=true, meaning the
> cluster can handle both TLS/SSL and regular/non-secure connections. So when
> you restart server 1 with sslQuorum=true, then it will be able to re-join
> to the quorum, as server 2 and 3 are capable of handling SSL connections
> (even if they are not using it for connection initiation). So ideally
> between restarting each servers with sslQuorum=true, you always should have
> a 3 node full quorum.
>
> > 2) The guideline says to check after restarting every broker that the
> quorum is healthy, is there any metric to track that?
>
> I send the "stat" command to all nodes to see if everyone is connected to
> the quorum. E.g.: echo "stat" | nc localhost 2181
> I usually use 4-letter-word commands but the REST admin API works as well,
> and actually that is the officially recommended way, as the 4-letter-words
> are / will be deprecated some time.
> For the admin server see:
> https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#sc_adminserver
>
> Kind regards,
> Mate
>
> On Tue, Jul 14, 2020 at 10:52 PM Sankalp Bhatia <sankalpbhatia92@gmail.com
> >
> wrote:
>
> > +users
> >
> > On Tue, 14 Jul 2020 at 21:51, Sankalp Bhatia <sa...@gmail.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > I am trying to follow the section "Upgrading existing non-TLS cluster
> > with
> > > no downtime" in the zookeeper guide :
> > > https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
> > >
> > > I have an ensemble of 3 servers. I have a couple of questions:
> > >
> > > 1) When I set sslQuorum=true and portUnification=true on the first
> > > server, does it go out of the quorum? And when these properties are set
> > > in the second server, a new quorum of first and second server is formed
> > and
> > > now the third server is out of quorum. When the 3rd server follows
> suit,
> > it
> > > is added back to the quorum.
> > >
> > > If this is the case, what is the use of a the port-unification feature
> > > here?
> > >
> > > 2) The guideline says to check after restarting every broker that the
> > > quorum is healthy, is there any metric to track that?
> > >
> > > Thanks,
> > > Sankalp
> > >
> > >
> > >
> > >
> >
>
Re: Upgrading existing non-TLS cluster with no downtime
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
echo "stat" | nc localhost 2182Hi,
I guess this is the part you are referring:
https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster
(your link was pointing to the 3.3.2 admin guide where this chapter was
missing)
> 1) When I set sslQuorum=true and portUnification=true on the first
server,
does it go out of the quorum? And when these properties are set in the
second server, a new quorum of first and second server is formed and now
the third server is out of quorum. When the 3rd server follows suit, it is
added back to the quorum.
the "sslQuorum=true and portUnification=true" setting is needed in step 4
(although the numbering is bad in the markdown...). After step 3 you
already have a 3 server quorum up with portUnification=true, meaning the
cluster can handle both TLS/SSL and regular/non-secure connections. So when
you restart server 1 with sslQuorum=true, then it will be able to re-join
to the quorum, as server 2 and 3 are capable of handling SSL connections
(even if they are not using it for connection initiation). So ideally
between restarting each servers with sslQuorum=true, you always should have
a 3 node full quorum.
> 2) The guideline says to check after restarting every broker that the
quorum is healthy, is there any metric to track that?
I send the "stat" command to all nodes to see if everyone is connected to
the quorum. E.g.: echo "stat" | nc localhost 2181
I usually use 4-letter-word commands but the REST admin API works as well,
and actually that is the officially recommended way, as the 4-letter-words
are / will be deprecated some time.
For the admin server see:
https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#sc_adminserver
Kind regards,
Mate
On Tue, Jul 14, 2020 at 10:52 PM Sankalp Bhatia <sa...@gmail.com>
wrote:
> +users
>
> On Tue, 14 Jul 2020 at 21:51, Sankalp Bhatia <sa...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > I am trying to follow the section "Upgrading existing non-TLS cluster
> with
> > no downtime" in the zookeeper guide :
> > https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
> >
> > I have an ensemble of 3 servers. I have a couple of questions:
> >
> > 1) When I set sslQuorum=true and portUnification=true on the first
> > server, does it go out of the quorum? And when these properties are set
> > in the second server, a new quorum of first and second server is formed
> and
> > now the third server is out of quorum. When the 3rd server follows suit,
> it
> > is added back to the quorum.
> >
> > If this is the case, what is the use of a the port-unification feature
> > here?
> >
> > 2) The guideline says to check after restarting every broker that the
> > quorum is healthy, is there any metric to track that?
> >
> > Thanks,
> > Sankalp
> >
> >
> >
> >
>
Re: Upgrading existing non-TLS cluster with no downtime
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
echo "stat" | nc localhost 2182Hi,
I guess this is the part you are referring:
https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#Upgrading+existing+nonTLS+cluster
(your link was pointing to the 3.3.2 admin guide where this chapter was
missing)
> 1) When I set sslQuorum=true and portUnification=true on the first
server,
does it go out of the quorum? And when these properties are set in the
second server, a new quorum of first and second server is formed and now
the third server is out of quorum. When the 3rd server follows suit, it is
added back to the quorum.
the "sslQuorum=true and portUnification=true" setting is needed in step 4
(although the numbering is bad in the markdown...). After step 3 you
already have a 3 server quorum up with portUnification=true, meaning the
cluster can handle both TLS/SSL and regular/non-secure connections. So when
you restart server 1 with sslQuorum=true, then it will be able to re-join
to the quorum, as server 2 and 3 are capable of handling SSL connections
(even if they are not using it for connection initiation). So ideally
between restarting each servers with sslQuorum=true, you always should have
a 3 node full quorum.
> 2) The guideline says to check after restarting every broker that the
quorum is healthy, is there any metric to track that?
I send the "stat" command to all nodes to see if everyone is connected to
the quorum. E.g.: echo "stat" | nc localhost 2181
I usually use 4-letter-word commands but the REST admin API works as well,
and actually that is the officially recommended way, as the 4-letter-words
are / will be deprecated some time.
For the admin server see:
https://zookeeper.apache.org/doc/r3.5.8/zookeeperAdmin.html#sc_adminserver
Kind regards,
Mate
On Tue, Jul 14, 2020 at 10:52 PM Sankalp Bhatia <sa...@gmail.com>
wrote:
> +users
>
> On Tue, 14 Jul 2020 at 21:51, Sankalp Bhatia <sa...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > I am trying to follow the section "Upgrading existing non-TLS cluster
> with
> > no downtime" in the zookeeper guide :
> > https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
> >
> > I have an ensemble of 3 servers. I have a couple of questions:
> >
> > 1) When I set sslQuorum=true and portUnification=true on the first
> > server, does it go out of the quorum? And when these properties are set
> > in the second server, a new quorum of first and second server is formed
> and
> > now the third server is out of quorum. When the 3rd server follows suit,
> it
> > is added back to the quorum.
> >
> > If this is the case, what is the use of a the port-unification feature
> > here?
> >
> > 2) The guideline says to check after restarting every broker that the
> > quorum is healthy, is there any metric to track that?
> >
> > Thanks,
> > Sankalp
> >
> >
> >
> >
>
Re: Upgrading existing non-TLS cluster with no downtime
Posted by Sankalp Bhatia <sa...@gmail.com>.
+users
On Tue, 14 Jul 2020 at 21:51, Sankalp Bhatia <sa...@gmail.com>
wrote:
> Hi All,
>
> I am trying to follow the section "Upgrading existing non-TLS cluster with
> no downtime" in the zookeeper guide :
> https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
>
> I have an ensemble of 3 servers. I have a couple of questions:
>
> 1) When I set sslQuorum=true and portUnification=true on the first
> server, does it go out of the quorum? And when these properties are set
> in the second server, a new quorum of first and second server is formed and
> now the third server is out of quorum. When the 3rd server follows suit, it
> is added back to the quorum.
>
> If this is the case, what is the use of a the port-unification feature
> here?
>
> 2) The guideline says to check after restarting every broker that the
> quorum is healthy, is there any metric to track that?
>
> Thanks,
> Sankalp
>
>
>
>
Re: Upgrading existing non-TLS cluster with no downtime
Posted by Sankalp Bhatia <sa...@gmail.com>.
+users
On Tue, 14 Jul 2020 at 21:51, Sankalp Bhatia <sa...@gmail.com>
wrote:
> Hi All,
>
> I am trying to follow the section "Upgrading existing non-TLS cluster with
> no downtime" in the zookeeper guide :
> https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html
>
> I have an ensemble of 3 servers. I have a couple of questions:
>
> 1) When I set sslQuorum=true and portUnification=true on the first
> server, does it go out of the quorum? And when these properties are set
> in the second server, a new quorum of first and second server is formed and
> now the third server is out of quorum. When the 3rd server follows suit, it
> is added back to the quorum.
>
> If this is the case, what is the use of a the port-unification feature
> here?
>
> 2) The guideline says to check after restarting every broker that the
> quorum is healthy, is there any metric to track that?
>
> Thanks,
> Sankalp
>
>
>
>