You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by LEE Ween Jiann <wj...@phdcs.smu.edu.sg> on 2019/09/27 14:33:30 UTC
Zookeeper client with single address pointing to multiple servers
Hi,
From the Zookeeper constructore in JAVA API:
“To create a ZooKeeper client object, the application needs to pass a connection string containing a comma separated list of host:port pairs, each corresponding to a ZooKeeper server.”
I see that zookeeper resolves all the IPs from an address and randomly picks one. Why would multiple addresses, one for each server be needed? Why couldn’t zk client resolve all the servers from a single address?
I’m asking this for helm deployment on Kubernetes as zookeeper is deployed with a single headless service that points to multiple server.
RE: Zookeeper client with single address pointing to multiple servers
Posted by LEE Ween Jiann <wj...@phdcs.smu.edu.sg>.
SMU Classification: Restricted
Alright. Thanks
-----Original Message-----
From: Cee Tee <c....@gmail.com>
Sent: Saturday, September 28, 2019 3:24 PM
To: user@zookeeper.apache.org
Subject: Re: Zookeeper client with single address pointing to multiple servers
You would have to grab the config file from the filesystem, or have some process push it into a zookeeper node.
Re: Zookeeper client with single address pointing to multiple servers
Posted by Cee Tee <c....@gmail.com>.
You would have to grab the config file from the filesystem, or have some
process push it into a zookeeper node.
Re: Zookeeper client with single address pointing to multiple servers
Posted by LEE Ween Jiann <wj...@phdcs.smu.edu.sg>.
Hi Cee Tee,
I've check and this is working for 3.5.x but 3.4.x does not have the node.
Is there another way for 3.4.x?
On 28/9/19, 1:09 AM, "Cee Tee" <c....@gmail.com> wrote:
The latest config is available in node /zookeeper/config or via the config
command in the cli. If you want to know what servers are actually taking
requests you will have to go over them one by one issuing admin api server
stats calls.
On 27 September 2019 19:04:35 Shawn Heisey <ap...@elyograg.org> wrote:
> On 9/27/2019 9:24 AM, Benjamin Reed wrote:
>> are you making the assumption that you have a single machine that will
>> always be up? that is not a common assumption these days, which is why
>> solr might be resistant to such a change.
>>
>> you can have a single DNS name resolve to multiple IP addresses and
>> ZooKeeper client will use all those addresses if you don't like
>> specifying a list on all the clients.
>
> Is there something in the ZK client API that will allow Solr to ask the
> ZK client for a list of active servers that it is connected to?
>
> Currently Solr just parses the zkHost string to obtain a server list for
> the "ZK status" portion of our admin UI. This code was written when we
> used ZK 3.4.x ... but because we're now using 3.5.x which has dynamic
> reconfiguration, the list of active servers can be different than the
> zkHost string used when Solr started.
>
> Thanks,
> Shawn
Re: Zookeeper client with single address pointing to multiple servers
Posted by Cee Tee <c....@gmail.com>.
The latest config is available in node /zookeeper/config or via the config
command in the cli. If you want to know what servers are actually taking
requests you will have to go over them one by one issuing admin api server
stats calls.
On 27 September 2019 19:04:35 Shawn Heisey <ap...@elyograg.org> wrote:
> On 9/27/2019 9:24 AM, Benjamin Reed wrote:
>> are you making the assumption that you have a single machine that will
>> always be up? that is not a common assumption these days, which is why
>> solr might be resistant to such a change.
>>
>> you can have a single DNS name resolve to multiple IP addresses and
>> ZooKeeper client will use all those addresses if you don't like
>> specifying a list on all the clients.
>
> Is there something in the ZK client API that will allow Solr to ask the
> ZK client for a list of active servers that it is connected to?
>
> Currently Solr just parses the zkHost string to obtain a server list for
> the "ZK status" portion of our admin UI. This code was written when we
> used ZK 3.4.x ... but because we're now using 3.5.x which has dynamic
> reconfiguration, the list of active servers can be different than the
> zkHost string used when Solr started.
>
> Thanks,
> Shawn
Re: Zookeeper client with single address pointing to multiple servers
Posted by Shawn Heisey <ap...@elyograg.org>.
On 9/27/2019 9:24 AM, Benjamin Reed wrote:
> are you making the assumption that you have a single machine that will
> always be up? that is not a common assumption these days, which is why
> solr might be resistant to such a change.
>
> you can have a single DNS name resolve to multiple IP addresses and
> ZooKeeper client will use all those addresses if you don't like
> specifying a list on all the clients.
Is there something in the ZK client API that will allow Solr to ask the
ZK client for a list of active servers that it is connected to?
Currently Solr just parses the zkHost string to obtain a server list for
the "ZK status" portion of our admin UI. This code was written when we
used ZK 3.4.x ... but because we're now using 3.5.x which has dynamic
reconfiguration, the list of active servers can be different than the
zkHost string used when Solr started.
Thanks,
Shawn
Re: Zookeeper client with single address pointing to multiple servers
Posted by LEE Ween Jiann <wj...@phdcs.smu.edu.sg>.
Thanks Ben. I'll continue to use the headless service which resolves to multiple IP.
On 27/9/19, 11:24 PM, "Benjamin Reed" <br...@apache.org> wrote:
are you making the assumption that you have a single machine that will
always be up? that is not a common assumption these days, which is why
solr might be resistant to such a change.
you can have a single DNS name resolve to multiple IP addresses and
ZooKeeper client will use all those addresses if you don't like
specifying a list on all the clients.
ben
On Fri, Sep 27, 2019 at 8:19 AM LEE Ween Jiann
<wj...@phdcs.smu.edu.sg> wrote:
>
> Thanks, I'm not sure whether Solr would make those change. I will ask them.
>
> Any reason for this design?
>
> On 27/9/19, 10:43 PM, "Cee Tee" <c....@gmail.com> wrote:
>
> You could do that as follows:
>
> 1 Connect to a single always online entrypoint zookeeper of the zookeeper
> cluster.
> 2 get Data the config node at /zookeeper/config
> 3 parse it into a multinode connect string and reconnect using that string.
>
>
> On 27 September 2019 16:33:48 LEE Ween Jiann <wj...@phdcs.smu.edu.sg>
> wrote:
>
> > Hi,
> >
> > From the Zookeeper constructore in JAVA API:
> > “To create a ZooKeeper client object, the application needs to pass a
> > connection string containing a comma separated list of host:port pairs,
> > each corresponding to a ZooKeeper server.”
> >
> > I see that zookeeper resolves all the IPs from an address and randomly
> > picks one. Why would multiple addresses, one for each server be needed? Why
> > couldn’t zk client resolve all the servers from a single address?
> >
> > I’m asking this for helm deployment on Kubernetes as zookeeper is deployed
> > with a single headless service that points to multiple server.
>
>
>
>
>
Re: Zookeeper client with single address pointing to multiple servers
Posted by Benjamin Reed <br...@apache.org>.
are you making the assumption that you have a single machine that will
always be up? that is not a common assumption these days, which is why
solr might be resistant to such a change.
you can have a single DNS name resolve to multiple IP addresses and
ZooKeeper client will use all those addresses if you don't like
specifying a list on all the clients.
ben
On Fri, Sep 27, 2019 at 8:19 AM LEE Ween Jiann
<wj...@phdcs.smu.edu.sg> wrote:
>
> Thanks, I'm not sure whether Solr would make those change. I will ask them.
>
> Any reason for this design?
>
> On 27/9/19, 10:43 PM, "Cee Tee" <c....@gmail.com> wrote:
>
> You could do that as follows:
>
> 1 Connect to a single always online entrypoint zookeeper of the zookeeper
> cluster.
> 2 get Data the config node at /zookeeper/config
> 3 parse it into a multinode connect string and reconnect using that string.
>
>
> On 27 September 2019 16:33:48 LEE Ween Jiann <wj...@phdcs.smu.edu.sg>
> wrote:
>
> > Hi,
> >
> > From the Zookeeper constructore in JAVA API:
> > “To create a ZooKeeper client object, the application needs to pass a
> > connection string containing a comma separated list of host:port pairs,
> > each corresponding to a ZooKeeper server.”
> >
> > I see that zookeeper resolves all the IPs from an address and randomly
> > picks one. Why would multiple addresses, one for each server be needed? Why
> > couldn’t zk client resolve all the servers from a single address?
> >
> > I’m asking this for helm deployment on Kubernetes as zookeeper is deployed
> > with a single headless service that points to multiple server.
>
>
>
>
>
Re: Zookeeper client with single address pointing to multiple servers
Posted by LEE Ween Jiann <wj...@phdcs.smu.edu.sg>.
Thanks, I'm not sure whether Solr would make those change. I will ask them.
Any reason for this design?
On 27/9/19, 10:43 PM, "Cee Tee" <c....@gmail.com> wrote:
You could do that as follows:
1 Connect to a single always online entrypoint zookeeper of the zookeeper
cluster.
2 get Data the config node at /zookeeper/config
3 parse it into a multinode connect string and reconnect using that string.
On 27 September 2019 16:33:48 LEE Ween Jiann <wj...@phdcs.smu.edu.sg>
wrote:
> Hi,
>
> From the Zookeeper constructore in JAVA API:
> “To create a ZooKeeper client object, the application needs to pass a
> connection string containing a comma separated list of host:port pairs,
> each corresponding to a ZooKeeper server.”
>
> I see that zookeeper resolves all the IPs from an address and randomly
> picks one. Why would multiple addresses, one for each server be needed? Why
> couldn’t zk client resolve all the servers from a single address?
>
> I’m asking this for helm deployment on Kubernetes as zookeeper is deployed
> with a single headless service that points to multiple server.
Re: Zookeeper client with single address pointing to multiple servers
Posted by Cee Tee <c....@gmail.com>.
You could do that as follows:
1 Connect to a single always online entrypoint zookeeper of the zookeeper
cluster.
2 get Data the config node at /zookeeper/config
3 parse it into a multinode connect string and reconnect using that string.
On 27 September 2019 16:33:48 LEE Ween Jiann <wj...@phdcs.smu.edu.sg>
wrote:
> Hi,
>
> From the Zookeeper constructore in JAVA API:
> “To create a ZooKeeper client object, the application needs to pass a
> connection string containing a comma separated list of host:port pairs,
> each corresponding to a ZooKeeper server.”
>
> I see that zookeeper resolves all the IPs from an address and randomly
> picks one. Why would multiple addresses, one for each server be needed? Why
> couldn’t zk client resolve all the servers from a single address?
>
> I’m asking this for helm deployment on Kubernetes as zookeeper is deployed
> with a single headless service that points to multiple server.