You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by Szalay-Bekő Máté <sz...@gmail.com> on 2020/02/18 09:32:20 UTC
using 0.0.0.0 in the server configs in ZK 3.5+
Hi All,
in a recent PR I try to fix an issue we found with Suhas Dantkale in
ZOOKEEPER-2146 (see https://github.com/apache/zookeeper/pull/1254). The
problem is that in ZooKeeper 3.5+ some quorum members can not rejoin to the
quorum after a restart if the server configs are set like this:
zoo.cfg in server 1:
server.1=0.0.0.0:2888:3888
server.2=some.fqdn-2.com:2888:3888
server.3=some.fqdn-3.com:2888:3888
zoo.cfg in server 2:
server.1=some.fqdn-1.com:2888:3888
server.2=0.0.0.0:2888:3888
server.3=some.fqdn-3.com:2888:3888
I am not exactly sure about the use case behind this config, but people
claim they need it for specific dockerized environments (see the comments
in the jira ticket). Is anyone familiar with such use cases? We never used
such configs in production as far as I can tell.
The above config worked without a problem in ZooKeeper 3.4.x, but not
perfectly for 3.5.x. It would be logical to keep supporting it. Still, I
think after the introduction of the dynamic reconfig, we kind of assume
that all the servers have the same server address configurations. So maybe
the config is not even valid anymore?
Using the 'quorumListenOnAllIPs' config property instead the 0.0.0.0 in the
configs might solve the issue. But if it is the case, then we definitely
should highlight this in the wiki / documentation. Maybe even printing out
a warning during ZooKeeper startup.
What do you think?
Kind regards,
Mate
Re: using 0.0.0.0 in the server configs in ZK 3.5+
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
we got an other question in the user list just about this, and also there
is a duplicate jira ticket opened 6 days ago:
https://issues.apache.org/jira/browse/ZOOKEEPER-3725
On Wed, Feb 19, 2020 at 10:04 AM Szalay-Bekő Máté <
szalay.beko.mate@gmail.com> wrote:
> Thanks for all the comments!
>
> My gut feeling is that the quorumListenOnAllIPs should work usually, e.g.
> even if there is proxy / tunneling between the servers.
>
> But I never operated ZooKeeper in a complex dockerized setup. Maybe you
> don't want to bind on all IPs locally for security reasons (having separate
> virtual networks client traffic and for quorum-internal traffic). For more
> complex cases I like the distinction of "local bind address" and
> "advertised address" proposed by Enrico. That gives more flexibility, while
> still keeps the single view of the server address configuration what can be
> distributed in the whole cluster with dynamic reconfig.
>
> If no one propose otherwise, then:
> - I will start a thread on the user mail list to ask around if it is a
> problem for many people and also to see if 'quorumListenOnAllIPs' is enough
> for them, or we should prioritize a more generic approach.
> - I think someone should ask the guys in the docker community to change
> the example configuration here, https://hub.docker.com/_/zookeeper not
> using 0.0.0.0, but include description of quorumListenOnAllIPs. Do we have
> any contact to them?
> - I still think for backward-compatibility reason it make sense to go
> ahead with the PR (https://github.com/apache/zookeeper/pull/1254) as we
> should support a working 3.4 config also in 3.5+, especially if the fix is
> this trivial
>
> What do you think?
>
> Mate
>
> On Wed, Feb 19, 2020 at 6:30 AM Andor Molnar <an...@apache.org> wrote:
>
>> What does '0.0.0.0:2888:3888' mean exactly in terms of bindind? Listen
>> on all available interfaces?
>>
>> In which case I think it would be better to advise using
>> quorumListenOnAllIPs to customers instead. We could also detect and
>> alert if configured with wildcard address.
>>
>> Andor
>>
>>
>>
>> On Tue, 2020-02-18 at 18:00 +0100, Enrico Olivelli wrote:
>> > I feel it would be better to simply have a "local bind address" and
>> > an
>> > "advertised address"
>> >
>> > server.1=advertisedaddress:port:port....
>> > localBindAddress=0.0.0.0
>> >
>> > it is clearer
>> > Enrico
>> >
>> > Il giorno mar 18 feb 2020 alle ore 11:29 Szalay-Bekő Máté
>> > <sz...@gmail.com> ha scritto:
>> > > also, we have the same 0.0.0.0 config suggested in the description
>> > > of the
>> > > zookeeper 'docker official' image:
>> > > https://hub.docker.com/_/zookeeper
>> > >
>> > > On Tue, Feb 18, 2020 at 10:59 AM Szalay-Bekő Máté <
>> > > szalay.beko.mate@gmail.com> wrote:
>> > >
>> > > > A quote from the Jira ticket where Sebastian tries to explain
>> > > > their setup:
>> > > > (see
>> > > >
>> https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941
>> > > > )
>> > > >
>> > > > "We are using 0.0.0.0 in our config of the 3-node-cluster as the
>> > > > nodes are
>> > > > running in separate docker-instances not directly connected to
>> > > > each other.
>> > > > Which means the zookeeper-container doesn't have its real IP
>> > > > assigned only
>> > > > using port-forwardings from the docker host to the container to
>> > > > make it
>> > > > accessible and so using the external IP or the FQDN doesn't allow
>> > > > zookeeper
>> > > > to start as it complains about not finding that IP in the
>> > > > container. Which
>> > > > is of course correct. Using the internal Network-IP instead of
>> > > > 0.0.0.0
>> > > > would result in the same problem as using 0.0.0.0 as all internal
>> > > > networks
>> > > > of the three docker hosts are using the same network-range in
>> > > > their
>> > > > internal networks."
>> > > >
>> > > > re-reading it, I think he meant a bit different config than what
>> > > > I wrote
>> > > > as an example before... I think he basically connects separate
>> > > > hosts with
>> > > > tunnels maybe and running ZK in docker in each host without any
>> > > > container
>> > > > orchestration tool. (?)
>> > > >
>> > > > On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli <
>> > > > eolivelli@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Honestly I think that using 0.0.0.0 is not so useful.
>> > > > > I find it very useful that the configuration is the same on
>> > > > > every peer,
>> > > > > except from myid file
>> > > > > What's the real gain in such configuration?
>> > > > >
>> > > > >
>> > > > > Enrico
>> > > > >
>> > > > >
>> > > > > Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté <
>> > > > > szalay.beko.mate@gmail.com>
>> > > > > ha
>> > > > > scritto:
>> > > > >
>> > > > > > Hi All,
>> > > > > >
>> > > > > > in a recent PR I try to fix an issue we found with Suhas
>> > > > > > Dantkale in
>> > > > > > ZOOKEEPER-2146 (see
>> > > > > > https://github.com/apache/zookeeper/pull/1254). The
>> > > > > > problem is that in ZooKeeper 3.5+ some quorum members can not
>> > > > > > rejoin to
>> > > > > the
>> > > > > > quorum after a restart if the server configs are set like
>> > > > > > this:
>> > > > > >
>> > > > > > zoo.cfg in server 1:
>> > > > > > server.1=0.0.0.0:2888:3888
>> > > > > > server.2=some.fqdn-2.com:2888:3888
>> > > > > > server.3=some.fqdn-3.com:2888:3888
>> > > > > >
>> > > > > > zoo.cfg in server 2:
>> > > > > > server.1=some.fqdn-1.com:2888:3888
>> > > > > > server.2=0.0.0.0:2888:3888
>> > > > > > server.3=some.fqdn-3.com:2888:3888
>> > > > > >
>> > > > > > I am not exactly sure about the use case behind this config,
>> > > > > > but people
>> > > > > > claim they need it for specific dockerized environments (see
>> > > > > > the
>> > > > > comments
>> > > > > > in the jira ticket). Is anyone familiar with such use cases?
>> > > > > > We never
>> > > > > used
>> > > > > > such configs in production as far as I can tell.
>> > > > > >
>> > > > > > The above config worked without a problem in ZooKeeper 3.4.x,
>> > > > > > but not
>> > > > > > perfectly for 3.5.x. It would be logical to keep supporting
>> > > > > > it. Still, I
>> > > > > > think after the introduction of the dynamic reconfig, we kind
>> > > > > > of assume
>> > > > > > that all the servers have the same server address
>> > > > > > configurations. So
>> > > > > maybe
>> > > > > > the config is not even valid anymore?
>> > > > > >
>> > > > > > Using the 'quorumListenOnAllIPs' config property instead the
>> > > > > > 0.0.0.0 in
>> > > > > the
>> > > > > > configs might solve the issue. But if it is the case, then we
>> > > > > > definitely
>> > > > > > should highlight this in the wiki / documentation. Maybe even
>> > > > > > printing
>> > > > > out
>> > > > > > a warning during ZooKeeper startup.
>> > > > > >
>> > > > > > What do you think?
>> > > > > >
>> > > > > > Kind regards,
>> > > > > > Mate
>> > > > > >
>>
>>
Re: using 0.0.0.0 in the server configs in ZK 3.5+
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
Thanks for all the comments!
My gut feeling is that the quorumListenOnAllIPs should work usually, e.g.
even if there is proxy / tunneling between the servers.
But I never operated ZooKeeper in a complex dockerized setup. Maybe you
don't want to bind on all IPs locally for security reasons (having separate
virtual networks client traffic and for quorum-internal traffic). For more
complex cases I like the distinction of "local bind address" and
"advertised address" proposed by Enrico. That gives more flexibility, while
still keeps the single view of the server address configuration what can be
distributed in the whole cluster with dynamic reconfig.
If no one propose otherwise, then:
- I will start a thread on the user mail list to ask around if it is a
problem for many people and also to see if 'quorumListenOnAllIPs' is enough
for them, or we should prioritize a more generic approach.
- I think someone should ask the guys in the docker community to change the
example configuration here, https://hub.docker.com/_/zookeeper not using
0.0.0.0, but include description of quorumListenOnAllIPs. Do we have any
contact to them?
- I still think for backward-compatibility reason it make sense to go ahead
with the PR (https://github.com/apache/zookeeper/pull/1254) as we should
support a working 3.4 config also in 3.5+, especially if the fix is this
trivial
What do you think?
Mate
On Wed, Feb 19, 2020 at 6:30 AM Andor Molnar <an...@apache.org> wrote:
> What does '0.0.0.0:2888:3888' mean exactly in terms of bindind? Listen
> on all available interfaces?
>
> In which case I think it would be better to advise using
> quorumListenOnAllIPs to customers instead. We could also detect and
> alert if configured with wildcard address.
>
> Andor
>
>
>
> On Tue, 2020-02-18 at 18:00 +0100, Enrico Olivelli wrote:
> > I feel it would be better to simply have a "local bind address" and
> > an
> > "advertised address"
> >
> > server.1=advertisedaddress:port:port....
> > localBindAddress=0.0.0.0
> >
> > it is clearer
> > Enrico
> >
> > Il giorno mar 18 feb 2020 alle ore 11:29 Szalay-Bekő Máté
> > <sz...@gmail.com> ha scritto:
> > > also, we have the same 0.0.0.0 config suggested in the description
> > > of the
> > > zookeeper 'docker official' image:
> > > https://hub.docker.com/_/zookeeper
> > >
> > > On Tue, Feb 18, 2020 at 10:59 AM Szalay-Bekő Máté <
> > > szalay.beko.mate@gmail.com> wrote:
> > >
> > > > A quote from the Jira ticket where Sebastian tries to explain
> > > > their setup:
> > > > (see
> > > >
> https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941
> > > > )
> > > >
> > > > "We are using 0.0.0.0 in our config of the 3-node-cluster as the
> > > > nodes are
> > > > running in separate docker-instances not directly connected to
> > > > each other.
> > > > Which means the zookeeper-container doesn't have its real IP
> > > > assigned only
> > > > using port-forwardings from the docker host to the container to
> > > > make it
> > > > accessible and so using the external IP or the FQDN doesn't allow
> > > > zookeeper
> > > > to start as it complains about not finding that IP in the
> > > > container. Which
> > > > is of course correct. Using the internal Network-IP instead of
> > > > 0.0.0.0
> > > > would result in the same problem as using 0.0.0.0 as all internal
> > > > networks
> > > > of the three docker hosts are using the same network-range in
> > > > their
> > > > internal networks."
> > > >
> > > > re-reading it, I think he meant a bit different config than what
> > > > I wrote
> > > > as an example before... I think he basically connects separate
> > > > hosts with
> > > > tunnels maybe and running ZK in docker in each host without any
> > > > container
> > > > orchestration tool. (?)
> > > >
> > > > On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli <
> > > > eolivelli@gmail.com>
> > > > wrote:
> > > >
> > > > > Honestly I think that using 0.0.0.0 is not so useful.
> > > > > I find it very useful that the configuration is the same on
> > > > > every peer,
> > > > > except from myid file
> > > > > What's the real gain in such configuration?
> > > > >
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > > Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté <
> > > > > szalay.beko.mate@gmail.com>
> > > > > ha
> > > > > scritto:
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > in a recent PR I try to fix an issue we found with Suhas
> > > > > > Dantkale in
> > > > > > ZOOKEEPER-2146 (see
> > > > > > https://github.com/apache/zookeeper/pull/1254). The
> > > > > > problem is that in ZooKeeper 3.5+ some quorum members can not
> > > > > > rejoin to
> > > > > the
> > > > > > quorum after a restart if the server configs are set like
> > > > > > this:
> > > > > >
> > > > > > zoo.cfg in server 1:
> > > > > > server.1=0.0.0.0:2888:3888
> > > > > > server.2=some.fqdn-2.com:2888:3888
> > > > > > server.3=some.fqdn-3.com:2888:3888
> > > > > >
> > > > > > zoo.cfg in server 2:
> > > > > > server.1=some.fqdn-1.com:2888:3888
> > > > > > server.2=0.0.0.0:2888:3888
> > > > > > server.3=some.fqdn-3.com:2888:3888
> > > > > >
> > > > > > I am not exactly sure about the use case behind this config,
> > > > > > but people
> > > > > > claim they need it for specific dockerized environments (see
> > > > > > the
> > > > > comments
> > > > > > in the jira ticket). Is anyone familiar with such use cases?
> > > > > > We never
> > > > > used
> > > > > > such configs in production as far as I can tell.
> > > > > >
> > > > > > The above config worked without a problem in ZooKeeper 3.4.x,
> > > > > > but not
> > > > > > perfectly for 3.5.x. It would be logical to keep supporting
> > > > > > it. Still, I
> > > > > > think after the introduction of the dynamic reconfig, we kind
> > > > > > of assume
> > > > > > that all the servers have the same server address
> > > > > > configurations. So
> > > > > maybe
> > > > > > the config is not even valid anymore?
> > > > > >
> > > > > > Using the 'quorumListenOnAllIPs' config property instead the
> > > > > > 0.0.0.0 in
> > > > > the
> > > > > > configs might solve the issue. But if it is the case, then we
> > > > > > definitely
> > > > > > should highlight this in the wiki / documentation. Maybe even
> > > > > > printing
> > > > > out
> > > > > > a warning during ZooKeeper startup.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Kind regards,
> > > > > > Mate
> > > > > >
>
>
Re: using 0.0.0.0 in the server configs in ZK 3.5+
Posted by Andor Molnar <an...@apache.org>.
What does '0.0.0.0:2888:3888' mean exactly in terms of bindind? Listen
on all available interfaces?
In which case I think it would be better to advise using
quorumListenOnAllIPs to customers instead. We could also detect and
alert if configured with wildcard address.
Andor
On Tue, 2020-02-18 at 18:00 +0100, Enrico Olivelli wrote:
> I feel it would be better to simply have a "local bind address" and
> an
> "advertised address"
>
> server.1=advertisedaddress:port:port....
> localBindAddress=0.0.0.0
>
> it is clearer
> Enrico
>
> Il giorno mar 18 feb 2020 alle ore 11:29 Szalay-Bekő Máté
> <sz...@gmail.com> ha scritto:
> > also, we have the same 0.0.0.0 config suggested in the description
> > of the
> > zookeeper 'docker official' image:
> > https://hub.docker.com/_/zookeeper
> >
> > On Tue, Feb 18, 2020 at 10:59 AM Szalay-Bekő Máté <
> > szalay.beko.mate@gmail.com> wrote:
> >
> > > A quote from the Jira ticket where Sebastian tries to explain
> > > their setup:
> > > (see
> > > https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941
> > > )
> > >
> > > "We are using 0.0.0.0 in our config of the 3-node-cluster as the
> > > nodes are
> > > running in separate docker-instances not directly connected to
> > > each other.
> > > Which means the zookeeper-container doesn't have its real IP
> > > assigned only
> > > using port-forwardings from the docker host to the container to
> > > make it
> > > accessible and so using the external IP or the FQDN doesn't allow
> > > zookeeper
> > > to start as it complains about not finding that IP in the
> > > container. Which
> > > is of course correct. Using the internal Network-IP instead of
> > > 0.0.0.0
> > > would result in the same problem as using 0.0.0.0 as all internal
> > > networks
> > > of the three docker hosts are using the same network-range in
> > > their
> > > internal networks."
> > >
> > > re-reading it, I think he meant a bit different config than what
> > > I wrote
> > > as an example before... I think he basically connects separate
> > > hosts with
> > > tunnels maybe and running ZK in docker in each host without any
> > > container
> > > orchestration tool. (?)
> > >
> > > On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli <
> > > eolivelli@gmail.com>
> > > wrote:
> > >
> > > > Honestly I think that using 0.0.0.0 is not so useful.
> > > > I find it very useful that the configuration is the same on
> > > > every peer,
> > > > except from myid file
> > > > What's the real gain in such configuration?
> > > >
> > > >
> > > > Enrico
> > > >
> > > >
> > > > Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté <
> > > > szalay.beko.mate@gmail.com>
> > > > ha
> > > > scritto:
> > > >
> > > > > Hi All,
> > > > >
> > > > > in a recent PR I try to fix an issue we found with Suhas
> > > > > Dantkale in
> > > > > ZOOKEEPER-2146 (see
> > > > > https://github.com/apache/zookeeper/pull/1254). The
> > > > > problem is that in ZooKeeper 3.5+ some quorum members can not
> > > > > rejoin to
> > > > the
> > > > > quorum after a restart if the server configs are set like
> > > > > this:
> > > > >
> > > > > zoo.cfg in server 1:
> > > > > server.1=0.0.0.0:2888:3888
> > > > > server.2=some.fqdn-2.com:2888:3888
> > > > > server.3=some.fqdn-3.com:2888:3888
> > > > >
> > > > > zoo.cfg in server 2:
> > > > > server.1=some.fqdn-1.com:2888:3888
> > > > > server.2=0.0.0.0:2888:3888
> > > > > server.3=some.fqdn-3.com:2888:3888
> > > > >
> > > > > I am not exactly sure about the use case behind this config,
> > > > > but people
> > > > > claim they need it for specific dockerized environments (see
> > > > > the
> > > > comments
> > > > > in the jira ticket). Is anyone familiar with such use cases?
> > > > > We never
> > > > used
> > > > > such configs in production as far as I can tell.
> > > > >
> > > > > The above config worked without a problem in ZooKeeper 3.4.x,
> > > > > but not
> > > > > perfectly for 3.5.x. It would be logical to keep supporting
> > > > > it. Still, I
> > > > > think after the introduction of the dynamic reconfig, we kind
> > > > > of assume
> > > > > that all the servers have the same server address
> > > > > configurations. So
> > > > maybe
> > > > > the config is not even valid anymore?
> > > > >
> > > > > Using the 'quorumListenOnAllIPs' config property instead the
> > > > > 0.0.0.0 in
> > > > the
> > > > > configs might solve the issue. But if it is the case, then we
> > > > > definitely
> > > > > should highlight this in the wiki / documentation. Maybe even
> > > > > printing
> > > > out
> > > > > a warning during ZooKeeper startup.
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Kind regards,
> > > > > Mate
> > > > >
Re: using 0.0.0.0 in the server configs in ZK 3.5+
Posted by Enrico Olivelli <eo...@gmail.com>.
I feel it would be better to simply have a "local bind address" and an
"advertised address"
server.1=advertisedaddress:port:port....
localBindAddress=0.0.0.0
it is clearer
Enrico
Il giorno mar 18 feb 2020 alle ore 11:29 Szalay-Bekő Máté
<sz...@gmail.com> ha scritto:
>
> also, we have the same 0.0.0.0 config suggested in the description of the
> zookeeper 'docker official' image: https://hub.docker.com/_/zookeeper
>
> On Tue, Feb 18, 2020 at 10:59 AM Szalay-Bekő Máté <
> szalay.beko.mate@gmail.com> wrote:
>
> > A quote from the Jira ticket where Sebastian tries to explain their setup:
> > (see
> > https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941
> > )
> >
> > "We are using 0.0.0.0 in our config of the 3-node-cluster as the nodes are
> > running in separate docker-instances not directly connected to each other.
> > Which means the zookeeper-container doesn't have its real IP assigned only
> > using port-forwardings from the docker host to the container to make it
> > accessible and so using the external IP or the FQDN doesn't allow zookeeper
> > to start as it complains about not finding that IP in the container. Which
> > is of course correct. Using the internal Network-IP instead of 0.0.0.0
> > would result in the same problem as using 0.0.0.0 as all internal networks
> > of the three docker hosts are using the same network-range in their
> > internal networks."
> >
> > re-reading it, I think he meant a bit different config than what I wrote
> > as an example before... I think he basically connects separate hosts with
> > tunnels maybe and running ZK in docker in each host without any container
> > orchestration tool. (?)
> >
> > On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> >> Honestly I think that using 0.0.0.0 is not so useful.
> >> I find it very useful that the configuration is the same on every peer,
> >> except from myid file
> >> What's the real gain in such configuration?
> >>
> >>
> >> Enrico
> >>
> >>
> >> Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté <sz...@gmail.com>
> >> ha
> >> scritto:
> >>
> >> > Hi All,
> >> >
> >> > in a recent PR I try to fix an issue we found with Suhas Dantkale in
> >> > ZOOKEEPER-2146 (see https://github.com/apache/zookeeper/pull/1254). The
> >> > problem is that in ZooKeeper 3.5+ some quorum members can not rejoin to
> >> the
> >> > quorum after a restart if the server configs are set like this:
> >> >
> >> > zoo.cfg in server 1:
> >> > server.1=0.0.0.0:2888:3888
> >> > server.2=some.fqdn-2.com:2888:3888
> >> > server.3=some.fqdn-3.com:2888:3888
> >> >
> >> > zoo.cfg in server 2:
> >> > server.1=some.fqdn-1.com:2888:3888
> >> > server.2=0.0.0.0:2888:3888
> >> > server.3=some.fqdn-3.com:2888:3888
> >> >
> >> > I am not exactly sure about the use case behind this config, but people
> >> > claim they need it for specific dockerized environments (see the
> >> comments
> >> > in the jira ticket). Is anyone familiar with such use cases? We never
> >> used
> >> > such configs in production as far as I can tell.
> >> >
> >> > The above config worked without a problem in ZooKeeper 3.4.x, but not
> >> > perfectly for 3.5.x. It would be logical to keep supporting it. Still, I
> >> > think after the introduction of the dynamic reconfig, we kind of assume
> >> > that all the servers have the same server address configurations. So
> >> maybe
> >> > the config is not even valid anymore?
> >> >
> >> > Using the 'quorumListenOnAllIPs' config property instead the 0.0.0.0 in
> >> the
> >> > configs might solve the issue. But if it is the case, then we definitely
> >> > should highlight this in the wiki / documentation. Maybe even printing
> >> out
> >> > a warning during ZooKeeper startup.
> >> >
> >> > What do you think?
> >> >
> >> > Kind regards,
> >> > Mate
> >> >
> >>
> >
Re: using 0.0.0.0 in the server configs in ZK 3.5+
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
also, we have the same 0.0.0.0 config suggested in the description of the
zookeeper 'docker official' image: https://hub.docker.com/_/zookeeper
On Tue, Feb 18, 2020 at 10:59 AM Szalay-Bekő Máté <
szalay.beko.mate@gmail.com> wrote:
> A quote from the Jira ticket where Sebastian tries to explain their setup:
> (see
> https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941
> )
>
> "We are using 0.0.0.0 in our config of the 3-node-cluster as the nodes are
> running in separate docker-instances not directly connected to each other.
> Which means the zookeeper-container doesn't have its real IP assigned only
> using port-forwardings from the docker host to the container to make it
> accessible and so using the external IP or the FQDN doesn't allow zookeeper
> to start as it complains about not finding that IP in the container. Which
> is of course correct. Using the internal Network-IP instead of 0.0.0.0
> would result in the same problem as using 0.0.0.0 as all internal networks
> of the three docker hosts are using the same network-range in their
> internal networks."
>
> re-reading it, I think he meant a bit different config than what I wrote
> as an example before... I think he basically connects separate hosts with
> tunnels maybe and running ZK in docker in each host without any container
> orchestration tool. (?)
>
> On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
>> Honestly I think that using 0.0.0.0 is not so useful.
>> I find it very useful that the configuration is the same on every peer,
>> except from myid file
>> What's the real gain in such configuration?
>>
>>
>> Enrico
>>
>>
>> Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté <sz...@gmail.com>
>> ha
>> scritto:
>>
>> > Hi All,
>> >
>> > in a recent PR I try to fix an issue we found with Suhas Dantkale in
>> > ZOOKEEPER-2146 (see https://github.com/apache/zookeeper/pull/1254). The
>> > problem is that in ZooKeeper 3.5+ some quorum members can not rejoin to
>> the
>> > quorum after a restart if the server configs are set like this:
>> >
>> > zoo.cfg in server 1:
>> > server.1=0.0.0.0:2888:3888
>> > server.2=some.fqdn-2.com:2888:3888
>> > server.3=some.fqdn-3.com:2888:3888
>> >
>> > zoo.cfg in server 2:
>> > server.1=some.fqdn-1.com:2888:3888
>> > server.2=0.0.0.0:2888:3888
>> > server.3=some.fqdn-3.com:2888:3888
>> >
>> > I am not exactly sure about the use case behind this config, but people
>> > claim they need it for specific dockerized environments (see the
>> comments
>> > in the jira ticket). Is anyone familiar with such use cases? We never
>> used
>> > such configs in production as far as I can tell.
>> >
>> > The above config worked without a problem in ZooKeeper 3.4.x, but not
>> > perfectly for 3.5.x. It would be logical to keep supporting it. Still, I
>> > think after the introduction of the dynamic reconfig, we kind of assume
>> > that all the servers have the same server address configurations. So
>> maybe
>> > the config is not even valid anymore?
>> >
>> > Using the 'quorumListenOnAllIPs' config property instead the 0.0.0.0 in
>> the
>> > configs might solve the issue. But if it is the case, then we definitely
>> > should highlight this in the wiki / documentation. Maybe even printing
>> out
>> > a warning during ZooKeeper startup.
>> >
>> > What do you think?
>> >
>> > Kind regards,
>> > Mate
>> >
>>
>
Re: using 0.0.0.0 in the server configs in ZK 3.5+
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
A quote from the Jira ticket where Sebastian tries to explain their setup:
(see
https://issues.apache.org/jira/browse/ZOOKEEPER-2164?focusedCommentId=17037941&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17037941
)
"We are using 0.0.0.0 in our config of the 3-node-cluster as the nodes are
running in separate docker-instances not directly connected to each other.
Which means the zookeeper-container doesn't have its real IP assigned only
using port-forwardings from the docker host to the container to make it
accessible and so using the external IP or the FQDN doesn't allow zookeeper
to start as it complains about not finding that IP in the container. Which
is of course correct. Using the internal Network-IP instead of 0.0.0.0
would result in the same problem as using 0.0.0.0 as all internal networks
of the three docker hosts are using the same network-range in their
internal networks."
re-reading it, I think he meant a bit different config than what I wrote as
an example before... I think he basically connects separate hosts with
tunnels maybe and running ZK in docker in each host without any container
orchestration tool. (?)
On Tue, Feb 18, 2020 at 10:49 AM Enrico Olivelli <eo...@gmail.com>
wrote:
> Honestly I think that using 0.0.0.0 is not so useful.
> I find it very useful that the configuration is the same on every peer,
> except from myid file
> What's the real gain in such configuration?
>
>
> Enrico
>
>
> Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté <sz...@gmail.com> ha
> scritto:
>
> > Hi All,
> >
> > in a recent PR I try to fix an issue we found with Suhas Dantkale in
> > ZOOKEEPER-2146 (see https://github.com/apache/zookeeper/pull/1254). The
> > problem is that in ZooKeeper 3.5+ some quorum members can not rejoin to
> the
> > quorum after a restart if the server configs are set like this:
> >
> > zoo.cfg in server 1:
> > server.1=0.0.0.0:2888:3888
> > server.2=some.fqdn-2.com:2888:3888
> > server.3=some.fqdn-3.com:2888:3888
> >
> > zoo.cfg in server 2:
> > server.1=some.fqdn-1.com:2888:3888
> > server.2=0.0.0.0:2888:3888
> > server.3=some.fqdn-3.com:2888:3888
> >
> > I am not exactly sure about the use case behind this config, but people
> > claim they need it for specific dockerized environments (see the comments
> > in the jira ticket). Is anyone familiar with such use cases? We never
> used
> > such configs in production as far as I can tell.
> >
> > The above config worked without a problem in ZooKeeper 3.4.x, but not
> > perfectly for 3.5.x. It would be logical to keep supporting it. Still, I
> > think after the introduction of the dynamic reconfig, we kind of assume
> > that all the servers have the same server address configurations. So
> maybe
> > the config is not even valid anymore?
> >
> > Using the 'quorumListenOnAllIPs' config property instead the 0.0.0.0 in
> the
> > configs might solve the issue. But if it is the case, then we definitely
> > should highlight this in the wiki / documentation. Maybe even printing
> out
> > a warning during ZooKeeper startup.
> >
> > What do you think?
> >
> > Kind regards,
> > Mate
> >
>
Re: using 0.0.0.0 in the server configs in ZK 3.5+
Posted by Enrico Olivelli <eo...@gmail.com>.
Honestly I think that using 0.0.0.0 is not so useful.
I find it very useful that the configuration is the same on every peer,
except from myid file
What's the real gain in such configuration?
Enrico
Il Mar 18 Feb 2020, 10:32 Szalay-Bekő Máté <sz...@gmail.com> ha
scritto:
> Hi All,
>
> in a recent PR I try to fix an issue we found with Suhas Dantkale in
> ZOOKEEPER-2146 (see https://github.com/apache/zookeeper/pull/1254). The
> problem is that in ZooKeeper 3.5+ some quorum members can not rejoin to the
> quorum after a restart if the server configs are set like this:
>
> zoo.cfg in server 1:
> server.1=0.0.0.0:2888:3888
> server.2=some.fqdn-2.com:2888:3888
> server.3=some.fqdn-3.com:2888:3888
>
> zoo.cfg in server 2:
> server.1=some.fqdn-1.com:2888:3888
> server.2=0.0.0.0:2888:3888
> server.3=some.fqdn-3.com:2888:3888
>
> I am not exactly sure about the use case behind this config, but people
> claim they need it for specific dockerized environments (see the comments
> in the jira ticket). Is anyone familiar with such use cases? We never used
> such configs in production as far as I can tell.
>
> The above config worked without a problem in ZooKeeper 3.4.x, but not
> perfectly for 3.5.x. It would be logical to keep supporting it. Still, I
> think after the introduction of the dynamic reconfig, we kind of assume
> that all the servers have the same server address configurations. So maybe
> the config is not even valid anymore?
>
> Using the 'quorumListenOnAllIPs' config property instead the 0.0.0.0 in the
> configs might solve the issue. But if it is the case, then we definitely
> should highlight this in the wiki / documentation. Maybe even printing out
> a warning during ZooKeeper startup.
>
> What do you think?
>
> Kind regards,
> Mate
>