You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by "Ruel, Ryan" <rr...@akamai.com.INVALID> on 2022/11/15 15:09:32 UTC
Confusion around client ports and dynamic reconfiguration
In my ZooKeeper setup, I am strictly using TLS for both client and quorum communication.
In zookeeper.conf, I have “secureClientPort=2281” defined, and do not have any “clientPort” option set.
In the 3.8.0 documentation on dynamic reconfiguration (https://zookeeper.apache.org/doc/r3.8.0/zookeeperReconfig.html), the documentation says that the old “clientPort” configuration option should not be specified, and instead the new server keyword specification should look like this:
server.<positive id> = <address1>:<port1>:<port2>[:role];[<client port address>:]<client port>**
However, this specification doesn’t consider the secure client port from what I can tell.
In some cases where the server keyword is used, I can just eliminate putting in the client port address and client port, such as within the quorum peer configuration (in zookeeper.conf or within the dynamic configuration file).
In other cases, however, such as using the “reconfig” command in the ZK cli utility, the client port MUST be specified, or a “bad argument” type error is produced.
I of course don’t want to put a dummy port number in the server specification which would then enable insecure communication.
What’s the recommendation for using secure communication only while also using dynamic reconfiguration?
P.S. Another interesting bit in the documentation is the example:
server.1=125.23.63.23:2780:2783:participant;2791
server.2=125.23.63.24:2781:2784:participant;2792
server.3=125.23.63.25:2782:2785:participant;2793
In what use case would you want to use entirely different ports for each server? Or is this just a demonstration that this is possible?
/Ryan
Re: Confusion around client ports and dynamic reconfiguration
Posted by "Chris T." <c....@gmail.com>.
We run secure client port at 2182 and standard port 2181. This 2181 is in
the dynamic config strings. Then we have a firewalld rule to block incoming
traffic on 2181.
For us this works, no problem with Curator Ensemble tracker either if I
recall. Our Curator based clients connect fine on the secure port and we
run frequent dynamic reconfiguration.
Regards
Chris
On 18 November 2022 20:15:39 "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
> Seems others have faced this same problem:
> https://issues.apache.org/jira/browse/ZOOKEEPER-3577
>
> /Ryan
>
> On 11/18/22, 1:26 PM, "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
>
> Enrico,
>
> Sure, I can create a PR for any documentation change suggestions.
>
> I am still having some trouble with this, though.
>
> Using the dynamic reconfiguration command within the CLI tool, I was able
> to specify a client port of 2281 in the server specification (the same port
> as what I have in secureClientPort in zookeeper.conf). This was accepted by
> the quorum and was working just fine.
>
> However, after installing a new ZooKeeper node and initially configuring
> it, it's unable to bind to 2281.
>
> I'm running the most recent stable release (3.7.3).
>
> My configuration is as follows:
>
> zookeeper.conf (I have removed the irrelevant bits for brevity):
> secureClientPort=2281
> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
> ssl.quorum.keyStore.location=/etc/zookeeper/keys/quorum/keyStore.jks
> sslQuorum=true
> X509AuthenticationProvider.superUser=SUPERUSER
> ssl.quorum.trustStore.location=/etc/zookeeper/keys/quorum/trustStore.jks
> authProvider.<our application>=<our custom authentication provider>
> reconfigEnabled=true
> ssl.trustStore.password=<our password>
> dynamicConfigFile=/etc/zookeeper/zookeeper.conf.dynamic.3b00000019
>
> zookeeper.conf.dynamic.3b00000019:
> server.1=100.80.2.1:2888:3888:participant;0.0.0.0:2281
> server.2=100.80.2.2:2888:3888:participant;0.0.0.0:2281
> server.3=100.80.2.3:2888:3888:participant;0.0.0.0:2281
>
> logs:
> 2022-11-18 18:07:26,990 [main] INFO server.NettyServerCnxnFactory - bound
> to port 2281
> 2022-11-18 18:07:26,990 [main] INFO server.NettyServerCnxnFactory -
> binding to port 0.0.0.0/0.0.0.0:2281
> 2022-11-18 18:07:26,995 [main] ERROR quorum.QuorumPeerMain - Unexpected
> exception, exiting abnormally
> java.net.BindException: Address already in use
> at java.base/sun.nio.ch.Net.bind0(Native Method)
> at java.base/sun.nio.ch.Net.bind(Net.java:459)
> at java.base/sun.nio.ch.Net.bind(Net.java:448)
> at
> java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
> at
> io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:141)
> at
> io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
> at
> io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
> at
> io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
> at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
> at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
> at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
>
> So, it seems like it's trying to bind twice to port 2281 using Netty.
>
> If I remove "secureClientPort" from zookeeper.conf and restart, then
> ZooKeeper starts up, but as you can see in the following logs connections
> exception out (presumably because TLS is disabled):
> 2022-11-18 18:23:57,851 [nioEventLoopGroup-4-1] WARN
> server.NettyServerCnxn - Closing connection to /100.80.2.5:36609
> java.io.IOException: Len error 369296129
> at
> org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:521)
> at
> org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:374)
> at
> org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:357)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
>
> /Ryan
>
> On 11/17/22, 8:16 AM, "Enrico Olivelli" <eo...@gmail.com> wrote:
>
> Ruel,
>
> Il giorno mer 16 nov 2022 alle ore 16:15 Ruel, Ryan
> <rr...@akamai.com.invalid> ha scritto:
> >
> > It seems that specifying the SECURE client port in the reconfig command
> does work, while also keeping the same port defined as "secureClientPort"
> in zookeeper.conf.
> >
> > (I thought I had tried this, but may have missed this combination)
> >
> > In any case, some clarification within the documentation may be helpful!
>
> Would you like to send a PR to add these clarifications?
>
> Thanks
> Enrico
>
> >
> > /Ryan
> >
> > On 11/15/22, 10:10 AM, "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
> >
> > In my ZooKeeper setup, I am strictly using TLS for both client and
> quorum communication.
> >
> > In zookeeper.conf, I have “secureClientPort=2281” defined, and do not
> have any “clientPort” option set.
> >
> > In the 3.8.0 documentation on dynamic reconfiguration
> (https://urldefense.com/v3/__https://zookeeper.apache.org/doc/r3.8.0/zookeeperReconfig.html__;!!GjvTz_vk!T5lRlM3A1syL82ZMBR5kWiVdxaCcKIlRhDY-6muCLvhez9gyJTYbbhMkWMiee0evVrX0MQc-eYwvZhl1$
> ), the documentation says that the old “clientPort” configuration option
> should not be specified, and instead the new server keyword specification
> should look like this:
> >
> > server.<positive id> = <address1>:<port1>:<port2>[:role];[<client
> port address>:]<client port>**
> >
> > However, this specification doesn’t consider the secure client port
> from what I can tell.
> >
> > In some cases where the server keyword is used, I can just eliminate
> putting in the client port address and client port, such as within the
> quorum peer configuration (in zookeeper.conf or within the dynamic
> configuration file).
> >
> > In other cases, however, such as using the “reconfig” command in the
> ZK cli utility, the client port MUST be specified, or a “bad argument” type
> error is produced.
> >
> > I of course don’t want to put a dummy port number in the server
> specification which would then enable insecure communication.
> >
> > What’s the recommendation for using secure communication only while
> also using dynamic reconfiguration?
> >
> > P.S. Another interesting bit in the documentation is the example:
> > server.1=125.23.63.23:2780:2783:participant;2791
> > server.2=125.23.63.24:2781:2784:participant;2792
> > server.3=125.23.63.25:2782:2785:participant;2793
> >
> > In what use case would you want to use entirely different ports for
> each server? Or is this just a demonstration that this is possible?
> >
> > /Ryan
> >
> >
> >
> >
> >
> >
Re: Confusion around client ports and dynamic reconfiguration
Posted by "Ruel, Ryan" <rr...@akamai.com.INVALID>.
Seems others have faced this same problem:
https://issues.apache.org/jira/browse/ZOOKEEPER-3577
/Ryan
On 11/18/22, 1:26 PM, "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
Enrico,
Sure, I can create a PR for any documentation change suggestions.
I am still having some trouble with this, though.
Using the dynamic reconfiguration command within the CLI tool, I was able to specify a client port of 2281 in the server specification (the same port as what I have in secureClientPort in zookeeper.conf). This was accepted by the quorum and was working just fine.
However, after installing a new ZooKeeper node and initially configuring it, it's unable to bind to 2281.
I'm running the most recent stable release (3.7.3).
My configuration is as follows:
zookeeper.conf (I have removed the irrelevant bits for brevity):
secureClientPort=2281
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.quorum.keyStore.location=/etc/zookeeper/keys/quorum/keyStore.jks
sslQuorum=true
X509AuthenticationProvider.superUser=SUPERUSER
ssl.quorum.trustStore.location=/etc/zookeeper/keys/quorum/trustStore.jks
authProvider.<our application>=<our custom authentication provider>
reconfigEnabled=true
ssl.trustStore.password=<our password>
dynamicConfigFile=/etc/zookeeper/zookeeper.conf.dynamic.3b00000019
zookeeper.conf.dynamic.3b00000019:
server.1=100.80.2.1:2888:3888:participant;0.0.0.0:2281
server.2=100.80.2.2:2888:3888:participant;0.0.0.0:2281
server.3=100.80.2.3:2888:3888:participant;0.0.0.0:2281
logs:
2022-11-18 18:07:26,990 [main] INFO server.NettyServerCnxnFactory - bound to port 2281
2022-11-18 18:07:26,990 [main] INFO server.NettyServerCnxnFactory - binding to port 0.0.0.0/0.0.0.0:2281
2022-11-18 18:07:26,995 [main] ERROR quorum.QuorumPeerMain - Unexpected exception, exiting abnormally
java.net.BindException: Address already in use
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:459)
at java.base/sun.nio.ch.Net.bind(Net.java:448)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:141)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
So, it seems like it's trying to bind twice to port 2281 using Netty.
If I remove "secureClientPort" from zookeeper.conf and restart, then ZooKeeper starts up, but as you can see in the following logs connections exception out (presumably because TLS is disabled):
2022-11-18 18:23:57,851 [nioEventLoopGroup-4-1] WARN server.NettyServerCnxn - Closing connection to /100.80.2.5:36609
java.io.IOException: Len error 369296129
at org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:521)
at org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:374)
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:357)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
/Ryan
On 11/17/22, 8:16 AM, "Enrico Olivelli" <eo...@gmail.com> wrote:
Ruel,
Il giorno mer 16 nov 2022 alle ore 16:15 Ruel, Ryan
<rr...@akamai.com.invalid> ha scritto:
>
> It seems that specifying the SECURE client port in the reconfig command does work, while also keeping the same port defined as "secureClientPort" in zookeeper.conf.
>
> (I thought I had tried this, but may have missed this combination)
>
> In any case, some clarification within the documentation may be helpful!
Would you like to send a PR to add these clarifications?
Thanks
Enrico
>
> /Ryan
>
> On 11/15/22, 10:10 AM, "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
>
> In my ZooKeeper setup, I am strictly using TLS for both client and quorum communication.
>
> In zookeeper.conf, I have “secureClientPort=2281” defined, and do not have any “clientPort” option set.
>
> In the 3.8.0 documentation on dynamic reconfiguration (https://urldefense.com/v3/__https://zookeeper.apache.org/doc/r3.8.0/zookeeperReconfig.html__;!!GjvTz_vk!T5lRlM3A1syL82ZMBR5kWiVdxaCcKIlRhDY-6muCLvhez9gyJTYbbhMkWMiee0evVrX0MQc-eYwvZhl1$ ), the documentation says that the old “clientPort” configuration option should not be specified, and instead the new server keyword specification should look like this:
>
> server.<positive id> = <address1>:<port1>:<port2>[:role];[<client port address>:]<client port>**
>
> However, this specification doesn’t consider the secure client port from what I can tell.
>
> In some cases where the server keyword is used, I can just eliminate putting in the client port address and client port, such as within the quorum peer configuration (in zookeeper.conf or within the dynamic configuration file).
>
> In other cases, however, such as using the “reconfig” command in the ZK cli utility, the client port MUST be specified, or a “bad argument” type error is produced.
>
> I of course don’t want to put a dummy port number in the server specification which would then enable insecure communication.
>
> What’s the recommendation for using secure communication only while also using dynamic reconfiguration?
>
> P.S. Another interesting bit in the documentation is the example:
> server.1=125.23.63.23:2780:2783:participant;2791
> server.2=125.23.63.24:2781:2784:participant;2792
> server.3=125.23.63.25:2782:2785:participant;2793
>
> In what use case would you want to use entirely different ports for each server? Or is this just a demonstration that this is possible?
>
> /Ryan
>
>
>
>
>
>
Re: Confusion around client ports and dynamic reconfiguration
Posted by "Ruel, Ryan" <rr...@akamai.com.INVALID>.
Enrico,
Sure, I can create a PR for any documentation change suggestions.
I am still having some trouble with this, though.
Using the dynamic reconfiguration command within the CLI tool, I was able to specify a client port of 2281 in the server specification (the same port as what I have in secureClientPort in zookeeper.conf). This was accepted by the quorum and was working just fine.
However, after installing a new ZooKeeper node and initially configuring it, it's unable to bind to 2281.
I'm running the most recent stable release (3.7.3).
My configuration is as follows:
zookeeper.conf (I have removed the irrelevant bits for brevity):
secureClientPort=2281
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.quorum.keyStore.location=/etc/zookeeper/keys/quorum/keyStore.jks
sslQuorum=true
X509AuthenticationProvider.superUser=SUPERUSER
ssl.quorum.trustStore.location=/etc/zookeeper/keys/quorum/trustStore.jks
authProvider.<our application>=<our custom authentication provider>
reconfigEnabled=true
ssl.trustStore.password=<our password>
dynamicConfigFile=/etc/zookeeper/zookeeper.conf.dynamic.3b00000019
zookeeper.conf.dynamic.3b00000019:
server.1=100.80.2.1:2888:3888:participant;0.0.0.0:2281
server.2=100.80.2.2:2888:3888:participant;0.0.0.0:2281
server.3=100.80.2.3:2888:3888:participant;0.0.0.0:2281
logs:
2022-11-18 18:07:26,990 [main] INFO server.NettyServerCnxnFactory - bound to port 2281
2022-11-18 18:07:26,990 [main] INFO server.NettyServerCnxnFactory - binding to port 0.0.0.0/0.0.0.0:2281
2022-11-18 18:07:26,995 [main] ERROR quorum.QuorumPeerMain - Unexpected exception, exiting abnormally
java.net.BindException: Address already in use
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:459)
at java.base/sun.nio.ch.Net.bind(Net.java:448)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:141)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:562)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1334)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:506)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:491)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:973)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:260)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:356)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
So, it seems like it's trying to bind twice to port 2281 using Netty.
If I remove "secureClientPort" from zookeeper.conf and restart, then ZooKeeper starts up, but as you can see in the following logs connections exception out (presumably because TLS is disabled):
2022-11-18 18:23:57,851 [nioEventLoopGroup-4-1] WARN server.NettyServerCnxn - Closing connection to /100.80.2.5:36609
java.io.IOException: Len error 369296129
at org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:521)
at org.apache.zookeeper.server.NettyServerCnxn.processMessage(NettyServerCnxn.java:374)
at org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.channelRead(NettyServerCnxnFactory.java:357)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
/Ryan
On 11/17/22, 8:16 AM, "Enrico Olivelli" <eo...@gmail.com> wrote:
Ruel,
Il giorno mer 16 nov 2022 alle ore 16:15 Ruel, Ryan
<rr...@akamai.com.invalid> ha scritto:
>
> It seems that specifying the SECURE client port in the reconfig command does work, while also keeping the same port defined as "secureClientPort" in zookeeper.conf.
>
> (I thought I had tried this, but may have missed this combination)
>
> In any case, some clarification within the documentation may be helpful!
Would you like to send a PR to add these clarifications?
Thanks
Enrico
>
> /Ryan
>
> On 11/15/22, 10:10 AM, "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
>
> In my ZooKeeper setup, I am strictly using TLS for both client and quorum communication.
>
> In zookeeper.conf, I have “secureClientPort=2281” defined, and do not have any “clientPort” option set.
>
> In the 3.8.0 documentation on dynamic reconfiguration (https://urldefense.com/v3/__https://zookeeper.apache.org/doc/r3.8.0/zookeeperReconfig.html__;!!GjvTz_vk!T5lRlM3A1syL82ZMBR5kWiVdxaCcKIlRhDY-6muCLvhez9gyJTYbbhMkWMiee0evVrX0MQc-eYwvZhl1$ ), the documentation says that the old “clientPort” configuration option should not be specified, and instead the new server keyword specification should look like this:
>
> server.<positive id> = <address1>:<port1>:<port2>[:role];[<client port address>:]<client port>**
>
> However, this specification doesn’t consider the secure client port from what I can tell.
>
> In some cases where the server keyword is used, I can just eliminate putting in the client port address and client port, such as within the quorum peer configuration (in zookeeper.conf or within the dynamic configuration file).
>
> In other cases, however, such as using the “reconfig” command in the ZK cli utility, the client port MUST be specified, or a “bad argument” type error is produced.
>
> I of course don’t want to put a dummy port number in the server specification which would then enable insecure communication.
>
> What’s the recommendation for using secure communication only while also using dynamic reconfiguration?
>
> P.S. Another interesting bit in the documentation is the example:
> server.1=125.23.63.23:2780:2783:participant;2791
> server.2=125.23.63.24:2781:2784:participant;2792
> server.3=125.23.63.25:2782:2785:participant;2793
>
> In what use case would you want to use entirely different ports for each server? Or is this just a demonstration that this is possible?
>
> /Ryan
>
>
>
>
>
>
Re: Confusion around client ports and dynamic reconfiguration
Posted by Enrico Olivelli <eo...@gmail.com>.
Ruel,
Il giorno mer 16 nov 2022 alle ore 16:15 Ruel, Ryan
<rr...@akamai.com.invalid> ha scritto:
>
> It seems that specifying the SECURE client port in the reconfig command does work, while also keeping the same port defined as "secureClientPort" in zookeeper.conf.
>
> (I thought I had tried this, but may have missed this combination)
>
> In any case, some clarification within the documentation may be helpful!
Would you like to send a PR to add these clarifications?
Thanks
Enrico
>
> /Ryan
>
> On 11/15/22, 10:10 AM, "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
>
> In my ZooKeeper setup, I am strictly using TLS for both client and quorum communication.
>
> In zookeeper.conf, I have “secureClientPort=2281” defined, and do not have any “clientPort” option set.
>
> In the 3.8.0 documentation on dynamic reconfiguration (https://urldefense.com/v3/__https://zookeeper.apache.org/doc/r3.8.0/zookeeperReconfig.html__;!!GjvTz_vk!T5lRlM3A1syL82ZMBR5kWiVdxaCcKIlRhDY-6muCLvhez9gyJTYbbhMkWMiee0evVrX0MQc-eYwvZhl1$ ), the documentation says that the old “clientPort” configuration option should not be specified, and instead the new server keyword specification should look like this:
>
> server.<positive id> = <address1>:<port1>:<port2>[:role];[<client port address>:]<client port>**
>
> However, this specification doesn’t consider the secure client port from what I can tell.
>
> In some cases where the server keyword is used, I can just eliminate putting in the client port address and client port, such as within the quorum peer configuration (in zookeeper.conf or within the dynamic configuration file).
>
> In other cases, however, such as using the “reconfig” command in the ZK cli utility, the client port MUST be specified, or a “bad argument” type error is produced.
>
> I of course don’t want to put a dummy port number in the server specification which would then enable insecure communication.
>
> What’s the recommendation for using secure communication only while also using dynamic reconfiguration?
>
> P.S. Another interesting bit in the documentation is the example:
> server.1=125.23.63.23:2780:2783:participant;2791
> server.2=125.23.63.24:2781:2784:participant;2792
> server.3=125.23.63.25:2782:2785:participant;2793
>
> In what use case would you want to use entirely different ports for each server? Or is this just a demonstration that this is possible?
>
> /Ryan
>
>
>
>
>
>
Re: Confusion around client ports and dynamic reconfiguration
Posted by "Ruel, Ryan" <rr...@akamai.com.INVALID>.
It seems that specifying the SECURE client port in the reconfig command does work, while also keeping the same port defined as "secureClientPort" in zookeeper.conf.
(I thought I had tried this, but may have missed this combination)
In any case, some clarification within the documentation may be helpful!
/Ryan
On 11/15/22, 10:10 AM, "Ruel, Ryan" <rr...@akamai.com.INVALID> wrote:
In my ZooKeeper setup, I am strictly using TLS for both client and quorum communication.
In zookeeper.conf, I have “secureClientPort=2281” defined, and do not have any “clientPort” option set.
In the 3.8.0 documentation on dynamic reconfiguration (https://urldefense.com/v3/__https://zookeeper.apache.org/doc/r3.8.0/zookeeperReconfig.html__;!!GjvTz_vk!T5lRlM3A1syL82ZMBR5kWiVdxaCcKIlRhDY-6muCLvhez9gyJTYbbhMkWMiee0evVrX0MQc-eYwvZhl1$ ), the documentation says that the old “clientPort” configuration option should not be specified, and instead the new server keyword specification should look like this:
server.<positive id> = <address1>:<port1>:<port2>[:role];[<client port address>:]<client port>**
However, this specification doesn’t consider the secure client port from what I can tell.
In some cases where the server keyword is used, I can just eliminate putting in the client port address and client port, such as within the quorum peer configuration (in zookeeper.conf or within the dynamic configuration file).
In other cases, however, such as using the “reconfig” command in the ZK cli utility, the client port MUST be specified, or a “bad argument” type error is produced.
I of course don’t want to put a dummy port number in the server specification which would then enable insecure communication.
What’s the recommendation for using secure communication only while also using dynamic reconfiguration?
P.S. Another interesting bit in the documentation is the example:
server.1=125.23.63.23:2780:2783:participant;2791
server.2=125.23.63.24:2781:2784:participant;2792
server.3=125.23.63.25:2782:2785:participant;2793
In what use case would you want to use entirely different ports for each server? Or is this just a demonstration that this is possible?
/Ryan