You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by André Mantas <an...@gmail.com> on 2016/07/21 02:28:26 UTC
Config options not being loaded?
Hi all.
I'm trying to experiment with some config options found in
https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#sc_configuration
but
they are not being loaded/used by the zookeeper server.
I've modified my config file 'zoo.cfg' to:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/root/amantas/zookeeper-3.4.8/data
# the port at which the clients will connect
clientPort=2181
# Requires updates to be synced to media of the transaction log before
finishing processing the update.
# If this option is set to no, ZooKeeper will not require updates to be
synced to the media.
forceSync=no
# Skips ACL checks. This results in a boost in throughput, but opens up
full access to the data tree to everyone.
skipACL=yes
# The observers now log transaction and write snapshot to disk by default
like the participants.
# This reduces the recovery time of the observers on restart.
# Set to "false" to disable this feature. Default is "true"
syncEnabled=false
# ZooKeeper logs transactions to a transaction log.
# After snapCount transactions are written to a log file a snapshot is
started and a new transaction log file is created.
# The default snapCount is 100,000
snapCount=1000000
Then, restarted the zookeeper server:
~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
However, when listing the server configuration I don't see some of the
config params:
~/amantas/zookeeper-3.4.8# nc localhost 2181
conf
clientPort=2181
dataDir=/root/amantas/zookeeper-3.4.8/data/version-2
dataLogDir=/root/amantas/zookeeper-3.4.8/data/version-2
tickTime=2000
maxClientCnxns=60
minSessionTimeout=4000
maxSessionTimeout=40000
serverId=0
What am I doing wrong?
Thanks in advance!
Re: Config options not being loaded?
Posted by Michael Han <ha...@cloudera.com>.
>> using zookeeper only in memory
I don't think you can completely disable persistency - ZK will at least
write transaction logs and snap shot to disk in any case. There is
ZOOKEEPER-866 <https://issues.apache.org/jira/browse/ZOOKEEPER-866> relates
to disable persistency completely for ZK, which is probably what you are
looking for ultimately, but that work is not done yet.
>> what would you suggest?
I think your original config looks OK, in particular:
- 'forceSync=no' will make transaction log write async.
- Set snapCount to a really large number to disable snap shot as it seems
you don't have durable requirement and not care about recovery time.
You might also want to clarify / provide more information regarding your
use case (e.g. work load type, read heavy or write heavy, etc), and maybe
some users here that had similar use cases can provide their insights.
On Thu, Jul 21, 2016 at 2:35 PM, André Mantas <an...@gmail.com>
wrote:
> Ok thanks I'll check it out. As for my other question of using zookeeper
> only in memory, what would you suggest?
>
> Thanks for the help so far.
>
> A 10:18PM Qui, 21 de Jul de 2016, Michael Han <ha...@cloudera.com>
> escreveu:
>
> > I think it is not guaranteed that we can find all configuration options
> in
> > server logs because ZK may not log all available configuration options.
> > Another idea to verify the configuration options are loaded with the
> value
> > you specified is to examine the Java system properties of the JVM ZK
> > process, because all the configuration options are loaded as Java system
> > properties. As an example, skipACL will also be presented as
> > zookeeper.skipACL property.
> >
> > On Thu, Jul 21, 2016 at 2:07 PM, André Mantas <an...@gmail.com>
> > wrote:
> >
> > > Thanks for the reply. Indeed I was able to see the skip ACL config in
> the
> > > server log but not any of the other 3 configs. Was it because I
> > specified a
> > > wrong value for these configs maybe?
> > >
> > > A 9:56PM Qui, 21 de Jul de 2016, Michael Han <ha...@cloudera.com>
> > escreveu:
> > >
> > > > The fact that 'conf' does not print out all configuration options
> > encoded
> > > > in zoo.cfg does not necessarily mean these options are not loaded in
> > ZK.
> > > In
> > > > face the options 'conf' will print out is pretty limited and you can
> > > check
> > > > those options in ZooKeeperServerConf.java
> > > > <
> > > >
> > >
> >
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerConf.java#L26
> > > > >.
> > > > It'll be good to make this class up to date with regards to latest
> > > > available configuration options in ZK but most if not all
> configuration
> > > > options could also be validated in server logs (like the skipACL one
> > > here).
> > > >
> > > > On Thu, Jul 21, 2016 at 11:55 AM, André Mantas <
> andremantas7@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Follow up:
> > > > > With start-foreground I'm able to see that skipACL=yes was
> effective
> > > > (full
> > > > > output in the end of the email):
> > > > >
> > > > > 2016-07-21 20:36:19,433 [myid:] - INFO
> [main:PrepRequestProcessor@87
> > ]
> > > -
> > > > > zookeeper.skipACL=="yes", ACL checks will be skipped
> > > > >
> > > > > What is the problem with the other config values? I'm trying to
> boost
> > > > > performance by disabling the persistency of zookeeper. I'm only
> > > > interested
> > > > > in using zookeeper in memory database (no need for data persistency
> > > after
> > > > > reboot).
> > > > >
> > > > > Any advice on how to achieve this?
> > > > >
> > > > > Thanks
> > > > >
> > > > > -----
> > > > >
> > > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start-foreground
> > > > > ZooKeeper JMX enabled by default
> > > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > > 2016-07-21 20:36:15,060 [myid:] - INFO [main:QuorumPeerConfig@103
> ]
> > -
> > > > > Reading configuration from:
> > > > > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > > 2016-07-21 20:36:15,069 [myid:] - INFO
> > [main:DatadirCleanupManager@78
> > > ]
> > > > -
> > > > > autopurge.snapRetainCount set to 3
> > > > > 2016-07-21 20:36:15,069 [myid:] - INFO
> > [main:DatadirCleanupManager@79
> > > ]
> > > > -
> > > > > autopurge.purgeInterval set to 0
> > > > > 2016-07-21 20:36:15,069 [myid:] - INFO
> > [main:DatadirCleanupManager@101
> > > ]
> > > > -
> > > > > Purge task is not scheduled.
> > > > > 2016-07-21 20:36:15,071 [myid:] - WARN [main:QuorumPeerMain@113]
> -
> > > > Either
> > > > > no config or no quorum defined in config, running in standalone
> mode
> > > > > 2016-07-21 20:36:15,093 [myid:] - INFO [main:QuorumPeerConfig@103
> ]
> > -
> > > > > Reading configuration from:
> > > > > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > > 2016-07-21 20:36:15,094 [myid:] - INFO
> [main:ZooKeeperServerMain@95
> > ]
> > > -
> > > > > Starting server
> > > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18
> GMT
> > > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:host.name=t8.wan
> > > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:java.version=1.8.0_91
> > > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:java.vendor=Oracle Corporation
> > > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:java.home=/usr/lib/jvm/java-8-oracle/jre
> > > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > >
> > > > >
> > > >
> > >
> >
> environment:java.class.path=/root/amantas/zookeeper-3.4.8/bin/../build/classes:/root/amantas/zookeeper-3.4.8/bin/../build/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-api-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/netty-3.7.0.Final.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/log4j-1.2.16.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/jline-0.9.94.jar:/root/amantas/zookeeper-3.4.8/bin/../zookeeper-3.4.8.jar:/root/amantas/zookeeper-3.4.8/bin/../src/java/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../conf:
> > > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > >
> > > > >
> > > >
> > >
> >
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:java.io.tmpdir=/tmp
> > > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:java.compiler=<NA>
> > > > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:os.name=Linux
> > > > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:os.arch=amd64
> > > > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:os.version=4.4.0-21-generic
> > > > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:user.name=root
> > > > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:user.home=/root
> > > > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> > > Server
> > > > > environment:user.dir=/root/amantas/zookeeper-3.4.8
> > > > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@787]
> -
> > > > > tickTime set to 2000
> > > > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@796]
> -
> > > > > minSessionTimeout set to -1
> > > > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@805]
> -
> > > > > maxSessionTimeout set to -1
> > > > > 2016-07-21 20:36:15,132 [myid:] - INFO
> [main:NIOServerCnxnFactory@89
> > ]
> > > -
> > > > > binding to port 0.0.0.0/0.0.0.0:2181
> > > > > 2016-07-21 20:36:19,433 [myid:] - INFO
> [main:PrepRequestProcessor@87
> > ]
> > > -
> > > > > zookeeper.skipACL=="yes", ACL checks will be skipped
> > > > >
> > > > > André Mantas <an...@gmail.com> escreveu no dia quinta,
> > > 21/07/2016
> > > > > às
> > > > > 03:28:
> > > > >
> > > > > > Hi all.
> > > > > >
> > > > > > I'm trying to experiment with some config options found in
> > > > > >
> > > > >
> > > >
> > >
> >
> https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#sc_configuration
> > > > > but
> > > > > > they are not being loaded/used by the zookeeper server.
> > > > > >
> > > > > > I've modified my config file 'zoo.cfg' to:
> > > > > >
> > > > > > # The number of milliseconds of each tick
> > > > > > tickTime=2000
> > > > > > # The number of ticks that the initial
> > > > > > # synchronization phase can take
> > > > > > initLimit=10
> > > > > > # The number of ticks that can pass between
> > > > > > # sending a request and getting an acknowledgement
> > > > > > syncLimit=5
> > > > > > # the directory where the snapshot is stored.
> > > > > > # do not use /tmp for storage, /tmp here is just
> > > > > > # example sakes.
> > > > > > dataDir=/root/amantas/zookeeper-3.4.8/data
> > > > > > # the port at which the clients will connect
> > > > > > clientPort=2181
> > > > > > # Requires updates to be synced to media of the transaction log
> > > before
> > > > > > finishing processing the update.
> > > > > > # If this option is set to no, ZooKeeper will not require updates
> > to
> > > be
> > > > > > synced to the media.
> > > > > > forceSync=no
> > > > > > # Skips ACL checks. This results in a boost in throughput, but
> > opens
> > > up
> > > > > > full access to the data tree to everyone.
> > > > > > skipACL=yes
> > > > > > # The observers now log transaction and write snapshot to disk by
> > > > default
> > > > > > like the participants.
> > > > > > # This reduces the recovery time of the observers on restart.
> > > > > > # Set to "false" to disable this feature. Default is "true"
> > > > > > syncEnabled=false
> > > > > > # ZooKeeper logs transactions to a transaction log.
> > > > > > # After snapCount transactions are written to a log file a
> snapshot
> > > is
> > > > > > started and a new transaction log file is created.
> > > > > > # The default snapCount is 100,000
> > > > > > snapCount=1000000
> > > > > >
> > > > > > Then, restarted the zookeeper server:
> > > > > >
> > > > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh stop
> > > > > > ZooKeeper JMX enabled by default
> > > > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > > > Stopping zookeeper ... STOPPED
> > > > > >
> > > > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start
> > > > > > ZooKeeper JMX enabled by default
> > > > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > > > Starting zookeeper ... STARTED
> > > > > >
> > > > > > However, when listing the server configuration I don't see some
> of
> > > the
> > > > > > config params:
> > > > > >
> > > > > > ~/amantas/zookeeper-3.4.8# nc localhost 2181
> > > > > > conf
> > > > > > clientPort=2181
> > > > > > dataDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > > > > dataLogDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > > > > tickTime=2000
> > > > > > maxClientCnxns=60
> > > > > > minSessionTimeout=4000
> > > > > > maxSessionTimeout=40000
> > > > > > serverId=0
> > > > > >
> > > > > > What am I doing wrong?
> > > > > >
> > > > > > Thanks in advance!
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Cheers
> > > > Michael.
> > > >
> > >
> >
> >
> >
> > --
> > Cheers
> > Michael.
> >
>
--
Cheers
Michael.
Re: Config options not being loaded?
Posted by André Mantas <an...@gmail.com>.
Ok thanks I'll check it out. As for my other question of using zookeeper
only in memory, what would you suggest?
Thanks for the help so far.
A 10:18PM Qui, 21 de Jul de 2016, Michael Han <ha...@cloudera.com> escreveu:
> I think it is not guaranteed that we can find all configuration options in
> server logs because ZK may not log all available configuration options.
> Another idea to verify the configuration options are loaded with the value
> you specified is to examine the Java system properties of the JVM ZK
> process, because all the configuration options are loaded as Java system
> properties. As an example, skipACL will also be presented as
> zookeeper.skipACL property.
>
> On Thu, Jul 21, 2016 at 2:07 PM, André Mantas <an...@gmail.com>
> wrote:
>
> > Thanks for the reply. Indeed I was able to see the skip ACL config in the
> > server log but not any of the other 3 configs. Was it because I
> specified a
> > wrong value for these configs maybe?
> >
> > A 9:56PM Qui, 21 de Jul de 2016, Michael Han <ha...@cloudera.com>
> escreveu:
> >
> > > The fact that 'conf' does not print out all configuration options
> encoded
> > > in zoo.cfg does not necessarily mean these options are not loaded in
> ZK.
> > In
> > > face the options 'conf' will print out is pretty limited and you can
> > check
> > > those options in ZooKeeperServerConf.java
> > > <
> > >
> >
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerConf.java#L26
> > > >.
> > > It'll be good to make this class up to date with regards to latest
> > > available configuration options in ZK but most if not all configuration
> > > options could also be validated in server logs (like the skipACL one
> > here).
> > >
> > > On Thu, Jul 21, 2016 at 11:55 AM, André Mantas <andremantas7@gmail.com
> >
> > > wrote:
> > >
> > > > Follow up:
> > > > With start-foreground I'm able to see that skipACL=yes was effective
> > > (full
> > > > output in the end of the email):
> > > >
> > > > 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87
> ]
> > -
> > > > zookeeper.skipACL=="yes", ACL checks will be skipped
> > > >
> > > > What is the problem with the other config values? I'm trying to boost
> > > > performance by disabling the persistency of zookeeper. I'm only
> > > interested
> > > > in using zookeeper in memory database (no need for data persistency
> > after
> > > > reboot).
> > > >
> > > > Any advice on how to achieve this?
> > > >
> > > > Thanks
> > > >
> > > > -----
> > > >
> > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start-foreground
> > > > ZooKeeper JMX enabled by default
> > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > 2016-07-21 20:36:15,060 [myid:] - INFO [main:QuorumPeerConfig@103]
> -
> > > > Reading configuration from:
> > > > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > 2016-07-21 20:36:15,069 [myid:] - INFO
> [main:DatadirCleanupManager@78
> > ]
> > > -
> > > > autopurge.snapRetainCount set to 3
> > > > 2016-07-21 20:36:15,069 [myid:] - INFO
> [main:DatadirCleanupManager@79
> > ]
> > > -
> > > > autopurge.purgeInterval set to 0
> > > > 2016-07-21 20:36:15,069 [myid:] - INFO
> [main:DatadirCleanupManager@101
> > ]
> > > -
> > > > Purge task is not scheduled.
> > > > 2016-07-21 20:36:15,071 [myid:] - WARN [main:QuorumPeerMain@113] -
> > > Either
> > > > no config or no quorum defined in config, running in standalone mode
> > > > 2016-07-21 20:36:15,093 [myid:] - INFO [main:QuorumPeerConfig@103]
> -
> > > > Reading configuration from:
> > > > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > 2016-07-21 20:36:15,094 [myid:] - INFO [main:ZooKeeperServerMain@95
> ]
> > -
> > > > Starting server
> > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT
> > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:host.name=t8.wan
> > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:java.version=1.8.0_91
> > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:java.vendor=Oracle Corporation
> > > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:java.home=/usr/lib/jvm/java-8-oracle/jre
> > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > Server
> > > >
> > > >
> > >
> >
> environment:java.class.path=/root/amantas/zookeeper-3.4.8/bin/../build/classes:/root/amantas/zookeeper-3.4.8/bin/../build/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-api-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/netty-3.7.0.Final.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/log4j-1.2.16.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/jline-0.9.94.jar:/root/amantas/zookeeper-3.4.8/bin/../zookeeper-3.4.8.jar:/root/amantas/zookeeper-3.4.8/bin/../src/java/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../conf:
> > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > Server
> > > >
> > > >
> > >
> >
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:java.io.tmpdir=/tmp
> > > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:java.compiler=<NA>
> > > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:os.name=Linux
> > > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:os.arch=amd64
> > > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:os.version=4.4.0-21-generic
> > > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:user.name=root
> > > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:user.home=/root
> > > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> > Server
> > > > environment:user.dir=/root/amantas/zookeeper-3.4.8
> > > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@787] -
> > > > tickTime set to 2000
> > > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@796] -
> > > > minSessionTimeout set to -1
> > > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@805] -
> > > > maxSessionTimeout set to -1
> > > > 2016-07-21 20:36:15,132 [myid:] - INFO [main:NIOServerCnxnFactory@89
> ]
> > -
> > > > binding to port 0.0.0.0/0.0.0.0:2181
> > > > 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87
> ]
> > -
> > > > zookeeper.skipACL=="yes", ACL checks will be skipped
> > > >
> > > > André Mantas <an...@gmail.com> escreveu no dia quinta,
> > 21/07/2016
> > > > às
> > > > 03:28:
> > > >
> > > > > Hi all.
> > > > >
> > > > > I'm trying to experiment with some config options found in
> > > > >
> > > >
> > >
> >
> https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#sc_configuration
> > > > but
> > > > > they are not being loaded/used by the zookeeper server.
> > > > >
> > > > > I've modified my config file 'zoo.cfg' to:
> > > > >
> > > > > # The number of milliseconds of each tick
> > > > > tickTime=2000
> > > > > # The number of ticks that the initial
> > > > > # synchronization phase can take
> > > > > initLimit=10
> > > > > # The number of ticks that can pass between
> > > > > # sending a request and getting an acknowledgement
> > > > > syncLimit=5
> > > > > # the directory where the snapshot is stored.
> > > > > # do not use /tmp for storage, /tmp here is just
> > > > > # example sakes.
> > > > > dataDir=/root/amantas/zookeeper-3.4.8/data
> > > > > # the port at which the clients will connect
> > > > > clientPort=2181
> > > > > # Requires updates to be synced to media of the transaction log
> > before
> > > > > finishing processing the update.
> > > > > # If this option is set to no, ZooKeeper will not require updates
> to
> > be
> > > > > synced to the media.
> > > > > forceSync=no
> > > > > # Skips ACL checks. This results in a boost in throughput, but
> opens
> > up
> > > > > full access to the data tree to everyone.
> > > > > skipACL=yes
> > > > > # The observers now log transaction and write snapshot to disk by
> > > default
> > > > > like the participants.
> > > > > # This reduces the recovery time of the observers on restart.
> > > > > # Set to "false" to disable this feature. Default is "true"
> > > > > syncEnabled=false
> > > > > # ZooKeeper logs transactions to a transaction log.
> > > > > # After snapCount transactions are written to a log file a snapshot
> > is
> > > > > started and a new transaction log file is created.
> > > > > # The default snapCount is 100,000
> > > > > snapCount=1000000
> > > > >
> > > > > Then, restarted the zookeeper server:
> > > > >
> > > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh stop
> > > > > ZooKeeper JMX enabled by default
> > > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > > Stopping zookeeper ... STOPPED
> > > > >
> > > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start
> > > > > ZooKeeper JMX enabled by default
> > > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > > Starting zookeeper ... STARTED
> > > > >
> > > > > However, when listing the server configuration I don't see some of
> > the
> > > > > config params:
> > > > >
> > > > > ~/amantas/zookeeper-3.4.8# nc localhost 2181
> > > > > conf
> > > > > clientPort=2181
> > > > > dataDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > > > dataLogDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > > > tickTime=2000
> > > > > maxClientCnxns=60
> > > > > minSessionTimeout=4000
> > > > > maxSessionTimeout=40000
> > > > > serverId=0
> > > > >
> > > > > What am I doing wrong?
> > > > >
> > > > > Thanks in advance!
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Cheers
> > > Michael.
> > >
> >
>
>
>
> --
> Cheers
> Michael.
>
Re: Config options not being loaded?
Posted by Michael Han <ha...@cloudera.com>.
I think it is not guaranteed that we can find all configuration options in
server logs because ZK may not log all available configuration options.
Another idea to verify the configuration options are loaded with the value
you specified is to examine the Java system properties of the JVM ZK
process, because all the configuration options are loaded as Java system
properties. As an example, skipACL will also be presented as
zookeeper.skipACL property.
On Thu, Jul 21, 2016 at 2:07 PM, André Mantas <an...@gmail.com>
wrote:
> Thanks for the reply. Indeed I was able to see the skip ACL config in the
> server log but not any of the other 3 configs. Was it because I specified a
> wrong value for these configs maybe?
>
> A 9:56PM Qui, 21 de Jul de 2016, Michael Han <ha...@cloudera.com> escreveu:
>
> > The fact that 'conf' does not print out all configuration options encoded
> > in zoo.cfg does not necessarily mean these options are not loaded in ZK.
> In
> > face the options 'conf' will print out is pretty limited and you can
> check
> > those options in ZooKeeperServerConf.java
> > <
> >
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerConf.java#L26
> > >.
> > It'll be good to make this class up to date with regards to latest
> > available configuration options in ZK but most if not all configuration
> > options could also be validated in server logs (like the skipACL one
> here).
> >
> > On Thu, Jul 21, 2016 at 11:55 AM, André Mantas <an...@gmail.com>
> > wrote:
> >
> > > Follow up:
> > > With start-foreground I'm able to see that skipACL=yes was effective
> > (full
> > > output in the end of the email):
> > >
> > > 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87]
> -
> > > zookeeper.skipACL=="yes", ACL checks will be skipped
> > >
> > > What is the problem with the other config values? I'm trying to boost
> > > performance by disabling the persistency of zookeeper. I'm only
> > interested
> > > in using zookeeper in memory database (no need for data persistency
> after
> > > reboot).
> > >
> > > Any advice on how to achieve this?
> > >
> > > Thanks
> > >
> > > -----
> > >
> > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start-foreground
> > > ZooKeeper JMX enabled by default
> > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > 2016-07-21 20:36:15,060 [myid:] - INFO [main:QuorumPeerConfig@103] -
> > > Reading configuration from:
> > > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@78
> ]
> > -
> > > autopurge.snapRetainCount set to 3
> > > 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@79
> ]
> > -
> > > autopurge.purgeInterval set to 0
> > > 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@101
> ]
> > -
> > > Purge task is not scheduled.
> > > 2016-07-21 20:36:15,071 [myid:] - WARN [main:QuorumPeerMain@113] -
> > Either
> > > no config or no quorum defined in config, running in standalone mode
> > > 2016-07-21 20:36:15,093 [myid:] - INFO [main:QuorumPeerConfig@103] -
> > > Reading configuration from:
> > > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > 2016-07-21 20:36:15,094 [myid:] - INFO [main:ZooKeeperServerMain@95]
> -
> > > Starting server
> > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT
> > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:host.name=t8.wan
> > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:java.version=1.8.0_91
> > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:java.vendor=Oracle Corporation
> > > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:java.home=/usr/lib/jvm/java-8-oracle/jre
> > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> Server
> > >
> > >
> >
> environment:java.class.path=/root/amantas/zookeeper-3.4.8/bin/../build/classes:/root/amantas/zookeeper-3.4.8/bin/../build/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-api-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/netty-3.7.0.Final.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/log4j-1.2.16.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/jline-0.9.94.jar:/root/amantas/zookeeper-3.4.8/bin/../zookeeper-3.4.8.jar:/root/amantas/zookeeper-3.4.8/bin/../src/java/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../conf:
> > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> Server
> > >
> > >
> >
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:java.io.tmpdir=/tmp
> > > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:java.compiler=<NA>
> > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:os.name=Linux
> > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:os.arch=amd64
> > > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:os.version=4.4.0-21-generic
> > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:user.name=root
> > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:user.home=/root
> > > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] -
> Server
> > > environment:user.dir=/root/amantas/zookeeper-3.4.8
> > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@787] -
> > > tickTime set to 2000
> > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@796] -
> > > minSessionTimeout set to -1
> > > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@805] -
> > > maxSessionTimeout set to -1
> > > 2016-07-21 20:36:15,132 [myid:] - INFO [main:NIOServerCnxnFactory@89]
> -
> > > binding to port 0.0.0.0/0.0.0.0:2181
> > > 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87]
> -
> > > zookeeper.skipACL=="yes", ACL checks will be skipped
> > >
> > > André Mantas <an...@gmail.com> escreveu no dia quinta,
> 21/07/2016
> > > às
> > > 03:28:
> > >
> > > > Hi all.
> > > >
> > > > I'm trying to experiment with some config options found in
> > > >
> > >
> >
> https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#sc_configuration
> > > but
> > > > they are not being loaded/used by the zookeeper server.
> > > >
> > > > I've modified my config file 'zoo.cfg' to:
> > > >
> > > > # The number of milliseconds of each tick
> > > > tickTime=2000
> > > > # The number of ticks that the initial
> > > > # synchronization phase can take
> > > > initLimit=10
> > > > # The number of ticks that can pass between
> > > > # sending a request and getting an acknowledgement
> > > > syncLimit=5
> > > > # the directory where the snapshot is stored.
> > > > # do not use /tmp for storage, /tmp here is just
> > > > # example sakes.
> > > > dataDir=/root/amantas/zookeeper-3.4.8/data
> > > > # the port at which the clients will connect
> > > > clientPort=2181
> > > > # Requires updates to be synced to media of the transaction log
> before
> > > > finishing processing the update.
> > > > # If this option is set to no, ZooKeeper will not require updates to
> be
> > > > synced to the media.
> > > > forceSync=no
> > > > # Skips ACL checks. This results in a boost in throughput, but opens
> up
> > > > full access to the data tree to everyone.
> > > > skipACL=yes
> > > > # The observers now log transaction and write snapshot to disk by
> > default
> > > > like the participants.
> > > > # This reduces the recovery time of the observers on restart.
> > > > # Set to "false" to disable this feature. Default is "true"
> > > > syncEnabled=false
> > > > # ZooKeeper logs transactions to a transaction log.
> > > > # After snapCount transactions are written to a log file a snapshot
> is
> > > > started and a new transaction log file is created.
> > > > # The default snapCount is 100,000
> > > > snapCount=1000000
> > > >
> > > > Then, restarted the zookeeper server:
> > > >
> > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh stop
> > > > ZooKeeper JMX enabled by default
> > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > Stopping zookeeper ... STOPPED
> > > >
> > > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start
> > > > ZooKeeper JMX enabled by default
> > > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > > Starting zookeeper ... STARTED
> > > >
> > > > However, when listing the server configuration I don't see some of
> the
> > > > config params:
> > > >
> > > > ~/amantas/zookeeper-3.4.8# nc localhost 2181
> > > > conf
> > > > clientPort=2181
> > > > dataDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > > dataLogDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > > tickTime=2000
> > > > maxClientCnxns=60
> > > > minSessionTimeout=4000
> > > > maxSessionTimeout=40000
> > > > serverId=0
> > > >
> > > > What am I doing wrong?
> > > >
> > > > Thanks in advance!
> > > >
> > >
> >
> >
> >
> > --
> > Cheers
> > Michael.
> >
>
--
Cheers
Michael.
Re: Config options not being loaded?
Posted by André Mantas <an...@gmail.com>.
Thanks for the reply. Indeed I was able to see the skip ACL config in the
server log but not any of the other 3 configs. Was it because I specified a
wrong value for these configs maybe?
A 9:56PM Qui, 21 de Jul de 2016, Michael Han <ha...@cloudera.com> escreveu:
> The fact that 'conf' does not print out all configuration options encoded
> in zoo.cfg does not necessarily mean these options are not loaded in ZK. In
> face the options 'conf' will print out is pretty limited and you can check
> those options in ZooKeeperServerConf.java
> <
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerConf.java#L26
> >.
> It'll be good to make this class up to date with regards to latest
> available configuration options in ZK but most if not all configuration
> options could also be validated in server logs (like the skipACL one here).
>
> On Thu, Jul 21, 2016 at 11:55 AM, André Mantas <an...@gmail.com>
> wrote:
>
> > Follow up:
> > With start-foreground I'm able to see that skipACL=yes was effective
> (full
> > output in the end of the email):
> >
> > 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87] -
> > zookeeper.skipACL=="yes", ACL checks will be skipped
> >
> > What is the problem with the other config values? I'm trying to boost
> > performance by disabling the persistency of zookeeper. I'm only
> interested
> > in using zookeeper in memory database (no need for data persistency after
> > reboot).
> >
> > Any advice on how to achieve this?
> >
> > Thanks
> >
> > -----
> >
> > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start-foreground
> > ZooKeeper JMX enabled by default
> > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > 2016-07-21 20:36:15,060 [myid:] - INFO [main:QuorumPeerConfig@103] -
> > Reading configuration from:
> > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@78]
> -
> > autopurge.snapRetainCount set to 3
> > 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@79]
> -
> > autopurge.purgeInterval set to 0
> > 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@101]
> -
> > Purge task is not scheduled.
> > 2016-07-21 20:36:15,071 [myid:] - WARN [main:QuorumPeerMain@113] -
> Either
> > no config or no quorum defined in config, running in standalone mode
> > 2016-07-21 20:36:15,093 [myid:] - INFO [main:QuorumPeerConfig@103] -
> > Reading configuration from:
> > /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > 2016-07-21 20:36:15,094 [myid:] - INFO [main:ZooKeeperServerMain@95] -
> > Starting server
> > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> > environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT
> > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> > environment:host.name=t8.wan
> > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> > environment:java.version=1.8.0_91
> > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> > environment:java.vendor=Oracle Corporation
> > 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> > environment:java.home=/usr/lib/jvm/java-8-oracle/jre
> > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
> >
> >
> environment:java.class.path=/root/amantas/zookeeper-3.4.8/bin/../build/classes:/root/amantas/zookeeper-3.4.8/bin/../build/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-api-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/netty-3.7.0.Final.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/log4j-1.2.16.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/jline-0.9.94.jar:/root/amantas/zookeeper-3.4.8/bin/../zookeeper-3.4.8.jar:/root/amantas/zookeeper-3.4.8/bin/../src/java/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../conf:
> > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
> >
> >
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
> > environment:java.io.tmpdir=/tmp
> > 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
> > environment:java.compiler=<NA>
> > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
> > environment:os.name=Linux
> > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
> > environment:os.arch=amd64
> > 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
> > environment:os.version=4.4.0-21-generic
> > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
> > environment:user.name=root
> > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
> > environment:user.home=/root
> > 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
> > environment:user.dir=/root/amantas/zookeeper-3.4.8
> > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@787] -
> > tickTime set to 2000
> > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@796] -
> > minSessionTimeout set to -1
> > 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@805] -
> > maxSessionTimeout set to -1
> > 2016-07-21 20:36:15,132 [myid:] - INFO [main:NIOServerCnxnFactory@89] -
> > binding to port 0.0.0.0/0.0.0.0:2181
> > 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87] -
> > zookeeper.skipACL=="yes", ACL checks will be skipped
> >
> > André Mantas <an...@gmail.com> escreveu no dia quinta, 21/07/2016
> > às
> > 03:28:
> >
> > > Hi all.
> > >
> > > I'm trying to experiment with some config options found in
> > >
> >
> https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#sc_configuration
> > but
> > > they are not being loaded/used by the zookeeper server.
> > >
> > > I've modified my config file 'zoo.cfg' to:
> > >
> > > # The number of milliseconds of each tick
> > > tickTime=2000
> > > # The number of ticks that the initial
> > > # synchronization phase can take
> > > initLimit=10
> > > # The number of ticks that can pass between
> > > # sending a request and getting an acknowledgement
> > > syncLimit=5
> > > # the directory where the snapshot is stored.
> > > # do not use /tmp for storage, /tmp here is just
> > > # example sakes.
> > > dataDir=/root/amantas/zookeeper-3.4.8/data
> > > # the port at which the clients will connect
> > > clientPort=2181
> > > # Requires updates to be synced to media of the transaction log before
> > > finishing processing the update.
> > > # If this option is set to no, ZooKeeper will not require updates to be
> > > synced to the media.
> > > forceSync=no
> > > # Skips ACL checks. This results in a boost in throughput, but opens up
> > > full access to the data tree to everyone.
> > > skipACL=yes
> > > # The observers now log transaction and write snapshot to disk by
> default
> > > like the participants.
> > > # This reduces the recovery time of the observers on restart.
> > > # Set to "false" to disable this feature. Default is "true"
> > > syncEnabled=false
> > > # ZooKeeper logs transactions to a transaction log.
> > > # After snapCount transactions are written to a log file a snapshot is
> > > started and a new transaction log file is created.
> > > # The default snapCount is 100,000
> > > snapCount=1000000
> > >
> > > Then, restarted the zookeeper server:
> > >
> > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh stop
> > > ZooKeeper JMX enabled by default
> > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > Stopping zookeeper ... STOPPED
> > >
> > > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start
> > > ZooKeeper JMX enabled by default
> > > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > > Starting zookeeper ... STARTED
> > >
> > > However, when listing the server configuration I don't see some of the
> > > config params:
> > >
> > > ~/amantas/zookeeper-3.4.8# nc localhost 2181
> > > conf
> > > clientPort=2181
> > > dataDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > dataLogDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > > tickTime=2000
> > > maxClientCnxns=60
> > > minSessionTimeout=4000
> > > maxSessionTimeout=40000
> > > serverId=0
> > >
> > > What am I doing wrong?
> > >
> > > Thanks in advance!
> > >
> >
>
>
>
> --
> Cheers
> Michael.
>
Re: Config options not being loaded?
Posted by Michael Han <ha...@cloudera.com>.
The fact that 'conf' does not print out all configuration options encoded
in zoo.cfg does not necessarily mean these options are not loaded in ZK. In
face the options 'conf' will print out is pretty limited and you can check
those options in ZooKeeperServerConf.java
<https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/ZooKeeperServerConf.java#L26>.
It'll be good to make this class up to date with regards to latest
available configuration options in ZK but most if not all configuration
options could also be validated in server logs (like the skipACL one here).
On Thu, Jul 21, 2016 at 11:55 AM, André Mantas <an...@gmail.com>
wrote:
> Follow up:
> With start-foreground I'm able to see that skipACL=yes was effective (full
> output in the end of the email):
>
> 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87] -
> zookeeper.skipACL=="yes", ACL checks will be skipped
>
> What is the problem with the other config values? I'm trying to boost
> performance by disabling the persistency of zookeeper. I'm only interested
> in using zookeeper in memory database (no need for data persistency after
> reboot).
>
> Any advice on how to achieve this?
>
> Thanks
>
> -----
>
> ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start-foreground
> ZooKeeper JMX enabled by default
> Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> 2016-07-21 20:36:15,060 [myid:] - INFO [main:QuorumPeerConfig@103] -
> Reading configuration from:
> /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@78] -
> autopurge.snapRetainCount set to 3
> 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@79] -
> autopurge.purgeInterval set to 0
> 2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@101] -
> Purge task is not scheduled.
> 2016-07-21 20:36:15,071 [myid:] - WARN [main:QuorumPeerMain@113] - Either
> no config or no quorum defined in config, running in standalone mode
> 2016-07-21 20:36:15,093 [myid:] - INFO [main:QuorumPeerConfig@103] -
> Reading configuration from:
> /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> 2016-07-21 20:36:15,094 [myid:] - INFO [main:ZooKeeperServerMain@95] -
> Starting server
> 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT
> 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> environment:host.name=t8.wan
> 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> environment:java.version=1.8.0_91
> 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> environment:java.vendor=Oracle Corporation
> 2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
> environment:java.home=/usr/lib/jvm/java-8-oracle/jre
> 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
>
> environment:java.class.path=/root/amantas/zookeeper-3.4.8/bin/../build/classes:/root/amantas/zookeeper-3.4.8/bin/../build/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-api-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/netty-3.7.0.Final.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/log4j-1.2.16.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/jline-0.9.94.jar:/root/amantas/zookeeper-3.4.8/bin/../zookeeper-3.4.8.jar:/root/amantas/zookeeper-3.4.8/bin/../src/java/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../conf:
> 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
>
> environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
> environment:java.io.tmpdir=/tmp
> 2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
> environment:java.compiler=<NA>
> 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
> environment:os.name=Linux
> 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
> environment:os.arch=amd64
> 2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
> environment:os.version=4.4.0-21-generic
> 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
> environment:user.name=root
> 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
> environment:user.home=/root
> 2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
> environment:user.dir=/root/amantas/zookeeper-3.4.8
> 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@787] -
> tickTime set to 2000
> 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@796] -
> minSessionTimeout set to -1
> 2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@805] -
> maxSessionTimeout set to -1
> 2016-07-21 20:36:15,132 [myid:] - INFO [main:NIOServerCnxnFactory@89] -
> binding to port 0.0.0.0/0.0.0.0:2181
> 2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87] -
> zookeeper.skipACL=="yes", ACL checks will be skipped
>
> André Mantas <an...@gmail.com> escreveu no dia quinta, 21/07/2016
> às
> 03:28:
>
> > Hi all.
> >
> > I'm trying to experiment with some config options found in
> >
> https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#sc_configuration
> but
> > they are not being loaded/used by the zookeeper server.
> >
> > I've modified my config file 'zoo.cfg' to:
> >
> > # The number of milliseconds of each tick
> > tickTime=2000
> > # The number of ticks that the initial
> > # synchronization phase can take
> > initLimit=10
> > # The number of ticks that can pass between
> > # sending a request and getting an acknowledgement
> > syncLimit=5
> > # the directory where the snapshot is stored.
> > # do not use /tmp for storage, /tmp here is just
> > # example sakes.
> > dataDir=/root/amantas/zookeeper-3.4.8/data
> > # the port at which the clients will connect
> > clientPort=2181
> > # Requires updates to be synced to media of the transaction log before
> > finishing processing the update.
> > # If this option is set to no, ZooKeeper will not require updates to be
> > synced to the media.
> > forceSync=no
> > # Skips ACL checks. This results in a boost in throughput, but opens up
> > full access to the data tree to everyone.
> > skipACL=yes
> > # The observers now log transaction and write snapshot to disk by default
> > like the participants.
> > # This reduces the recovery time of the observers on restart.
> > # Set to "false" to disable this feature. Default is "true"
> > syncEnabled=false
> > # ZooKeeper logs transactions to a transaction log.
> > # After snapCount transactions are written to a log file a snapshot is
> > started and a new transaction log file is created.
> > # The default snapCount is 100,000
> > snapCount=1000000
> >
> > Then, restarted the zookeeper server:
> >
> > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh stop
> > ZooKeeper JMX enabled by default
> > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > Stopping zookeeper ... STOPPED
> >
> > ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start
> > ZooKeeper JMX enabled by default
> > Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> > Starting zookeeper ... STARTED
> >
> > However, when listing the server configuration I don't see some of the
> > config params:
> >
> > ~/amantas/zookeeper-3.4.8# nc localhost 2181
> > conf
> > clientPort=2181
> > dataDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > dataLogDir=/root/amantas/zookeeper-3.4.8/data/version-2
> > tickTime=2000
> > maxClientCnxns=60
> > minSessionTimeout=4000
> > maxSessionTimeout=40000
> > serverId=0
> >
> > What am I doing wrong?
> >
> > Thanks in advance!
> >
>
--
Cheers
Michael.
Re: Config options not being loaded?
Posted by André Mantas <an...@gmail.com>.
Follow up:
With start-foreground I'm able to see that skipACL=yes was effective (full
output in the end of the email):
2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87] -
zookeeper.skipACL=="yes", ACL checks will be skipped
What is the problem with the other config values? I'm trying to boost
performance by disabling the persistency of zookeeper. I'm only interested
in using zookeeper in memory database (no need for data persistency after
reboot).
Any advice on how to achieve this?
Thanks
-----
~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start-foreground
ZooKeeper JMX enabled by default
Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
2016-07-21 20:36:15,060 [myid:] - INFO [main:QuorumPeerConfig@103] -
Reading configuration from:
/root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@78] -
autopurge.snapRetainCount set to 3
2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@79] -
autopurge.purgeInterval set to 0
2016-07-21 20:36:15,069 [myid:] - INFO [main:DatadirCleanupManager@101] -
Purge task is not scheduled.
2016-07-21 20:36:15,071 [myid:] - WARN [main:QuorumPeerMain@113] - Either
no config or no quorum defined in config, running in standalone mode
2016-07-21 20:36:15,093 [myid:] - INFO [main:QuorumPeerConfig@103] -
Reading configuration from:
/root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
2016-07-21 20:36:15,094 [myid:] - INFO [main:ZooKeeperServerMain@95] -
Starting server
2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
environment:zookeeper.version=3.4.8--1, built on 02/06/2016 03:18 GMT
2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
environment:host.name=t8.wan
2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
environment:java.version=1.8.0_91
2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
environment:java.vendor=Oracle Corporation
2016-07-21 20:36:15,105 [myid:] - INFO [main:Environment@100] - Server
environment:java.home=/usr/lib/jvm/java-8-oracle/jre
2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
environment:java.class.path=/root/amantas/zookeeper-3.4.8/bin/../build/classes:/root/amantas/zookeeper-3.4.8/bin/../build/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-log4j12-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/slf4j-api-1.6.1.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/netty-3.7.0.Final.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/log4j-1.2.16.jar:/root/amantas/zookeeper-3.4.8/bin/../lib/jline-0.9.94.jar:/root/amantas/zookeeper-3.4.8/bin/../zookeeper-3.4.8.jar:/root/amantas/zookeeper-3.4.8/bin/../src/java/lib/*.jar:/root/amantas/zookeeper-3.4.8/bin/../conf:
2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
environment:java.io.tmpdir=/tmp
2016-07-21 20:36:15,106 [myid:] - INFO [main:Environment@100] - Server
environment:java.compiler=<NA>
2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
environment:os.name=Linux
2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
environment:os.arch=amd64
2016-07-21 20:36:15,108 [myid:] - INFO [main:Environment@100] - Server
environment:os.version=4.4.0-21-generic
2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
environment:user.name=root
2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
environment:user.home=/root
2016-07-21 20:36:15,109 [myid:] - INFO [main:Environment@100] - Server
environment:user.dir=/root/amantas/zookeeper-3.4.8
2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@787] -
tickTime set to 2000
2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@796] -
minSessionTimeout set to -1
2016-07-21 20:36:15,118 [myid:] - INFO [main:ZooKeeperServer@805] -
maxSessionTimeout set to -1
2016-07-21 20:36:15,132 [myid:] - INFO [main:NIOServerCnxnFactory@89] -
binding to port 0.0.0.0/0.0.0.0:2181
2016-07-21 20:36:19,433 [myid:] - INFO [main:PrepRequestProcessor@87] -
zookeeper.skipACL=="yes", ACL checks will be skipped
André Mantas <an...@gmail.com> escreveu no dia quinta, 21/07/2016 às
03:28:
> Hi all.
>
> I'm trying to experiment with some config options found in
> https://zookeeper.apache.org/doc/r3.4.8/zookeeperAdmin.html#sc_configuration but
> they are not being loaded/used by the zookeeper server.
>
> I've modified my config file 'zoo.cfg' to:
>
> # The number of milliseconds of each tick
> tickTime=2000
> # The number of ticks that the initial
> # synchronization phase can take
> initLimit=10
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=5
> # the directory where the snapshot is stored.
> # do not use /tmp for storage, /tmp here is just
> # example sakes.
> dataDir=/root/amantas/zookeeper-3.4.8/data
> # the port at which the clients will connect
> clientPort=2181
> # Requires updates to be synced to media of the transaction log before
> finishing processing the update.
> # If this option is set to no, ZooKeeper will not require updates to be
> synced to the media.
> forceSync=no
> # Skips ACL checks. This results in a boost in throughput, but opens up
> full access to the data tree to everyone.
> skipACL=yes
> # The observers now log transaction and write snapshot to disk by default
> like the participants.
> # This reduces the recovery time of the observers on restart.
> # Set to "false" to disable this feature. Default is "true"
> syncEnabled=false
> # ZooKeeper logs transactions to a transaction log.
> # After snapCount transactions are written to a log file a snapshot is
> started and a new transaction log file is created.
> # The default snapCount is 100,000
> snapCount=1000000
>
> Then, restarted the zookeeper server:
>
> ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh stop
> ZooKeeper JMX enabled by default
> Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> Stopping zookeeper ... STOPPED
>
> ~/amantas/zookeeper-3.4.8# ./bin/zkServer.sh start
> ZooKeeper JMX enabled by default
> Using config: /root/amantas/zookeeper-3.4.8/bin/../conf/zoo.cfg
> Starting zookeeper ... STARTED
>
> However, when listing the server configuration I don't see some of the
> config params:
>
> ~/amantas/zookeeper-3.4.8# nc localhost 2181
> conf
> clientPort=2181
> dataDir=/root/amantas/zookeeper-3.4.8/data/version-2
> dataLogDir=/root/amantas/zookeeper-3.4.8/data/version-2
> tickTime=2000
> maxClientCnxns=60
> minSessionTimeout=4000
> maxSessionTimeout=40000
> serverId=0
>
> What am I doing wrong?
>
> Thanks in advance!
>