You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by David Bosschaert <da...@gmail.com> on 2012/05/25 23:23:15 UTC

Using zookeeper in a hightly restricted (cloud) environment

Hi,

I'm trying to use zookeeper in a cloud environment where I have an
extremely limited set of server ports available.
The setup is as follows. I want to run the ZooKeeper server inside my
cloud node so that it can be accessed from outside. In the node I can
open a Server Socket on an IP that looks like this 127.3.241.1:8080.
Port 8080 is then only Server Port I can open, no other ServerSockets
are allowed.
>From the outside this gets translated into some host.somedomain.com:80
(port 80!).

The zkServer starts up fairly happy (I set
clientPortAddress=127.3.241.1 and clientPort=8080) - see the log
below.

However, when I start the client console
$ bin/zkCli.sh -server host.somedomain.com:80
it does not connect properly, and 'ls /' just hangs. I've also pasted
the log below

My question is: is it possible to use ZooKeeper in such a restricted
environment at all? If so how should I configure it?

Thanks,

David

Here's the server log:
remote: JMX disabled by user request
remote: Using config:
/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
remote: 2012-05-25 11:11:54,642 [myid:] - INFO
[main:QuorumPeerConfig@101] - Reading configuration from:
/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
remote: 2012-05-25 11:11:54,735 [myid:] - INFO
[main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
remote: 2012-05-25 11:11:54,736 [myid:] - INFO
[main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
remote: 2012-05-25 11:11:54,737 [myid:] - INFO
[main:DatadirCleanupManager@101] - Purge task is not scheduled.
remote: 2012-05-25 11:11:54,738 [myid:] - WARN
[main:QuorumPeerMain@113] - Either no config or no quorum defined in
config, running  in standalone mode
remote: 2012-05-25 11:11:55,053 [myid:] - INFO
[main:QuorumPeerConfig@101] - Reading configuration from:
/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
remote: 2012-05-25 11:11:55,054 [myid:] - INFO
[main:ZooKeeperServerMain@95] - Starting server
remote: 2012-05-25 11:11:55,142 [myid:] - INFO  [main:Environment@98]
- Server environment:zookeeper.version=3.4.3-1240972, built on
02/06/2012 10:48 GMT
remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
- Server environment:host.name=ex-std-node68.prod.rhcloud.com
remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
- Server environment:java.version=1.6.0_22
remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
- Server environment:java.vendor=Sun Microsystems Inc.
remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
- Server environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre
remote: 2012-05-25 11:11:55,145 [myid:] - INFO  [main:Environment@98]
- Server environment:java.class.path=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf:
remote: 2012-05-25 11:11:55,146 [myid:] - INFO  [main:Environment@98]
- Server environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
remote: 2012-05-25 11:11:55,147 [myid:] - INFO  [main:Environment@98]
- Server environment:java.io.tmpdir=/tmp
remote: 2012-05-25 11:11:55,151 [myid:] - INFO  [main:Environment@98]
- Server environment:java.compiler=<NA>
remote: 2012-05-25 11:11:55,152 [myid:] - INFO  [main:Environment@98]
- Server environment:os.name=Linux
remote: 2012-05-25 11:11:55,153 [myid:] - INFO  [main:Environment@98]
- Server environment:os.arch=i386
remote: 2012-05-25 11:11:55,154 [myid:] - INFO  [main:Environment@98]
- Server environment:os.version=2.6.32-220.17.1.el6.x86_64
remote: 2012-05-25 11:11:55,155 [myid:] - INFO  [main:Environment@98]
- Server environment:user.name=80d2b6413ce14d3eb25c86e33f69c275
remote: 2012-05-25 11:11:55,230 [myid:] - INFO  [main:Environment@98]
- Server environment:user.home=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275
remote: 2012-05-25 11:11:55,231 [myid:] - INFO  [main:Environment@98]
- Server environment:user.dir=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3
remote: 2012-05-25 11:11:55,241 [myid:] - INFO
[main:ZooKeeperServer@733] - tickTime set to 2000
remote: 2012-05-25 11:11:55,242 [myid:] - INFO
[main:ZooKeeperServer@742] - minSessionTimeout set to -1
remote: 2012-05-25 11:11:55,242 [myid:] - INFO
[main:ZooKeeperServer@751] - maxSessionTimeout set to -1
remote: 2012-05-25 11:11:55,345 [myid:] - INFO
[main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080
remote: 2012-05-25 11:11:55,443 [myid:] - INFO
[main:FileTxnSnapLog@238] - Snapshotting: 0x0 to
./zk_data/version-2/snapshot.0

The client log:
$ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80
Connecting to zks1-davidosgi.rhcloud.com:80
2012-05-25 22:16:29,123 [myid:] - INFO  [main:Environment@98] - Client
environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
GMT
2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
environment:host.name=192.168.1.2
2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
environment:java.version=1.6.0_31
2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
environment:java.vendor=Apple Inc.
2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
2012-05-25 22:16:29,131 [myid:] - INFO  [main:Environment@98] - Client
environment:java.class.path=/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../conf:
2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
environment:java.io.tmpdir=/var/folders/s9/8mc72j514lgb0mj11czx585m0000gq/T/
2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
environment:java.compiler=<NA>
2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
environment:os.name=Mac OS X
2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
environment:os.arch=x86_64
2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
environment:os.version=10.7.4
2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
environment:user.name=david
2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
environment:user.home=/Users/david
2012-05-25 22:16:29,138 [myid:] - INFO  [main:Environment@98] - Client
environment:user.dir=/Users/david/proj/zk-test/zookeeper-3.4.3
2012-05-25 22:16:29,139 [myid:] - INFO  [main:ZooKeeper@433] -
Initiating client connection,
connectString=zks1-davidosgi.rhcloud.com:80 sessionTimeout=30000
watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3
Welcome to ZooKeeper!
2012-05-25 22:16:29,582 [myid:] - INFO
[main-SendThread():ClientCnxn$SendThread@933] - Opening socket
connection to server /23.22.86.141:80
2012-05-25 22:16:29,591 [myid:] - WARN
[main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@123]
- SecurityException: java.lang.SecurityException: Unable to locate a
login configuration occurred when trying to find JAAS configuration.
2012-05-25 22:16:29,591 [myid:] - INFO
[main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@125]
- Client will not SASL-authenticate because the default JAAS
configuration section 'Client' could not be found. If you are not
using SASL, you may ignore this. On the other hand, if you expected
SASL to work, please fix your JAAS configuration.
JLine support is enabled
[zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25
22:16:29,735 [myid:] - INFO
[main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@846]
- Socket connection established to
ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating
session

[zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls /
2012-05-25 22:16:59,733 [myid:] - INFO
[main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@1051]
- Client session timed out, have not heard from server in 30000ms for
sessionid 0x0, closing socket connection and attempting reconnect
Exception in thread "main"
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
	at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717)
	at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:593)
	at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:365)
	at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
	at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server
zks1-davidosgi.rhcloud.com:80

Re: Using zookeeper in a hightly restricted (cloud) environment

Posted by David Bosschaert <da...@gmail.com>.
Just to follow up, the blog that uses ZooKeeper as a discovery
component for an OSGi Cloud domain is now here:
http://coderthoughts.blogspot.com/2012/06/cloud-ecosystems-with-osgi.html
It doesn't go into much detail around setting up ZooKeeper, that's all
encapsulated in here:
https://github.com/bosschaert/osgi-cloud-discovery

Not sure whether this is suitable for a ZooKeeper FAQ, but I thought
I'd share it anyway.

Best regards,

David

On 11 June 2012 20:47, David Bosschaert <da...@gmail.com> wrote:
> Hi Patrick,
>
> This was in relation to getting it to work on RedHat OpenShift. I'll
> write a blog article about it in the not too distant future. I'll ping
> back with the link when that's done. We could then possibly put the
> link into one of the ZK pages...
>
> Best regards,
>
> David
>
> On 8 June 2012 23:15, Patrick Hunt <ph...@apache.org> wrote:
>> Sweet. Perhaps you could update the FAQ or troubleshooting guide with
>> your findings (ie the problem and how you debugged/fixed it), if you
>> think it's appropriate?
>>
>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting
>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/FAQ
>>
>> Patrick
>>
>> On Thu, Jun 7, 2012 at 2:17 PM, David Bosschaert
>> <da...@gmail.com> wrote:
>>> Thanks for confirming this Patrick, this helped me isolate the problem
>>> I was having.
>>> It turned out that the problem was outside of Zookeeper and I have now
>>> successfully been able to use Zookeeper in my environment.
>>>
>>> Thanks again,
>>>
>>> David
>>>
>>> On 30 May 2012 17:45, Patrick Hunt <ph...@apache.org> wrote:
>>>> This should work fine. Notice that while the connection is established
>>>> the session is never established, eventually it times out.
>>>>
>>>> Have you tried running a 4 letter word and see if that at least works?
>>>> http://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_zkCommands
>>>> (try 'stat' for example)
>>>>
>>>> Was there anything further in the server log after the client
>>>> attempted to connect? There should have been a message that the client
>>>> was attempting to connect, if this is not the case then perhaps the
>>>> firewall is blocking. Try the 4lw or using telnet to verify that your
>>>> client can get past the firewall to the socket bound by the server.
>>>>
>>>> Patrick
>>>>
>>>> On Fri, May 25, 2012 at 2:23 PM, David Bosschaert
>>>> <da...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I'm trying to use zookeeper in a cloud environment where I have an
>>>>> extremely limited set of server ports available.
>>>>> The setup is as follows. I want to run the ZooKeeper server inside my
>>>>> cloud node so that it can be accessed from outside. In the node I can
>>>>> open a Server Socket on an IP that looks like this 127.3.241.1:8080.
>>>>> Port 8080 is then only Server Port I can open, no other ServerSockets
>>>>> are allowed.
>>>>> From the outside this gets translated into some host.somedomain.com:80
>>>>> (port 80!).
>>>>>
>>>>> The zkServer starts up fairly happy (I set
>>>>> clientPortAddress=127.3.241.1 and clientPort=8080) - see the log
>>>>> below.
>>>>>
>>>>> However, when I start the client console
>>>>> $ bin/zkCli.sh -server host.somedomain.com:80
>>>>> it does not connect properly, and 'ls /' just hangs. I've also pasted
>>>>> the log below
>>>>>
>>>>> My question is: is it possible to use ZooKeeper in such a restricted
>>>>> environment at all? If so how should I configure it?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> David
>>>>>
>>>>> Here's the server log:
>>>>> remote: JMX disabled by user request
>>>>> remote: Using config:
>>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>>> remote: 2012-05-25 11:11:54,642 [myid:] - INFO
>>>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>>> remote: 2012-05-25 11:11:54,735 [myid:] - INFO
>>>>> [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
>>>>> remote: 2012-05-25 11:11:54,736 [myid:] - INFO
>>>>> [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
>>>>> remote: 2012-05-25 11:11:54,737 [myid:] - INFO
>>>>> [main:DatadirCleanupManager@101] - Purge task is not scheduled.
>>>>> remote: 2012-05-25 11:11:54,738 [myid:] - WARN
>>>>> [main:QuorumPeerMain@113] - Either no config or no quorum defined in
>>>>> config, running  in standalone mode
>>>>> remote: 2012-05-25 11:11:55,053 [myid:] - INFO
>>>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>>> remote: 2012-05-25 11:11:55,054 [myid:] - INFO
>>>>> [main:ZooKeeperServerMain@95] - Starting server
>>>>> remote: 2012-05-25 11:11:55,142 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:zookeeper.version=3.4.3-1240972, built on
>>>>> 02/06/2012 10:48 GMT
>>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:host.name=ex-std-node68.prod.rhcloud.com
>>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:java.version=1.6.0_22
>>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:java.vendor=Sun Microsystems Inc.
>>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre
>>>>> remote: 2012-05-25 11:11:55,145 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:java.class.path=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf:
>>>>> remote: 2012-05-25 11:11:55,146 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
>>>>> remote: 2012-05-25 11:11:55,147 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:java.io.tmpdir=/tmp
>>>>> remote: 2012-05-25 11:11:55,151 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:java.compiler=<NA>
>>>>> remote: 2012-05-25 11:11:55,152 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:os.name=Linux
>>>>> remote: 2012-05-25 11:11:55,153 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:os.arch=i386
>>>>> remote: 2012-05-25 11:11:55,154 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:os.version=2.6.32-220.17.1.el6.x86_64
>>>>> remote: 2012-05-25 11:11:55,155 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:user.name=80d2b6413ce14d3eb25c86e33f69c275
>>>>> remote: 2012-05-25 11:11:55,230 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:user.home=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275
>>>>> remote: 2012-05-25 11:11:55,231 [myid:] - INFO  [main:Environment@98]
>>>>> - Server environment:user.dir=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3
>>>>> remote: 2012-05-25 11:11:55,241 [myid:] - INFO
>>>>> [main:ZooKeeperServer@733] - tickTime set to 2000
>>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>>>> [main:ZooKeeperServer@742] - minSessionTimeout set to -1
>>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>>>> [main:ZooKeeperServer@751] - maxSessionTimeout set to -1
>>>>> remote: 2012-05-25 11:11:55,345 [myid:] - INFO
>>>>> [main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080
>>>>> remote: 2012-05-25 11:11:55,443 [myid:] - INFO
>>>>> [main:FileTxnSnapLog@238] - Snapshotting: 0x0 to
>>>>> ./zk_data/version-2/snapshot.0
>>>>>
>>>>> The client log:
>>>>> $ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80
>>>>> Connecting to zks1-davidosgi.rhcloud.com:80
>>>>> 2012-05-25 22:16:29,123 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
>>>>> GMT
>>>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:host.name=192.168.1.2
>>>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:java.version=1.6.0_31
>>>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:java.vendor=Apple Inc.
>>>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
>>>>> 2012-05-25 22:16:29,131 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:java.class.path=/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../conf:
>>>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
>>>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:java.io.tmpdir=/var/folders/s9/8mc72j514lgb0mj11czx585m0000gq/T/
>>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:java.compiler=<NA>
>>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:os.name=Mac OS X
>>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:os.arch=x86_64
>>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:os.version=10.7.4
>>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:user.name=david
>>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:user.home=/Users/david
>>>>> 2012-05-25 22:16:29,138 [myid:] - INFO  [main:Environment@98] - Client
>>>>> environment:user.dir=/Users/david/proj/zk-test/zookeeper-3.4.3
>>>>> 2012-05-25 22:16:29,139 [myid:] - INFO  [main:ZooKeeper@433] -
>>>>> Initiating client connection,
>>>>> connectString=zks1-davidosgi.rhcloud.com:80 sessionTimeout=30000
>>>>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3
>>>>> Welcome to ZooKeeper!
>>>>> 2012-05-25 22:16:29,582 [myid:] - INFO
>>>>> [main-SendThread():ClientCnxn$SendThread@933] - Opening socket
>>>>> connection to server /23.22.86.141:80
>>>>> 2012-05-25 22:16:29,591 [myid:] - WARN
>>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@123]
>>>>> - SecurityException: java.lang.SecurityException: Unable to locate a
>>>>> login configuration occurred when trying to find JAAS configuration.
>>>>> 2012-05-25 22:16:29,591 [myid:] - INFO
>>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@125]
>>>>> - Client will not SASL-authenticate because the default JAAS
>>>>> configuration section 'Client' could not be found. If you are not
>>>>> using SASL, you may ignore this. On the other hand, if you expected
>>>>> SASL to work, please fix your JAAS configuration.
>>>>> JLine support is enabled
>>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25
>>>>> 22:16:29,735 [myid:] - INFO
>>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@846]
>>>>> - Socket connection established to
>>>>> ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating
>>>>> session
>>>>>
>>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls /
>>>>> 2012-05-25 22:16:59,733 [myid:] - INFO
>>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@1051]
>>>>> - Client session timed out, have not heard from server in 30000ms for
>>>>> sessionid 0x0, closing socket connection and attempting reconnect
>>>>> Exception in thread "main"
>>>>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>>>>> KeeperErrorCode = ConnectionLoss for /
>>>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>>>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>>>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
>>>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
>>>>>        at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717)
>>>>>        at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:593)
>>>>>        at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:365)
>>>>>        at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
>>>>>        at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
>>>>> david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server
>>>>> zks1-davidosgi.rhcloud.com:80

Re: Using zookeeper in a hightly restricted (cloud) environment

Posted by David Bosschaert <da...@gmail.com>.
Hi Patrick,

This was in relation to getting it to work on RedHat OpenShift. I'll
write a blog article about it in the not too distant future. I'll ping
back with the link when that's done. We could then possibly put the
link into one of the ZK pages...

Best regards,

David

On 8 June 2012 23:15, Patrick Hunt <ph...@apache.org> wrote:
> Sweet. Perhaps you could update the FAQ or troubleshooting guide with
> your findings (ie the problem and how you debugged/fixed it), if you
> think it's appropriate?
>
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/FAQ
>
> Patrick
>
> On Thu, Jun 7, 2012 at 2:17 PM, David Bosschaert
> <da...@gmail.com> wrote:
>> Thanks for confirming this Patrick, this helped me isolate the problem
>> I was having.
>> It turned out that the problem was outside of Zookeeper and I have now
>> successfully been able to use Zookeeper in my environment.
>>
>> Thanks again,
>>
>> David
>>
>> On 30 May 2012 17:45, Patrick Hunt <ph...@apache.org> wrote:
>>> This should work fine. Notice that while the connection is established
>>> the session is never established, eventually it times out.
>>>
>>> Have you tried running a 4 letter word and see if that at least works?
>>> http://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_zkCommands
>>> (try 'stat' for example)
>>>
>>> Was there anything further in the server log after the client
>>> attempted to connect? There should have been a message that the client
>>> was attempting to connect, if this is not the case then perhaps the
>>> firewall is blocking. Try the 4lw or using telnet to verify that your
>>> client can get past the firewall to the socket bound by the server.
>>>
>>> Patrick
>>>
>>> On Fri, May 25, 2012 at 2:23 PM, David Bosschaert
>>> <da...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I'm trying to use zookeeper in a cloud environment where I have an
>>>> extremely limited set of server ports available.
>>>> The setup is as follows. I want to run the ZooKeeper server inside my
>>>> cloud node so that it can be accessed from outside. In the node I can
>>>> open a Server Socket on an IP that looks like this 127.3.241.1:8080.
>>>> Port 8080 is then only Server Port I can open, no other ServerSockets
>>>> are allowed.
>>>> From the outside this gets translated into some host.somedomain.com:80
>>>> (port 80!).
>>>>
>>>> The zkServer starts up fairly happy (I set
>>>> clientPortAddress=127.3.241.1 and clientPort=8080) - see the log
>>>> below.
>>>>
>>>> However, when I start the client console
>>>> $ bin/zkCli.sh -server host.somedomain.com:80
>>>> it does not connect properly, and 'ls /' just hangs. I've also pasted
>>>> the log below
>>>>
>>>> My question is: is it possible to use ZooKeeper in such a restricted
>>>> environment at all? If so how should I configure it?
>>>>
>>>> Thanks,
>>>>
>>>> David
>>>>
>>>> Here's the server log:
>>>> remote: JMX disabled by user request
>>>> remote: Using config:
>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>> remote: 2012-05-25 11:11:54,642 [myid:] - INFO
>>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>> remote: 2012-05-25 11:11:54,735 [myid:] - INFO
>>>> [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
>>>> remote: 2012-05-25 11:11:54,736 [myid:] - INFO
>>>> [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
>>>> remote: 2012-05-25 11:11:54,737 [myid:] - INFO
>>>> [main:DatadirCleanupManager@101] - Purge task is not scheduled.
>>>> remote: 2012-05-25 11:11:54,738 [myid:] - WARN
>>>> [main:QuorumPeerMain@113] - Either no config or no quorum defined in
>>>> config, running  in standalone mode
>>>> remote: 2012-05-25 11:11:55,053 [myid:] - INFO
>>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>>> remote: 2012-05-25 11:11:55,054 [myid:] - INFO
>>>> [main:ZooKeeperServerMain@95] - Starting server
>>>> remote: 2012-05-25 11:11:55,142 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:zookeeper.version=3.4.3-1240972, built on
>>>> 02/06/2012 10:48 GMT
>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:host.name=ex-std-node68.prod.rhcloud.com
>>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.version=1.6.0_22
>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.vendor=Sun Microsystems Inc.
>>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre
>>>> remote: 2012-05-25 11:11:55,145 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.class.path=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf:
>>>> remote: 2012-05-25 11:11:55,146 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
>>>> remote: 2012-05-25 11:11:55,147 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.io.tmpdir=/tmp
>>>> remote: 2012-05-25 11:11:55,151 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:java.compiler=<NA>
>>>> remote: 2012-05-25 11:11:55,152 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:os.name=Linux
>>>> remote: 2012-05-25 11:11:55,153 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:os.arch=i386
>>>> remote: 2012-05-25 11:11:55,154 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:os.version=2.6.32-220.17.1.el6.x86_64
>>>> remote: 2012-05-25 11:11:55,155 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:user.name=80d2b6413ce14d3eb25c86e33f69c275
>>>> remote: 2012-05-25 11:11:55,230 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:user.home=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275
>>>> remote: 2012-05-25 11:11:55,231 [myid:] - INFO  [main:Environment@98]
>>>> - Server environment:user.dir=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3
>>>> remote: 2012-05-25 11:11:55,241 [myid:] - INFO
>>>> [main:ZooKeeperServer@733] - tickTime set to 2000
>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>>> [main:ZooKeeperServer@742] - minSessionTimeout set to -1
>>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>>> [main:ZooKeeperServer@751] - maxSessionTimeout set to -1
>>>> remote: 2012-05-25 11:11:55,345 [myid:] - INFO
>>>> [main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080
>>>> remote: 2012-05-25 11:11:55,443 [myid:] - INFO
>>>> [main:FileTxnSnapLog@238] - Snapshotting: 0x0 to
>>>> ./zk_data/version-2/snapshot.0
>>>>
>>>> The client log:
>>>> $ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80
>>>> Connecting to zks1-davidosgi.rhcloud.com:80
>>>> 2012-05-25 22:16:29,123 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
>>>> GMT
>>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:host.name=192.168.1.2
>>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.version=1.6.0_31
>>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.vendor=Apple Inc.
>>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
>>>> 2012-05-25 22:16:29,131 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.class.path=/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../conf:
>>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
>>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.io.tmpdir=/var/folders/s9/8mc72j514lgb0mj11czx585m0000gq/T/
>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:java.compiler=<NA>
>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:os.name=Mac OS X
>>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:os.arch=x86_64
>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:os.version=10.7.4
>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:user.name=david
>>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:user.home=/Users/david
>>>> 2012-05-25 22:16:29,138 [myid:] - INFO  [main:Environment@98] - Client
>>>> environment:user.dir=/Users/david/proj/zk-test/zookeeper-3.4.3
>>>> 2012-05-25 22:16:29,139 [myid:] - INFO  [main:ZooKeeper@433] -
>>>> Initiating client connection,
>>>> connectString=zks1-davidosgi.rhcloud.com:80 sessionTimeout=30000
>>>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3
>>>> Welcome to ZooKeeper!
>>>> 2012-05-25 22:16:29,582 [myid:] - INFO
>>>> [main-SendThread():ClientCnxn$SendThread@933] - Opening socket
>>>> connection to server /23.22.86.141:80
>>>> 2012-05-25 22:16:29,591 [myid:] - WARN
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@123]
>>>> - SecurityException: java.lang.SecurityException: Unable to locate a
>>>> login configuration occurred when trying to find JAAS configuration.
>>>> 2012-05-25 22:16:29,591 [myid:] - INFO
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@125]
>>>> - Client will not SASL-authenticate because the default JAAS
>>>> configuration section 'Client' could not be found. If you are not
>>>> using SASL, you may ignore this. On the other hand, if you expected
>>>> SASL to work, please fix your JAAS configuration.
>>>> JLine support is enabled
>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25
>>>> 22:16:29,735 [myid:] - INFO
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@846]
>>>> - Socket connection established to
>>>> ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating
>>>> session
>>>>
>>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls /
>>>> 2012-05-25 22:16:59,733 [myid:] - INFO
>>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@1051]
>>>> - Client session timed out, have not heard from server in 30000ms for
>>>> sessionid 0x0, closing socket connection and attempting reconnect
>>>> Exception in thread "main"
>>>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>>>> KeeperErrorCode = ConnectionLoss for /
>>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
>>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
>>>>        at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717)
>>>>        at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:593)
>>>>        at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:365)
>>>>        at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
>>>>        at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
>>>> david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server
>>>> zks1-davidosgi.rhcloud.com:80

Re: Using zookeeper in a hightly restricted (cloud) environment

Posted by Patrick Hunt <ph...@apache.org>.
Sweet. Perhaps you could update the FAQ or troubleshooting guide with
your findings (ie the problem and how you debugged/fixed it), if you
think it's appropriate?

https://cwiki.apache.org/confluence/display/ZOOKEEPER/Troubleshooting
https://cwiki.apache.org/confluence/display/ZOOKEEPER/FAQ

Patrick

On Thu, Jun 7, 2012 at 2:17 PM, David Bosschaert
<da...@gmail.com> wrote:
> Thanks for confirming this Patrick, this helped me isolate the problem
> I was having.
> It turned out that the problem was outside of Zookeeper and I have now
> successfully been able to use Zookeeper in my environment.
>
> Thanks again,
>
> David
>
> On 30 May 2012 17:45, Patrick Hunt <ph...@apache.org> wrote:
>> This should work fine. Notice that while the connection is established
>> the session is never established, eventually it times out.
>>
>> Have you tried running a 4 letter word and see if that at least works?
>> http://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_zkCommands
>> (try 'stat' for example)
>>
>> Was there anything further in the server log after the client
>> attempted to connect? There should have been a message that the client
>> was attempting to connect, if this is not the case then perhaps the
>> firewall is blocking. Try the 4lw or using telnet to verify that your
>> client can get past the firewall to the socket bound by the server.
>>
>> Patrick
>>
>> On Fri, May 25, 2012 at 2:23 PM, David Bosschaert
>> <da...@gmail.com> wrote:
>>> Hi,
>>>
>>> I'm trying to use zookeeper in a cloud environment where I have an
>>> extremely limited set of server ports available.
>>> The setup is as follows. I want to run the ZooKeeper server inside my
>>> cloud node so that it can be accessed from outside. In the node I can
>>> open a Server Socket on an IP that looks like this 127.3.241.1:8080.
>>> Port 8080 is then only Server Port I can open, no other ServerSockets
>>> are allowed.
>>> From the outside this gets translated into some host.somedomain.com:80
>>> (port 80!).
>>>
>>> The zkServer starts up fairly happy (I set
>>> clientPortAddress=127.3.241.1 and clientPort=8080) - see the log
>>> below.
>>>
>>> However, when I start the client console
>>> $ bin/zkCli.sh -server host.somedomain.com:80
>>> it does not connect properly, and 'ls /' just hangs. I've also pasted
>>> the log below
>>>
>>> My question is: is it possible to use ZooKeeper in such a restricted
>>> environment at all? If so how should I configure it?
>>>
>>> Thanks,
>>>
>>> David
>>>
>>> Here's the server log:
>>> remote: JMX disabled by user request
>>> remote: Using config:
>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>> remote: 2012-05-25 11:11:54,642 [myid:] - INFO
>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>> remote: 2012-05-25 11:11:54,735 [myid:] - INFO
>>> [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
>>> remote: 2012-05-25 11:11:54,736 [myid:] - INFO
>>> [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
>>> remote: 2012-05-25 11:11:54,737 [myid:] - INFO
>>> [main:DatadirCleanupManager@101] - Purge task is not scheduled.
>>> remote: 2012-05-25 11:11:54,738 [myid:] - WARN
>>> [main:QuorumPeerMain@113] - Either no config or no quorum defined in
>>> config, running  in standalone mode
>>> remote: 2012-05-25 11:11:55,053 [myid:] - INFO
>>> [main:QuorumPeerConfig@101] - Reading configuration from:
>>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>>> remote: 2012-05-25 11:11:55,054 [myid:] - INFO
>>> [main:ZooKeeperServerMain@95] - Starting server
>>> remote: 2012-05-25 11:11:55,142 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:zookeeper.version=3.4.3-1240972, built on
>>> 02/06/2012 10:48 GMT
>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:host.name=ex-std-node68.prod.rhcloud.com
>>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:java.version=1.6.0_22
>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:java.vendor=Sun Microsystems Inc.
>>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre
>>> remote: 2012-05-25 11:11:55,145 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:java.class.path=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf:
>>> remote: 2012-05-25 11:11:55,146 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
>>> remote: 2012-05-25 11:11:55,147 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:java.io.tmpdir=/tmp
>>> remote: 2012-05-25 11:11:55,151 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:java.compiler=<NA>
>>> remote: 2012-05-25 11:11:55,152 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:os.name=Linux
>>> remote: 2012-05-25 11:11:55,153 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:os.arch=i386
>>> remote: 2012-05-25 11:11:55,154 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:os.version=2.6.32-220.17.1.el6.x86_64
>>> remote: 2012-05-25 11:11:55,155 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:user.name=80d2b6413ce14d3eb25c86e33f69c275
>>> remote: 2012-05-25 11:11:55,230 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:user.home=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275
>>> remote: 2012-05-25 11:11:55,231 [myid:] - INFO  [main:Environment@98]
>>> - Server environment:user.dir=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3
>>> remote: 2012-05-25 11:11:55,241 [myid:] - INFO
>>> [main:ZooKeeperServer@733] - tickTime set to 2000
>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>> [main:ZooKeeperServer@742] - minSessionTimeout set to -1
>>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>>> [main:ZooKeeperServer@751] - maxSessionTimeout set to -1
>>> remote: 2012-05-25 11:11:55,345 [myid:] - INFO
>>> [main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080
>>> remote: 2012-05-25 11:11:55,443 [myid:] - INFO
>>> [main:FileTxnSnapLog@238] - Snapshotting: 0x0 to
>>> ./zk_data/version-2/snapshot.0
>>>
>>> The client log:
>>> $ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80
>>> Connecting to zks1-davidosgi.rhcloud.com:80
>>> 2012-05-25 22:16:29,123 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
>>> GMT
>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:host.name=192.168.1.2
>>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:java.version=1.6.0_31
>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:java.vendor=Apple Inc.
>>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
>>> 2012-05-25 22:16:29,131 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:java.class.path=/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../conf:
>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
>>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:java.io.tmpdir=/var/folders/s9/8mc72j514lgb0mj11czx585m0000gq/T/
>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:java.compiler=<NA>
>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:os.name=Mac OS X
>>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:os.arch=x86_64
>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:os.version=10.7.4
>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:user.name=david
>>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:user.home=/Users/david
>>> 2012-05-25 22:16:29,138 [myid:] - INFO  [main:Environment@98] - Client
>>> environment:user.dir=/Users/david/proj/zk-test/zookeeper-3.4.3
>>> 2012-05-25 22:16:29,139 [myid:] - INFO  [main:ZooKeeper@433] -
>>> Initiating client connection,
>>> connectString=zks1-davidosgi.rhcloud.com:80 sessionTimeout=30000
>>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3
>>> Welcome to ZooKeeper!
>>> 2012-05-25 22:16:29,582 [myid:] - INFO
>>> [main-SendThread():ClientCnxn$SendThread@933] - Opening socket
>>> connection to server /23.22.86.141:80
>>> 2012-05-25 22:16:29,591 [myid:] - WARN
>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@123]
>>> - SecurityException: java.lang.SecurityException: Unable to locate a
>>> login configuration occurred when trying to find JAAS configuration.
>>> 2012-05-25 22:16:29,591 [myid:] - INFO
>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@125]
>>> - Client will not SASL-authenticate because the default JAAS
>>> configuration section 'Client' could not be found. If you are not
>>> using SASL, you may ignore this. On the other hand, if you expected
>>> SASL to work, please fix your JAAS configuration.
>>> JLine support is enabled
>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25
>>> 22:16:29,735 [myid:] - INFO
>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@846]
>>> - Socket connection established to
>>> ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating
>>> session
>>>
>>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls /
>>> 2012-05-25 22:16:59,733 [myid:] - INFO
>>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@1051]
>>> - Client session timed out, have not heard from server in 30000ms for
>>> sessionid 0x0, closing socket connection and attempting reconnect
>>> Exception in thread "main"
>>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>>> KeeperErrorCode = ConnectionLoss for /
>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
>>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
>>>        at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717)
>>>        at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:593)
>>>        at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:365)
>>>        at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
>>>        at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
>>> david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server
>>> zks1-davidosgi.rhcloud.com:80

Re: Using zookeeper in a hightly restricted (cloud) environment

Posted by David Bosschaert <da...@gmail.com>.
Thanks for confirming this Patrick, this helped me isolate the problem
I was having.
It turned out that the problem was outside of Zookeeper and I have now
successfully been able to use Zookeeper in my environment.

Thanks again,

David

On 30 May 2012 17:45, Patrick Hunt <ph...@apache.org> wrote:
> This should work fine. Notice that while the connection is established
> the session is never established, eventually it times out.
>
> Have you tried running a 4 letter word and see if that at least works?
> http://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_zkCommands
> (try 'stat' for example)
>
> Was there anything further in the server log after the client
> attempted to connect? There should have been a message that the client
> was attempting to connect, if this is not the case then perhaps the
> firewall is blocking. Try the 4lw or using telnet to verify that your
> client can get past the firewall to the socket bound by the server.
>
> Patrick
>
> On Fri, May 25, 2012 at 2:23 PM, David Bosschaert
> <da...@gmail.com> wrote:
>> Hi,
>>
>> I'm trying to use zookeeper in a cloud environment where I have an
>> extremely limited set of server ports available.
>> The setup is as follows. I want to run the ZooKeeper server inside my
>> cloud node so that it can be accessed from outside. In the node I can
>> open a Server Socket on an IP that looks like this 127.3.241.1:8080.
>> Port 8080 is then only Server Port I can open, no other ServerSockets
>> are allowed.
>> From the outside this gets translated into some host.somedomain.com:80
>> (port 80!).
>>
>> The zkServer starts up fairly happy (I set
>> clientPortAddress=127.3.241.1 and clientPort=8080) - see the log
>> below.
>>
>> However, when I start the client console
>> $ bin/zkCli.sh -server host.somedomain.com:80
>> it does not connect properly, and 'ls /' just hangs. I've also pasted
>> the log below
>>
>> My question is: is it possible to use ZooKeeper in such a restricted
>> environment at all? If so how should I configure it?
>>
>> Thanks,
>>
>> David
>>
>> Here's the server log:
>> remote: JMX disabled by user request
>> remote: Using config:
>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>> remote: 2012-05-25 11:11:54,642 [myid:] - INFO
>> [main:QuorumPeerConfig@101] - Reading configuration from:
>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>> remote: 2012-05-25 11:11:54,735 [myid:] - INFO
>> [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
>> remote: 2012-05-25 11:11:54,736 [myid:] - INFO
>> [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
>> remote: 2012-05-25 11:11:54,737 [myid:] - INFO
>> [main:DatadirCleanupManager@101] - Purge task is not scheduled.
>> remote: 2012-05-25 11:11:54,738 [myid:] - WARN
>> [main:QuorumPeerMain@113] - Either no config or no quorum defined in
>> config, running  in standalone mode
>> remote: 2012-05-25 11:11:55,053 [myid:] - INFO
>> [main:QuorumPeerConfig@101] - Reading configuration from:
>> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
>> remote: 2012-05-25 11:11:55,054 [myid:] - INFO
>> [main:ZooKeeperServerMain@95] - Starting server
>> remote: 2012-05-25 11:11:55,142 [myid:] - INFO  [main:Environment@98]
>> - Server environment:zookeeper.version=3.4.3-1240972, built on
>> 02/06/2012 10:48 GMT
>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>> - Server environment:host.name=ex-std-node68.prod.rhcloud.com
>> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
>> - Server environment:java.version=1.6.0_22
>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>> - Server environment:java.vendor=Sun Microsystems Inc.
>> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
>> - Server environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre
>> remote: 2012-05-25 11:11:55,145 [myid:] - INFO  [main:Environment@98]
>> - Server environment:java.class.path=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf:
>> remote: 2012-05-25 11:11:55,146 [myid:] - INFO  [main:Environment@98]
>> - Server environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
>> remote: 2012-05-25 11:11:55,147 [myid:] - INFO  [main:Environment@98]
>> - Server environment:java.io.tmpdir=/tmp
>> remote: 2012-05-25 11:11:55,151 [myid:] - INFO  [main:Environment@98]
>> - Server environment:java.compiler=<NA>
>> remote: 2012-05-25 11:11:55,152 [myid:] - INFO  [main:Environment@98]
>> - Server environment:os.name=Linux
>> remote: 2012-05-25 11:11:55,153 [myid:] - INFO  [main:Environment@98]
>> - Server environment:os.arch=i386
>> remote: 2012-05-25 11:11:55,154 [myid:] - INFO  [main:Environment@98]
>> - Server environment:os.version=2.6.32-220.17.1.el6.x86_64
>> remote: 2012-05-25 11:11:55,155 [myid:] - INFO  [main:Environment@98]
>> - Server environment:user.name=80d2b6413ce14d3eb25c86e33f69c275
>> remote: 2012-05-25 11:11:55,230 [myid:] - INFO  [main:Environment@98]
>> - Server environment:user.home=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275
>> remote: 2012-05-25 11:11:55,231 [myid:] - INFO  [main:Environment@98]
>> - Server environment:user.dir=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3
>> remote: 2012-05-25 11:11:55,241 [myid:] - INFO
>> [main:ZooKeeperServer@733] - tickTime set to 2000
>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>> [main:ZooKeeperServer@742] - minSessionTimeout set to -1
>> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
>> [main:ZooKeeperServer@751] - maxSessionTimeout set to -1
>> remote: 2012-05-25 11:11:55,345 [myid:] - INFO
>> [main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080
>> remote: 2012-05-25 11:11:55,443 [myid:] - INFO
>> [main:FileTxnSnapLog@238] - Snapshotting: 0x0 to
>> ./zk_data/version-2/snapshot.0
>>
>> The client log:
>> $ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80
>> Connecting to zks1-davidosgi.rhcloud.com:80
>> 2012-05-25 22:16:29,123 [myid:] - INFO  [main:Environment@98] - Client
>> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
>> GMT
>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>> environment:host.name=192.168.1.2
>> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
>> environment:java.version=1.6.0_31
>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>> environment:java.vendor=Apple Inc.
>> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
>> environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
>> 2012-05-25 22:16:29,131 [myid:] - INFO  [main:Environment@98] - Client
>> environment:java.class.path=/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../conf:
>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
>> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
>> environment:java.io.tmpdir=/var/folders/s9/8mc72j514lgb0mj11czx585m0000gq/T/
>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>> environment:java.compiler=<NA>
>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>> environment:os.name=Mac OS X
>> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
>> environment:os.arch=x86_64
>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>> environment:os.version=10.7.4
>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>> environment:user.name=david
>> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
>> environment:user.home=/Users/david
>> 2012-05-25 22:16:29,138 [myid:] - INFO  [main:Environment@98] - Client
>> environment:user.dir=/Users/david/proj/zk-test/zookeeper-3.4.3
>> 2012-05-25 22:16:29,139 [myid:] - INFO  [main:ZooKeeper@433] -
>> Initiating client connection,
>> connectString=zks1-davidosgi.rhcloud.com:80 sessionTimeout=30000
>> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3
>> Welcome to ZooKeeper!
>> 2012-05-25 22:16:29,582 [myid:] - INFO
>> [main-SendThread():ClientCnxn$SendThread@933] - Opening socket
>> connection to server /23.22.86.141:80
>> 2012-05-25 22:16:29,591 [myid:] - WARN
>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@123]
>> - SecurityException: java.lang.SecurityException: Unable to locate a
>> login configuration occurred when trying to find JAAS configuration.
>> 2012-05-25 22:16:29,591 [myid:] - INFO
>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@125]
>> - Client will not SASL-authenticate because the default JAAS
>> configuration section 'Client' could not be found. If you are not
>> using SASL, you may ignore this. On the other hand, if you expected
>> SASL to work, please fix your JAAS configuration.
>> JLine support is enabled
>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25
>> 22:16:29,735 [myid:] - INFO
>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@846]
>> - Socket connection established to
>> ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating
>> session
>>
>> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls /
>> 2012-05-25 22:16:59,733 [myid:] - INFO
>> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@1051]
>> - Client session timed out, have not heard from server in 30000ms for
>> sessionid 0x0, closing socket connection and attempting reconnect
>> Exception in thread "main"
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss for /
>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
>>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
>>        at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717)
>>        at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:593)
>>        at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:365)
>>        at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
>>        at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
>> david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server
>> zks1-davidosgi.rhcloud.com:80

Re: Using zookeeper in a hightly restricted (cloud) environment

Posted by Patrick Hunt <ph...@apache.org>.
This should work fine. Notice that while the connection is established
the session is never established, eventually it times out.

Have you tried running a 4 letter word and see if that at least works?
http://zookeeper.apache.org/doc/r3.4.3/zookeeperAdmin.html#sc_zkCommands
(try 'stat' for example)

Was there anything further in the server log after the client
attempted to connect? There should have been a message that the client
was attempting to connect, if this is not the case then perhaps the
firewall is blocking. Try the 4lw or using telnet to verify that your
client can get past the firewall to the socket bound by the server.

Patrick

On Fri, May 25, 2012 at 2:23 PM, David Bosschaert
<da...@gmail.com> wrote:
> Hi,
>
> I'm trying to use zookeeper in a cloud environment where I have an
> extremely limited set of server ports available.
> The setup is as follows. I want to run the ZooKeeper server inside my
> cloud node so that it can be accessed from outside. In the node I can
> open a Server Socket on an IP that looks like this 127.3.241.1:8080.
> Port 8080 is then only Server Port I can open, no other ServerSockets
> are allowed.
> From the outside this gets translated into some host.somedomain.com:80
> (port 80!).
>
> The zkServer starts up fairly happy (I set
> clientPortAddress=127.3.241.1 and clientPort=8080) - see the log
> below.
>
> However, when I start the client console
> $ bin/zkCli.sh -server host.somedomain.com:80
> it does not connect properly, and 'ls /' just hangs. I've also pasted
> the log below
>
> My question is: is it possible to use ZooKeeper in such a restricted
> environment at all? If so how should I configure it?
>
> Thanks,
>
> David
>
> Here's the server log:
> remote: JMX disabled by user request
> remote: Using config:
> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
> remote: 2012-05-25 11:11:54,642 [myid:] - INFO
> [main:QuorumPeerConfig@101] - Reading configuration from:
> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
> remote: 2012-05-25 11:11:54,735 [myid:] - INFO
> [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3
> remote: 2012-05-25 11:11:54,736 [myid:] - INFO
> [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0
> remote: 2012-05-25 11:11:54,737 [myid:] - INFO
> [main:DatadirCleanupManager@101] - Purge task is not scheduled.
> remote: 2012-05-25 11:11:54,738 [myid:] - WARN
> [main:QuorumPeerMain@113] - Either no config or no quorum defined in
> config, running  in standalone mode
> remote: 2012-05-25 11:11:55,053 [myid:] - INFO
> [main:QuorumPeerConfig@101] - Reading configuration from:
> /var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf/zoo.cfg
> remote: 2012-05-25 11:11:55,054 [myid:] - INFO
> [main:ZooKeeperServerMain@95] - Starting server
> remote: 2012-05-25 11:11:55,142 [myid:] - INFO  [main:Environment@98]
> - Server environment:zookeeper.version=3.4.3-1240972, built on
> 02/06/2012 10:48 GMT
> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
> - Server environment:host.name=ex-std-node68.prod.rhcloud.com
> remote: 2012-05-25 11:11:55,143 [myid:] - INFO  [main:Environment@98]
> - Server environment:java.version=1.6.0_22
> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
> - Server environment:java.vendor=Sun Microsystems Inc.
> remote: 2012-05-25 11:11:55,144 [myid:] - INFO  [main:Environment@98]
> - Server environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre
> remote: 2012-05-25 11:11:55,145 [myid:] - INFO  [main:Environment@98]
> - Server environment:java.class.path=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/classes:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../build/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/repo/zookeeper-3.4.3/bin/../conf:
> remote: 2012-05-25 11:11:55,146 [myid:] - INFO  [main:Environment@98]
> - Server environment:java.library.path=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/lib/i386:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
> remote: 2012-05-25 11:11:55,147 [myid:] - INFO  [main:Environment@98]
> - Server environment:java.io.tmpdir=/tmp
> remote: 2012-05-25 11:11:55,151 [myid:] - INFO  [main:Environment@98]
> - Server environment:java.compiler=<NA>
> remote: 2012-05-25 11:11:55,152 [myid:] - INFO  [main:Environment@98]
> - Server environment:os.name=Linux
> remote: 2012-05-25 11:11:55,153 [myid:] - INFO  [main:Environment@98]
> - Server environment:os.arch=i386
> remote: 2012-05-25 11:11:55,154 [myid:] - INFO  [main:Environment@98]
> - Server environment:os.version=2.6.32-220.17.1.el6.x86_64
> remote: 2012-05-25 11:11:55,155 [myid:] - INFO  [main:Environment@98]
> - Server environment:user.name=80d2b6413ce14d3eb25c86e33f69c275
> remote: 2012-05-25 11:11:55,230 [myid:] - INFO  [main:Environment@98]
> - Server environment:user.home=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275
> remote: 2012-05-25 11:11:55,231 [myid:] - INFO  [main:Environment@98]
> - Server environment:user.dir=/var/lib/stickshift/80d2b6413ce14d3eb25c86e33f69c275/zks1/runtime/repo/zookeeper-3.4.3
> remote: 2012-05-25 11:11:55,241 [myid:] - INFO
> [main:ZooKeeperServer@733] - tickTime set to 2000
> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
> [main:ZooKeeperServer@742] - minSessionTimeout set to -1
> remote: 2012-05-25 11:11:55,242 [myid:] - INFO
> [main:ZooKeeperServer@751] - maxSessionTimeout set to -1
> remote: 2012-05-25 11:11:55,345 [myid:] - INFO
> [main:NIOServerCnxnFactory@110] - binding to port /127.3.241.1:8080
> remote: 2012-05-25 11:11:55,443 [myid:] - INFO
> [main:FileTxnSnapLog@238] - Snapshotting: 0x0 to
> ./zk_data/version-2/snapshot.0
>
> The client log:
> $ bin/zkCli.sh -server zks1-davidosgi.rhcloud.com:80
> Connecting to zks1-davidosgi.rhcloud.com:80
> 2012-05-25 22:16:29,123 [myid:] - INFO  [main:Environment@98] - Client
> environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48
> GMT
> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
> environment:host.name=192.168.1.2
> 2012-05-25 22:16:29,129 [myid:] - INFO  [main:Environment@98] - Client
> environment:java.version=1.6.0_31
> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
> environment:java.vendor=Apple Inc.
> 2012-05-25 22:16:29,130 [myid:] - INFO  [main:Environment@98] - Client
> environment:java.home=/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
> 2012-05-25 22:16:29,131 [myid:] - INFO  [main:Environment@98] - Client
> environment:java.class.path=/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/classes:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../build/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-log4j12-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/slf4j-api-1.6.1.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/netty-3.2.2.Final.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/log4j-1.2.15.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../lib/jline-0.9.94.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../zookeeper-3.4.3.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../src/java/lib/*.jar:/Users/david/proj/zk-test/zookeeper-3.4.3/bin/../conf:
> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
> environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
> 2012-05-25 22:16:29,135 [myid:] - INFO  [main:Environment@98] - Client
> environment:java.io.tmpdir=/var/folders/s9/8mc72j514lgb0mj11czx585m0000gq/T/
> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
> environment:java.compiler=<NA>
> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
> environment:os.name=Mac OS X
> 2012-05-25 22:16:29,136 [myid:] - INFO  [main:Environment@98] - Client
> environment:os.arch=x86_64
> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
> environment:os.version=10.7.4
> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
> environment:user.name=david
> 2012-05-25 22:16:29,137 [myid:] - INFO  [main:Environment@98] - Client
> environment:user.home=/Users/david
> 2012-05-25 22:16:29,138 [myid:] - INFO  [main:Environment@98] - Client
> environment:user.dir=/Users/david/proj/zk-test/zookeeper-3.4.3
> 2012-05-25 22:16:29,139 [myid:] - INFO  [main:ZooKeeper@433] -
> Initiating client connection,
> connectString=zks1-davidosgi.rhcloud.com:80 sessionTimeout=30000
> watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5210f6d3
> Welcome to ZooKeeper!
> 2012-05-25 22:16:29,582 [myid:] - INFO
> [main-SendThread():ClientCnxn$SendThread@933] - Opening socket
> connection to server /23.22.86.141:80
> 2012-05-25 22:16:29,591 [myid:] - WARN
> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@123]
> - SecurityException: java.lang.SecurityException: Unable to locate a
> login configuration occurred when trying to find JAAS configuration.
> 2012-05-25 22:16:29,591 [myid:] - INFO
> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ZooKeeperSaslClient@125]
> - Client will not SASL-authenticate because the default JAAS
> configuration section 'Client' could not be found. If you are not
> using SASL, you may ignore this. On the other hand, if you expected
> SASL to work, please fix your JAAS configuration.
> JLine support is enabled
> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] 2012-05-25
> 22:16:29,735 [myid:] - INFO
> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@846]
> - Socket connection established to
> ec2-23-22-86-141.compute-1.amazonaws.com/23.22.86.141:80, initiating
> session
>
> [zk: zks1-davidosgi.rhcloud.com:80(CONNECTING) 0] ls /
> 2012-05-25 22:16:59,733 [myid:] - INFO
> [main-SendThread(ec2-23-22-86-141.compute-1.amazonaws.com:80):ClientCnxn$SendThread@1051]
> - Client session timed out, have not heard from server in 30000ms for
> sessionid 0x0, closing socket connection and attempting reconnect
> Exception in thread "main"
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /
>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1448)
>        at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1476)
>        at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:717)
>        at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:593)
>        at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:365)
>        at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:323)
>        at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:282)
> david@pop ~/proj/zk-test/zookeeper-3.4.3 $ bin/zkCli.sh -server
> zks1-davidosgi.rhcloud.com:80