You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Alex Jiao Ziheng (JIRA)" <ji...@apache.org> on 2015/03/26 07:03:52 UTC
[jira] [Updated] (KAFKA-2052) zookeeper.connect does not work when
specifying multiple zk nodes with chroot
[ https://issues.apache.org/jira/browse/KAFKA-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Jiao Ziheng updated KAFKA-2052:
------------------------------------
Description:
I wish to specify multiple zk nodes with chroot strings in server.properties config file but Kafka is unable to separate the zk nodes properly in the presence of chroot strings:
zookeeper.connect=zookeeper-staging-a-1.bezurk.org:2181/kafka,zookeeper-staging-b-1.bezurk.org:2181/kafka
Error logs:
==> kafka_init_stdout.log <==
[2015-03-26 13:58:21,330] INFO [Kafka Server 1], Connecting to zookeeper on zookeeper-staging-b-1.bezurk.org:2181/kafka,zookeeper-staging-a-1.bezurk.org:2181/kafka (kafka.server.KafkaServer)
[2015-03-26 13:58:21,438] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-03-26 13:58:21,454] INFO Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:host.name=ip-10-0-11-38.ap-southeast-1.compute.internal (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:java.version=1.7.0_75 (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:java.home=/usr/lib/jvm/jdk1.7.0_75/jre (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.class.path=:/opt/kafka/bin/../core/build/dependant-libs-2.8.0/*.jar:/opt/kafka/bin/../perf/build/libs//kafka-perf_2.8.0*.jar:/opt/kafka/bin/../clients/build/libs//kafka-clients*.jar:/opt/kafka/bin/../examples/build/libs//kafka-examples*.jar:/opt/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/opt/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/opt/kafka/bin/../libs/jopt-simple-3.2.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-javadoc.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-scaladoc.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-sources.jar:/opt/kafka/bin/../libs/log4j-1.2.15.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/scala-library-2.9.2.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.2.jar:/opt/kafka/bin/../libs/snappy-java-1.0.5.jar:/opt/kafka/bin/../libs/zkclient-0.3.jar:/opt/kafka/bin/../libs/zookeeper-3.3.4.jar:/opt/kafka/bin/../core/build/libs/kafka_2.8.0*.jar (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:os.version=3.2.0-58-virtual (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:user.name=kafka (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:user.home=/home/kafka (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:user.dir=/home/kafka (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,457] INFO Initiating client connection, connectString=zookeeper-staging-b-1.bezurk.org:2181/kafka,zookeeper-staging-a-1.bezurk.org:2181/kafka sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@1111c0d7 (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,564] INFO Opening socket connection to server zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181 (org.apache.zookeeper.ClientCnxn)
[2015-03-26 13:58:21,578] INFO Socket connection established to zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-03-26 13:58:21,596] INFO Session establishment complete on server zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181, sessionid = 0x24c50668a78000d, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2015-03-26 13:58:21,604] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
==> server.log <==
2015-03-26 13:58:21,750 FATAL kafka.server.KafkaServerStartable: Fatal error during KafkaServerStable startup. Prepare to shutdown
java.lang.IllegalArgumentException: Path length must be > 0
at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:48)
at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:626)
at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:213)
at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
at kafka.utils.ZkUtils$.makeSurePersistentPathExists(ZkUtils.scala:236)
at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:105)
at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:104)
at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
at scala.collection.immutable.List.foreach(List.scala:76)
at kafka.utils.ZkUtils$.setupCommonPaths(ZkUtils.scala:104)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:114)
at kafka.server.KafkaServer.startup(KafkaServer.scala:69)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
was:
I wish to specify multiple zk nodes in server.properties config file:
zookeeper.connect=zookeeper-staging-a-1.bezurk.org:2181/kafka,zookeeper-staging-b-1.bezurk.org:2181/kafka
Error logs:
==> kafka_init_stdout.log <==
[2015-03-26 13:58:21,330] INFO [Kafka Server 1], Connecting to zookeeper on zookeeper-staging-b-1.bezurk.org:2181/kafka,zookeeper-staging-a-1.bezurk.org:2181/kafka (kafka.server.KafkaServer)
[2015-03-26 13:58:21,438] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-03-26 13:58:21,454] INFO Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:host.name=ip-10-0-11-38.ap-southeast-1.compute.internal (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:java.version=1.7.0_75 (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,454] INFO Client environment:java.home=/usr/lib/jvm/jdk1.7.0_75/jre (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.class.path=:/opt/kafka/bin/../core/build/dependant-libs-2.8.0/*.jar:/opt/kafka/bin/../perf/build/libs//kafka-perf_2.8.0*.jar:/opt/kafka/bin/../clients/build/libs//kafka-clients*.jar:/opt/kafka/bin/../examples/build/libs//kafka-examples*.jar:/opt/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/opt/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/opt/kafka/bin/../libs/jopt-simple-3.2.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-javadoc.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-scaladoc.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-sources.jar:/opt/kafka/bin/../libs/log4j-1.2.15.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/scala-library-2.9.2.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.2.jar:/opt/kafka/bin/../libs/snappy-java-1.0.5.jar:/opt/kafka/bin/../libs/zkclient-0.3.jar:/opt/kafka/bin/../libs/zookeeper-3.3.4.jar:/opt/kafka/bin/../core/build/libs/kafka_2.8.0*.jar (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,455] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:os.version=3.2.0-58-virtual (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:user.name=kafka (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:user.home=/home/kafka (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,456] INFO Client environment:user.dir=/home/kafka (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,457] INFO Initiating client connection, connectString=zookeeper-staging-b-1.bezurk.org:2181/kafka,zookeeper-staging-a-1.bezurk.org:2181/kafka sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@1111c0d7 (org.apache.zookeeper.ZooKeeper)
[2015-03-26 13:58:21,564] INFO Opening socket connection to server zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181 (org.apache.zookeeper.ClientCnxn)
[2015-03-26 13:58:21,578] INFO Socket connection established to zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-03-26 13:58:21,596] INFO Session establishment complete on server zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181, sessionid = 0x24c50668a78000d, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2015-03-26 13:58:21,604] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
==> server.log <==
2015-03-26 13:58:21,750 FATAL kafka.server.KafkaServerStartable: Fatal error during KafkaServerStable startup. Prepare to shutdown
java.lang.IllegalArgumentException: Path length must be > 0
at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:48)
at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:626)
at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:213)
at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
at kafka.utils.ZkUtils$.makeSurePersistentPathExists(ZkUtils.scala:236)
at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:105)
at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:104)
at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
at scala.collection.immutable.List.foreach(List.scala:76)
at kafka.utils.ZkUtils$.setupCommonPaths(ZkUtils.scala:104)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:114)
at kafka.server.KafkaServer.startup(KafkaServer.scala:69)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
> zookeeper.connect does not work when specifying multiple zk nodes with chroot
> ------------------------------------------------------------------------------
>
> Key: KAFKA-2052
> URL: https://issues.apache.org/jira/browse/KAFKA-2052
> Project: Kafka
> Issue Type: Bug
> Components: config
> Reporter: Alex Jiao Ziheng
> Priority: Minor
>
> I wish to specify multiple zk nodes with chroot strings in server.properties config file but Kafka is unable to separate the zk nodes properly in the presence of chroot strings:
> zookeeper.connect=zookeeper-staging-a-1.bezurk.org:2181/kafka,zookeeper-staging-b-1.bezurk.org:2181/kafka
> Error logs:
> ==> kafka_init_stdout.log <==
> [2015-03-26 13:58:21,330] INFO [Kafka Server 1], Connecting to zookeeper on zookeeper-staging-b-1.bezurk.org:2181/kafka,zookeeper-staging-a-1.bezurk.org:2181/kafka (kafka.server.KafkaServer)
> [2015-03-26 13:58:21,438] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
> [2015-03-26 13:58:21,454] INFO Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,454] INFO Client environment:host.name=ip-10-0-11-38.ap-southeast-1.compute.internal (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,454] INFO Client environment:java.version=1.7.0_75 (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,454] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,454] INFO Client environment:java.home=/usr/lib/jvm/jdk1.7.0_75/jre (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,455] INFO Client environment:java.class.path=:/opt/kafka/bin/../core/build/dependant-libs-2.8.0/*.jar:/opt/kafka/bin/../perf/build/libs//kafka-perf_2.8.0*.jar:/opt/kafka/bin/../clients/build/libs//kafka-clients*.jar:/opt/kafka/bin/../examples/build/libs//kafka-examples*.jar:/opt/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/opt/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/opt/kafka/bin/../libs/jopt-simple-3.2.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-javadoc.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-scaladoc.jar:/opt/kafka/bin/../libs/kafka_2.9.2-0.8.1.1-sources.jar:/opt/kafka/bin/../libs/log4j-1.2.15.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/scala-library-2.9.2.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.2.jar:/opt/kafka/bin/../libs/snappy-java-1.0.5.jar:/opt/kafka/bin/../libs/zkclient-0.3.jar:/opt/kafka/bin/../libs/zookeeper-3.3.4.jar:/opt/kafka/bin/../core/build/libs/kafka_2.8.0*.jar (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,455] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,455] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,455] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,455] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,455] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,456] INFO Client environment:os.version=3.2.0-58-virtual (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,456] INFO Client environment:user.name=kafka (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,456] INFO Client environment:user.home=/home/kafka (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,456] INFO Client environment:user.dir=/home/kafka (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,457] INFO Initiating client connection, connectString=zookeeper-staging-b-1.bezurk.org:2181/kafka,zookeeper-staging-a-1.bezurk.org:2181/kafka sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@1111c0d7 (org.apache.zookeeper.ZooKeeper)
> [2015-03-26 13:58:21,564] INFO Opening socket connection to server zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181 (org.apache.zookeeper.ClientCnxn)
> [2015-03-26 13:58:21,578] INFO Socket connection established to zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181, initiating session (org.apache.zookeeper.ClientCnxn)
> [2015-03-26 13:58:21,596] INFO Session establishment complete on server zookeeper-staging-b-1.bezurk.org/10.0.29.191:2181, sessionid = 0x24c50668a78000d, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
> [2015-03-26 13:58:21,604] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
> ==> server.log <==
> 2015-03-26 13:58:21,750 FATAL kafka.server.KafkaServerStartable: Fatal error during KafkaServerStable startup. Prepare to shutdown
> java.lang.IllegalArgumentException: Path length must be > 0
> at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:48)
> at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:626)
> at org.I0Itec.zkclient.ZkConnection.create(ZkConnection.java:87)
> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:308)
> at org.I0Itec.zkclient.ZkClient$1.call(ZkClient.java:304)
> at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
> at org.I0Itec.zkclient.ZkClient.create(ZkClient.java:304)
> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:213)
> at org.I0Itec.zkclient.ZkClient.createPersistent(ZkClient.java:223)
> at kafka.utils.ZkUtils$.makeSurePersistentPathExists(ZkUtils.scala:236)
> at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:105)
> at kafka.utils.ZkUtils$$anonfun$setupCommonPaths$1.apply(ZkUtils.scala:104)
> at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
> at scala.collection.immutable.List.foreach(List.scala:76)
> at kafka.utils.ZkUtils$.setupCommonPaths(ZkUtils.scala:104)
> at kafka.server.KafkaServer.initZk(KafkaServer.scala:114)
> at kafka.server.KafkaServer.startup(KafkaServer.scala:69)
> at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
> at kafka.Kafka$.main(Kafka.scala:46)
> at kafka.Kafka.main(Kafka.scala)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)