You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Brice Dutheil <br...@gmail.com> on 2016/03/22 15:05:48 UTC
Consumers / Producers won't connect upon Kafka/Zookeeper restart
I first noticed this (wrong) behavior when I want to connect to some feed
via kafkacat, hereby represented as bier-bar.
kafkacat -b localhost:9092 -t bier-bar -f '%p:%o -> %s' -C
% ERROR: Topic bier-bar error: Broker: Leader not available
When looking at the logs, the only thing that did struck me is this trace
in the Zookeeper log :
Got user-level KeeperException when processing
sessionid:0x1539e7fddbb0000 type:create cxid:0xce zxid:0x282
txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode =
NodeExists for /brokers
(org.apache.zookeeper.server.PrepRequestProcessor)
This trace appears at each Zookeeper restart.
A bit more context on the zookeeper logs:
[2016-03-22 14:24:53,192] INFO tickTime set to 3000
(org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-22 14:24:53,192] INFO minSessionTimeout set to -1
(org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-22 14:24:53,192] INFO maxSessionTimeout set to -1
(org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-22 14:24:53,213] INFO binding to port 0.0.0.0/0.0.0.0:2181
(org.apache.zookeeper.server.NIOServerCnxnFactory)
[2016-03-22 14:24:57,240] INFO Accepted socket connection from
/127.0.0.1:53451 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2016-03-22 14:24:57,300] INFO Client attempting to establish new
session at /127.0.0.1:53451
(org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-22 14:24:57,302] INFO Creating new log file: log.27e
(org.apache.zookeeper.server.persistence.FileTxnLog)
[2016-03-22 14:24:57,325] INFO Established session 0x1539e7fddbb0000
with negotiated timeout 6000 for client /127.0.0.1:53451
(org.apache.zookeeper.server.ZooKeeperServer)
[2016-03-22 14:24:58,682] INFO Got user-level KeeperException when
processing sessionid:0x1539e7fddbb0000 type:delete cxid:0xc5
zxid:0x281 txntype:-1 reqpath:n/a Error
Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode
for /admin/preferred_replica_election
(org.apache.zookeeper.server.PrepRequestProcessor)
[2016-03-22 14:24:58,789] INFO Got user-level KeeperException when
processing sessionid:0x1539e7fddbb0000 type:create cxid:0xce
zxid:0x282 txntype:-1 reqpath:n/a Error Path:/brokers
Error:KeeperErrorCode = NodeExists for /brokers
(org.apache.zookeeper.server.PrepRequestProcessor)
Then at some point after some kafka server restart, I got thit error in the
Kafka logs, this stacktrace don’t always appear in the logs.
[2016-03-22 14:24:58,792] INFO Result of znode creation is: OK
(kafka.utils.ZKCheckedEphemeral)
[2016-03-22 14:24:58,794] INFO Registered broker 1 at path
/brokers/ids/1 with addresses: PLAINTEXT ->
EndPoint(10.1.101.177,9092,PLAINTEXT) (kafka.utils.ZkUtils)
[2016-03-22 14:24:58,798] ERROR [KafkaApi-1] error when handling
request Name: TopicMetadataRequest; Version: 0; CorrelationId: 565;
ClientId: producer-1; Topics: bier-bar (kafka.server.KafkaApis)
kafka.admin.AdminOperationException: replication factor: 1 larger than
available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:77)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:236)
at kafka.server.KafkaApis$$anonfun$20.apply(KafkaApis.scala:572)
at kafka.server.KafkaApis$$anonfun$20.apply(KafkaApis.scala:555)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.immutable.Set$Set1.foreach(Set.scala:79)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
at scala.collection.SetLike$class.map(SetLike.scala:92)
at scala.collection.AbstractSet.map(Set.scala:47)
at kafka.server.KafkaApis.getTopicMetadata(KafkaApis.scala:555)
at kafka.server.KafkaApis.handleTopicMetadataRequest(KafkaApis.scala:624)
at kafka.server.KafkaApis.handle(KafkaApis.scala:71)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
at java.lang.Thread.run(Thread.java:745)
As you can guess from the logs I’m running a single kafka broker, with id 1,
and I don’t want more brokers, also I created my topic with the following
command :
./bin/kafka-topics.sh --create --topic bier-bar2 --partition 3
--replication-factor 1 --zookeeper localhost:2181
I used the default replication-factor, and using this topic worked right
after creation, but know, I cannot use bier-bar, I don’t know what has gone
wrong here.
Note the topic describe :
./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic bier-bar
Topic:bier-bar PartitionCount:3 ReplicationFactor:1 Configs:
Topic: bier-bar Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic: bier-bar Partition: 1 Leader: 1 Replicas: 1 Isr: 1
Topic: bier-bar Partition: 2 Leader: 1 Replicas: 1 Isr: 1
To be sure I created another topic with the same parameters, and I don’t
have these issues. Also the describe got the me the very same result.
Just for the record it seems this guy has the same problem as me, but got
response.
http://stackoverflow.com/questions/34393837/zookeeper-kafka-error-keepererrorcode-nodeexists
— Brice
Re: Consumers / Producers won't connect upon Kafka/Zookeeper restart
Posted by Brice Dutheil <br...@gmail.com>.
Nobody knows ?
-- Brice
On Tue, Mar 22, 2016 at 3:05 PM, Brice Dutheil <br...@gmail.com>
wrote:
> I first noticed this (wrong) behavior when I want to connect to some feed
> via kafkacat, hereby represented as bier-bar.
>
> kafkacat -b localhost:9092 -t bier-bar -f '%p:%o -> %s' -C
> % ERROR: Topic bier-bar error: Broker: Leader not available
>
> When looking at the logs, the only thing that did struck me is this trace
> in the Zookeeper log :
>
> Got user-level KeeperException when processing sessionid:0x1539e7fddbb0000 type:create cxid:0xce zxid:0x282 txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
>
> This trace appears at each Zookeeper restart.
>
> A bit more context on the zookeeper logs:
>
> [2016-03-22 14:24:53,192] INFO tickTime set to 3000 (org.apache.zookeeper.server.ZooKeeperServer)
> [2016-03-22 14:24:53,192] INFO minSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
> [2016-03-22 14:24:53,192] INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer)
> [2016-03-22 14:24:53,213] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
> [2016-03-22 14:24:57,240] INFO Accepted socket connection from /127.0.0.1:53451 (org.apache.zookeeper.server.NIOServerCnxnFactory)
> [2016-03-22 14:24:57,300] INFO Client attempting to establish new session at /127.0.0.1:53451 (org.apache.zookeeper.server.ZooKeeperServer)
> [2016-03-22 14:24:57,302] INFO Creating new log file: log.27e (org.apache.zookeeper.server.persistence.FileTxnLog)
> [2016-03-22 14:24:57,325] INFO Established session 0x1539e7fddbb0000 with negotiated timeout 6000 for client /127.0.0.1:53451 (org.apache.zookeeper.server.ZooKeeperServer)
> [2016-03-22 14:24:58,682] INFO Got user-level KeeperException when processing sessionid:0x1539e7fddbb0000 type:delete cxid:0xc5 zxid:0x281 txntype:-1 reqpath:n/a Error Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election (org.apache.zookeeper.server.PrepRequestProcessor)
> [2016-03-22 14:24:58,789] INFO Got user-level KeeperException when processing sessionid:0x1539e7fddbb0000 type:create cxid:0xce zxid:0x282 txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NodeExists for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
>
> Then at some point after some kafka server restart, I got thit error in
> the Kafka logs, this stacktrace don’t always appear in the logs.
>
> [2016-03-22 14:24:58,792] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
> [2016-03-22 14:24:58,794] INFO Registered broker 1 at path /brokers/ids/1 with addresses: PLAINTEXT -> EndPoint(10.1.101.177,9092,PLAINTEXT) (kafka.utils.ZkUtils)
> [2016-03-22 14:24:58,798] ERROR [KafkaApi-1] error when handling request Name: TopicMetadataRequest; Version: 0; CorrelationId: 565; ClientId: producer-1; Topics: bier-bar (kafka.server.KafkaApis)
> kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
> at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:77)
> at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:236)
> at kafka.server.KafkaApis$$anonfun$20.apply(KafkaApis.scala:572)
> at kafka.server.KafkaApis$$anonfun$20.apply(KafkaApis.scala:555)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
> at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
> at scala.collection.immutable.Set$Set1.foreach(Set.scala:79)
> at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
> at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.scala:47)
> at scala.collection.SetLike$class.map(SetLike.scala:92)
> at scala.collection.AbstractSet.map(Set.scala:47)
> at kafka.server.KafkaApis.getTopicMetadata(KafkaApis.scala:555)
> at kafka.server.KafkaApis.handleTopicMetadataRequest(KafkaApis.scala:624)
> at kafka.server.KafkaApis.handle(KafkaApis.scala:71)
> at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
> at java.lang.Thread.run(Thread.java:745)
>
> As you can guess from the logs I’m running a single kafka broker, with id
> 1, and I don’t want more brokers, also I created my topic with the
> following command :
>
> ./bin/kafka-topics.sh --create --topic bier-bar2 --partition 3 --replication-factor 1 --zookeeper localhost:2181
>
> I used the default replication-factor, and using this topic worked right
> after creation, but know, I cannot use bier-bar, I don’t know what has gone
> wrong here.
>
> Note the topic describe :
>
> ./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic bier-bar
> Topic:bier-bar PartitionCount:3 ReplicationFactor:1 Configs:
> Topic: bier-bar Partition: 0 Leader: 1 Replicas: 1 Isr: 1
> Topic: bier-bar Partition: 1 Leader: 1 Replicas: 1 Isr: 1
> Topic: bier-bar Partition: 2 Leader: 1 Replicas: 1 Isr: 1
>
> To be sure I created another topic with the same parameters, and I don’t
> have these issues. Also the describe got the me the very same result.
>
> Just for the record it seems this guy has the same problem as me, but got
> response.
>
> http://stackoverflow.com/questions/34393837/zookeeper-kafka-error-keepererrorcode-nodeexists
> — Brice
>
>