You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Sunil CHAUDHARI <su...@dbs.com.INVALID> on 2020/03/12 10:35:47 UTC

Very Urgent: Error while creating ephemeral at /brokers/ids/0, node already exists and owner

Hi,
Whenever I try to do rolling restart and start one of my broker, I get this error.
Can Anyone help me to get rid of this?

[2020-03-12 18:33:34,661] INFO Logs loading complete in 1553 ms. (kafka.log.LogManager)
[2020-03-12 18:33:34,677] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2020-03-12 18:33:34,678] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2020-03-12 18:33:35,005] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2020-03-12 18:33:35,049] INFO [SocketServer brokerId=0] Created data-plane acceptor and processors for endpoint : EndPoint(null,9092,ListenerName(PLAINTEXT),PLAINTEXT) (kafka.network.SocketServer)
[2020-03-12 18:33:35,050] INFO [SocketServer brokerId=0] Started 1 acceptor threads for data-plane (kafka.network.SocketServer)
[2020-03-12 18:33:35,109] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-ElectPreferredLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2020-03-12 18:33:35,121] INFO [ExpirationReaper-0-ListOffsets]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2020-03-12 18:33:35,139] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
[2020-03-12 18:33:35,225] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient)
[2020-03-12 18:33:35,245] ERROR Error while creating ephemeral at /brokers/ids/0, node already exists and owner '216411990614016005' does not match current session '144234981936660481' (kafka.zk.KafkaZkClient$CheckedEphemeral)
[2020-03-12 18:33:35,252] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:122)
        at kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:1784)
        at kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:1722)
        at kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:1689)
        at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:97)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:300)
        at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:114)
        at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:66)
CONFIDENTIAL NOTE:
The information contained in this email is intended only for the use of the individual or entity named above and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete the mail. Thank you.

RE: [External] Re: Very Urgent: Error while creating ephemeral at /brokers/ids/0, node already exists and owner

Posted by Sunil CHAUDHARI <su...@dbs.com.INVALID>.

Hi Jonathan,

"are you gracefully shutting down the broker?"



above question solved my prob gracefully 😊.  Hahaha

Thanks a lot!





Regards,

Sunil.



-----Original Message-----
From: Jonathan Santilli <jo...@gmail.com>
Sent: Thursday, March 12, 2020 5:17 PM
To: Kafka Users <us...@kafka.apache.org>
Subject: [External] Re: Very Urgent: Error while creating ephemeral at /brokers/ids/0, node already exists and owner



This mail originated from an external party outside DBS - users-return-39785-sunilchaudhari=dbs.com@kafka.apache.org<ma...@kafka.apache.org>. Do not click on links or open attachments unless you recognize the sender and know the content is safe.



Hello Sunil,



are you gracefully shutting down the broker?



can you verify there is no other broker within the cluster with the same id 0?



can you check in the logs that the connection to ZooKeeper is getting close/disconnected when shutting down the Broker?





Cheers!







On Thu, Mar 12, 2020 at 10:36 AM Sunil CHAUDHARI <su...@dbs.com.invalid>> wrote:



> Hi,

> Whenever I try to do rolling restart and start one of my broker, I get

> this error.

> Can Anyone help me to get rid of this?

>

> [2020-03-12 18:33:34,661] INFO Logs loading complete in 1553 ms.

> (kafka.log.LogManager)

> [2020-03-12 18:33:34,677] INFO Starting log cleanup with a period of

> 300000 ms. (kafka.log.LogManager)

> [2020-03-12 18:33:34,678] INFO Starting log flusher with a default

> period of 9223372036854775807 ms. (kafka.log.LogManager)

> [2020-03-12 18:33:35,005] INFO Awaiting socket connections on 0.0.0.0:9092.

> (kafka.network.Acceptor)

> [2020-03-12 18:33:35,049] INFO [SocketServer brokerId=0] Created

> data-plane acceptor and processors for endpoint :

> EndPoint(null,9092,ListenerName(PLAINTEXT),PLAINTEXT)

> (kafka.network.SocketServer)

> [2020-03-12 18:33:35,050] INFO [SocketServer brokerId=0] Started 1

> acceptor threads for data-plane (kafka.network.SocketServer)

> [2020-03-12 18:33:35,109] INFO [ExpirationReaper-0-Produce]: Starting

> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

> [2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-ElectPreferredLeader]:

> Starting

> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

> [2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-DeleteRecords]:

> Starting

> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

> [2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-Fetch]: Starting

> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

> [2020-03-12 18:33:35,121] INFO [ExpirationReaper-0-ListOffsets]:

> Starting

> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

> [2020-03-12 18:33:35,139] INFO [LogDirFailureHandler]: Starting

> (kafka.server.ReplicaManager$LogDirFailureHandler)

> [2020-03-12 18:33:35,225] INFO Creating /brokers/ids/0 (is it secure?

> false) (kafka.zk.KafkaZkClient)

> [2020-03-12 18:33:35,245] ERROR Error while creating ephemeral at

> /brokers/ids/0, node already exists and owner '216411990614016005'

> does not match current session '144234981936660481'

> (kafka.zk.KafkaZkClient$CheckedEphemeral)

> [2020-03-12 18:33:35,252] ERROR [KafkaServer id=0] Fatal error during

> KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)

> org.apache.zookeeper.KeeperException$NodeExistsException:

> KeeperErrorCode = NodeExists

>         at

> org.apache.zookeeper.KeeperException.create(KeeperException.java:122)

>         at

> kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:1784)

>         at

> kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:1722)

>         at

> kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:1689)

>         at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:97)

>         at kafka.server.KafkaServer.startup(KafkaServer.scala:300)

>         at

> io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:114)

>         at

> io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:6

> 6)

> CONFIDENTIAL NOTE:

> The information contained in this email is intended only for the use

> of the individual or entity named above and may contain information

> that is privileged, confidential and exempt from disclosure under applicable law.

> If the reader of this message is not the intended recipient, you are

> hereby notified that any dissemination, distribution or copying of

> this communication is strictly prohibited. If you have received this

> message in error, please immediately notify the sender and delete the mail. Thank you.

>





--

Santilli Jonathan



CONFIDENTIAL NOTE:
The information contained in this email is intended only for the use of the individual or entity named above and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this message in error, please immediately notify the sender and delete the mail. Thank you.

Re: Very Urgent: Error while creating ephemeral at /brokers/ids/0, node already exists and owner

Posted by Jonathan Santilli <jo...@gmail.com>.

Hello Sunil,

are you gracefully shutting down the broker?

can you verify there is no other broker within the cluster with the same id
0?

can you check in the logs that the connection to ZooKeeper is getting
close/disconnected when shutting down the Broker?


Cheers!



On Thu, Mar 12, 2020 at 10:36 AM Sunil CHAUDHARI
<su...@dbs.com.invalid> wrote:

> Hi,
> Whenever I try to do rolling restart and start one of my broker, I get
> this error.
> Can Anyone help me to get rid of this?
>
> [2020-03-12 18:33:34,661] INFO Logs loading complete in 1553 ms.
> (kafka.log.LogManager)
> [2020-03-12 18:33:34,677] INFO Starting log cleanup with a period of
> 300000 ms. (kafka.log.LogManager)
> [2020-03-12 18:33:34,678] INFO Starting log flusher with a default period
> of 9223372036854775807 ms. (kafka.log.LogManager)
> [2020-03-12 18:33:35,005] INFO Awaiting socket connections on 0.0.0.0:9092.
> (kafka.network.Acceptor)
> [2020-03-12 18:33:35,049] INFO [SocketServer brokerId=0] Created
> data-plane acceptor and processors for endpoint :
> EndPoint(null,9092,ListenerName(PLAINTEXT),PLAINTEXT)
> (kafka.network.SocketServer)
> [2020-03-12 18:33:35,050] INFO [SocketServer brokerId=0] Started 1
> acceptor threads for data-plane (kafka.network.SocketServer)
> [2020-03-12 18:33:35,109] INFO [ExpirationReaper-0-Produce]: Starting
> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
> [2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-ElectPreferredLeader]:
> Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
> [2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-DeleteRecords]:
> Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
> [2020-03-12 18:33:35,116] INFO [ExpirationReaper-0-Fetch]: Starting
> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
> [2020-03-12 18:33:35,121] INFO [ExpirationReaper-0-ListOffsets]: Starting
> (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
> [2020-03-12 18:33:35,139] INFO [LogDirFailureHandler]: Starting
> (kafka.server.ReplicaManager$LogDirFailureHandler)
> [2020-03-12 18:33:35,225] INFO Creating /brokers/ids/0 (is it secure?
> false) (kafka.zk.KafkaZkClient)
> [2020-03-12 18:33:35,245] ERROR Error while creating ephemeral at
> /brokers/ids/0, node already exists and owner '216411990614016005' does not
> match current session '144234981936660481'
> (kafka.zk.KafkaZkClient$CheckedEphemeral)
> [2020-03-12 18:33:35,252] ERROR [KafkaServer id=0] Fatal error during
> KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode
> = NodeExists
>         at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:122)
>         at
> kafka.zk.KafkaZkClient$CheckedEphemeral.getAfterNodeExists(KafkaZkClient.scala:1784)
>         at
> kafka.zk.KafkaZkClient$CheckedEphemeral.create(KafkaZkClient.scala:1722)
>         at
> kafka.zk.KafkaZkClient.checkedEphemeralCreate(KafkaZkClient.scala:1689)
>         at kafka.zk.KafkaZkClient.registerBroker(KafkaZkClient.scala:97)
>         at kafka.server.KafkaServer.startup(KafkaServer.scala:300)
>         at
> io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:114)
>         at
> io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:66)
> CONFIDENTIAL NOTE:
> The information contained in this email is intended only for the use of
> the individual or entity named above and may contain information that is
> privileged, confidential and exempt from disclosure under applicable law.
> If the reader of this message is not the intended recipient, you are hereby
> notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you have received this message in
> error, please immediately notify the sender and delete the mail. Thank you.
>


-- 
Santilli Jonathan