You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@zookeeper.apache.org by Pushkar Deole <pd...@gmail.com> on 2018/10/12 11:32:21 UTC

zookeeper fails to start due to EOFException

Using zookeeper version 3.4.10
Zookeeper seems to have crashed due to no space available on the the device
and after that zookeeper is failing to start.
Cleared the unwanted data so the disk space is now sufficient. Rebooted the
system, zookeeper still fails to start with below error:

Any help will be appreciated...

[2018-10-12 08:17:38,828] ERROR Unexpected exception, exiting abnormally
(org.apache.zookeeper.server.ZooKeeperServerMain)
java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:392)
        at
org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
        at
org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570)
        at
org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652)
        at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:166)
        at
org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
        at
org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283)
        at
org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:410)
        at
org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118)
        at
org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:119)
        at
org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87)
        at
org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
        at
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)

Re: zookeeper fails to start due to EOFException

Posted by Norbert Kalmar <nk...@cloudera.com.INVALID>.

Hi,

Looks like after you run out of disk space, the transaction log file could
not be finished (no end of file character added). So it's corrupted.
Does all quorum member fail? If not, just startup the ones that can run,
delete the logs from datalogdir, and replace them from another node.

Or if losing the information stored in ZooKeeper is no concern, you can
just delete datalogdir's content and startup again (that's a big IF!)

You can also try the TxnLogToolkit, although it is not in 3.4.10 I'm afraid:
https://zookeeper.apache.org/doc/r3.4.13/zookeeperAdmin.html#Recovery+-+TxnLogToolkit

Regards,
Norbert


On Fri, Oct 12, 2018 at 1:32 PM Pushkar Deole <pd...@gmail.com> wrote:

> Using zookeeper version 3.4.10
> Zookeeper seems to have crashed due to no space available on the the device
> and after that zookeeper is failing to start.
> Cleared the unwanted data so the disk space is now sufficient. Rebooted the
> system, zookeeper still fails to start with below error:
>
> Any help will be appreciated...
>
> [2018-10-12 08:17:38,828] ERROR Unexpected exception, exiting abnormally
> (org.apache.zookeeper.server.ZooKeeperServerMain)
> java.io.EOFException
>         at java.io.DataInputStream.readInt(DataInputStream.java:392)
>         at
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
>         at
>
> org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
>         at
>
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585)
>         at
>
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604)
>         at
>
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570)
>         at
>
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652)
>         at
>
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:166)
>         at
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
>         at
>
> org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283)
>         at
>
> org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:410)
>         at
>
> org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118)
>         at
>
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:119)
>         at
>
> org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87)
>         at
>
> org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
>         at
>
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
>         at
>
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
>