You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Daniel Chan <da...@oracle.com> on 2017/04/25 22:54:53 UTC
EOFException on snapshot dump
I am using org.apache.zookeeper.server.SnapshotFormatter to dump out the content of a snapshot (908M) and got this exception:
Exception in thread "main" java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at org.apache.jute.BinaryInputArchive.readString(BinaryInputArchive.java:82)
at org.apache.zookeeper.server.DataTree.deserialize(DataTree.java:1217)
at org.apache.zookeeper.server.util.SerializeUtils.deserializeSnapshot(SerializeUtils.java:127)
at org.apache.zookeeper.server.persistence.FileSnap.deserialize(FileSnap.java:127)
at org.apache.zookeeper.server.SnapshotFormatter.run(SnapshotFormatter.java:65)
at org.apache.zookeeper.server.SnapshotFormatter.main(SnapshotFormatter.java:51)
Is there any debug mode I can turn on (in SnapshotFormatter) to find out what happened?
Background: a standalone Zookeeper failed on restart with below exception. I tried to see what is being stored from the last snapshot.
2017-04-25 10:47:27,831 [myid:] - ERROR [main:NIOServerCnxnFactory$1@44] - Thread Thread[main,5,main] died
java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArchive.java:97)
at org.apache.zookeeper.server.DataNode.deserialize(DataNode.java:158)
at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103)
at org.apache.zookeeper.server.DataTree.deserialize(DataTree.java:1194)
at org.apache.zookeeper.server.util.SerializeUtils.deserializeSnapshot(SerializeUtils.java:127)
at org.apache.zookeeper.server.persistence.FileSnap.deserialize(FileSnap.java:127)
at org.apache.zookeeper.server.persistence.FileSnap.deserialize(FileSnap.java:87)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:130)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:272)
at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:399)
at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:122)
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:113)
at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)
at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Thanks,
Daniel
Re: EOFException on snapshot dump
Posted by Michael Han <ha...@cloudera.com>.
No debug mode afaik. I suspect the snapshot was corrupted - it's partial so
the read was expecting more bytes than the file actually has, thus
EOFException. A workaround would be patch SnapshotFormatter so it caught
the exception and print what's already loaded and parsed, instead of bail
out and throw everything away.
On Tue, Apr 25, 2017 at 3:54 PM, Daniel Chan <da...@oracle.com>
wrote:
> I am using org.apache.zookeeper.server.SnapshotFormatter to dump out the
> content of a snapshot (908M) and got this exception:
> Exception in thread "main" java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:197)
> at java.io.DataInputStream.readFully(DataInputStream.java:169)
> at org.apache.jute.BinaryInputArchive.readString(BinaryInputArc
> hive.java:82)
> at org.apache.zookeeper.server.DataTree.deserialize(DataTree.ja
> va:1217)
> at org.apache.zookeeper.server.util.SerializeUtils.deserializeS
> napshot(SerializeUtils.java:127)
> at org.apache.zookeeper.server.persistence.FileSnap.deserialize
> (FileSnap.java:127)
> at org.apache.zookeeper.server.SnapshotFormatter.run(SnapshotFo
> rmatter.java:65)
> at org.apache.zookeeper.server.SnapshotFormatter.main(SnapshotF
> ormatter.java:51)
>
> Is there any debug mode I can turn on (in SnapshotFormatter) to find out
> what happened?
>
> Background: a standalone Zookeeper failed on restart with below exception.
> I tried to see what is being stored from the last snapshot.
> 2017-04-25 10:47:27,831 [myid:] - ERROR [main:NIOServerCnxnFactory$1@44]
> - Thread Thread[main,5,main] died
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at org.apache.jute.BinaryInputArchive.readBuffer(BinaryInputArc
> hive.java:97)
> at org.apache.zookeeper.server.DataNode.deserialize(DataNode.java:158)
> at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArc
> hive.java:103)
> at org.apache.zookeeper.server.DataTree.deserialize(DataTree.ja
> va:1194)
> at org.apache.zookeeper.server.util.SerializeUtils.deserializeS
> napshot(SerializeUtils.java:127)
> at org.apache.zookeeper.server.persistence.FileSnap.deserialize
> (FileSnap.java:127)
> at org.apache.zookeeper.server.persistence.FileSnap.deserialize
> (FileSnap.java:87)
> at org.apache.zookeeper.server.persistence.FileTxnSnapLog.resto
> re(FileTxnSnapLog.java:130)
> at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDataba
> se.java:223)
> at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeep
> erServer.java:272)
> at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKee
> perServer.java:399)
> at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIO
> ServerCnxnFactory.java:122)
> at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfi
> g(ZooKeeperServerMain.java:113)
> at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAn
> dRun(ZooKeeperServerMain.java:86)
> at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeep
> erServerMain.java:52)
> at org.apache.zookeeper.server.quorum.QuorumPeerMain.initialize
> AndRun(QuorumPeerMain.java:116)
> at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(Quoru
> mPeerMain.java:78)
>
> Thanks,
> Daniel
>
--
Cheers
Michael.