You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@zookeeper.apache.org by "ZheyuanLin (Jira)" <ji...@apache.org> on 2023/03/23 03:37:00 UTC
[jira] [Created] (ZOOKEEPER-4684) Unable to load database on disk exception occurs when the cluster startup
ZheyuanLin created ZOOKEEPER-4684:
-------------------------------------
Summary: Unable to load database on disk exception occurs when the cluster startup
Key: ZOOKEEPER-4684
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4684
Project: ZooKeeper
Issue Type: Bug
Affects Versions: 3.4.10, 3.4.9, 3.4.6, 3.4.5
Reporter: ZheyuanLin
I found a cluster startup error where two of the three nodes failed to start properly due to this exception, so the cluster couldn't hold elections, eventually crashing:
{code:java}
2023-03-16 09:08:31,945 [myid:3] - INFO [main:QuorumPeer@429] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2023-03-16 09:08:31,962 [myid:3] - INFO [main:QuorumPeer@444] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2023-03-16 09:08:31,972 [myid:3] - ERROR [main:QuorumPeer@453] - Unable to load database on disk
java.io.IOException: Could not rename temporary file /zookeeper/zookeeper-3.4.5/zkdata/version-2/currentEpoch.tmp to /zookeeper/zookeeper-3.4.5/zkdata/version-2/currentEpoch
at org.apache.zookeeper.common.AtomicFileOutputStream.close(AtomicFileOutputStream.java:82)
at org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(QuorumPeer.java:1117)
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:447)
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:409)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
2023-03-16 09:08:31,973 [myid:3] - ERROR [main:QuorumPeerMain@89] - Unexpected exception, exiting abnormally
java.lang.RuntimeException: Unable to run quorum server
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:454)
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:409)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:111)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Caused by: java.io.IOException: Could not rename temporary file /zookeeper/zookeeper-3.4.5/zkdata/version-2/currentEpoch.tmp to /zookeeper/zookeeper-3.4.5/zkdata/version-2/currentEpoch
at org.apache.zookeeper.common.AtomicFileOutputStream.close(AtomicFileOutputStream.java:82)
at org.apache.zookeeper.server.quorum.QuorumPeer.writeLongToFile(QuorumPeer.java:1117)
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:447)
... 4 more
{code}
I almost found this exception on version 3.4.X, and the cluster starts normally on version 3.5.0 and above.
I think there are several potential causes for this error. One possibility is that there is a permission issue with the directory or file, which is preventing ZooKeeper from renaming the temporary file. Another possibility is that the disk is full or experiencing other issues, preventing the file rename from occurring.
Hope someone can come and help me solve it, thanks a lot.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)