You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Jyotirmoy Sinha (Jira)" <ji...@apache.org> on 2022/10/25 06:06:00 UTC
[jira] [Updated] (HDDS-7404) [snapshot] Creating snapshot with snapshot name starting with # successful with different snapshot name
[ https://issues.apache.org/jira/browse/HDDS-7404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jyotirmoy Sinha updated HDDS-7404:
----------------------------------
Description:
Steps :
# Create snapshot with snapshot name starting with #
# Snapshot created successfully with some different name
Note - The output in docker logs display a lot of detailed data when snapshot names start with #. In other scenarios the docker logs data is not so verbose.
Console output :
{code:java}
bash-4.2$ ozone sh snapshot create vol1/buck1 #snap3
bash-4.2$ {code}
Snapshot console info:
{code:java}
bash-4.2$ ozone fs -ls /vol1/buck1/.snapshot/s20221025-055549.226/
Found 1 items
-rw-rw-rw- 3 hadoop hadoop 17539 2022-10-21 10:45 /vol1/buck1/.snapshot/s20221025-055549.226/key11 {code}
Docker logs:
{code:java}
2022-10-25 05:55:49,226 [OM StateMachine ApplyTransaction Thread - 0] INFO snapshot.OMSnapshotCreateRequest: created snapshot: name s20221025-055549.226 in snapshotPath: vol1/buck1
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000094.sst
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000094.sst,
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000094.sst:: 1
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
2022-10-25 05:55:49,272 [OMDoubleBufferFlushThread] INFO db.RDBCheckpointManager: Created checkpoint at /data/metadata/db.snapshots/om.db-e8ed0cf0-bd40-4e33-a743-6458092ba3da in 42 milliseconds
2022-10-25 05:56:02,594 [IPC Server handler 13 on default port 9862] WARN db.RDBStore: Unable to get delta updates since sequenceNumber 35. This exception will be thrown to the client
org.apache.hadoop.hdds.utils.db.SequenceNumberNotFoundException: Unable to read data from RocksDB wal to get delta updates. It may have already been flushed to SSTs.
at org.apache.hadoop.hdds.utils.db.RDBStore.getUpdatesSince(RDBStore.java:359)
at org.apache.hadoop.ozone.om.OzoneManager.getDBUpdates(OzoneManager.java:3810)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.getOMDBUpdates(OzoneManagerRequestHandler.java:321)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleReadRequest(OzoneManagerRequestHandler.java:223)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:226)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:175)
at org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
at org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:466)
at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:574)
at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:552)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1035)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:963)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2966)
2022-10-25 05:56:02,604 [qtp1810327249-66] INFO utils.DBCheckpointServlet: Received request to obtain DB checkpoint snapshot
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000097.sst
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000092.sst
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000088.sst
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000085.sst
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000083.sst
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000098.sst,
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000098.sst:: 1
2022-10-25 05:56:02,626 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000097.sst:: 1
2022-10-25 05:56:02,626 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000092.sst:: 1
2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000088.sst:: 1
2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000085.sst:: 1
2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000088.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000097.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000098.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000085.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000083.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000088.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000097.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000092.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000085.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000092.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000088.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000097.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000083.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000085.sst
2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000092.sst
2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000099.sst
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000099.sst,
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000099.sst:: 1
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000088.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000097.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000099.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000098.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000085.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000083.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000088.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000097.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000092.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000085.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000092.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000088.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000097.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000099.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000083.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000085.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000092.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
2022-10-25 05:56:02,639 [qtp1810327249-66] INFO db.RDBCheckpointManager: Created checkpoint at /data/metadata/db.checkpoints/om.db_checkpoint_1666677362604 in 34 milliseconds
2022-10-25 05:56:02,654 [qtp1810327249-66] INFO utils.DBCheckpointServlet: Time taken to write the checkpoint to response output stream: 14 milliseconds
2022-10-25 05:56:02,654 [qtp1810327249-66] INFO db.RocksDBCheckpoint: Cleaning up RocksDB checkpoint at /data/metadata/db.checkpoints/om.db_checkpoint_1666677362604 {code}
was:
Steps :
# Create snapshot with snapshot name starting with #
# Snapshot created successfully with some different name
Console output :
{code:java}
bash-4.2$ ozone sh snapshot create vol1/buck1 #snap3
bash-4.2$ {code}
Snapshot console info:
{code:java}
bash-4.2$ ozone fs -ls /vol1/buck1/.snapshot/s20221025-055549.226/
Found 1 items
-rw-rw-rw- 3 hadoop hadoop 17539 2022-10-21 10:45 /vol1/buck1/.snapshot/s20221025-055549.226/key11 {code}
Docker logs:
{code:java}
2022-10-25 05:55:49,226 [OM StateMachine ApplyTransaction Thread - 0] INFO snapshot.OMSnapshotCreateRequest: created snapshot: name s20221025-055549.226 in snapshotPath: vol1/buck1
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000094.sst
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000094.sst,
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000094.sst:: 1
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
2022-10-25 05:55:49,272 [OMDoubleBufferFlushThread] INFO db.RDBCheckpointManager: Created checkpoint at /data/metadata/db.snapshots/om.db-e8ed0cf0-bd40-4e33-a743-6458092ba3da in 42 milliseconds
2022-10-25 05:56:02,594 [IPC Server handler 13 on default port 9862] WARN db.RDBStore: Unable to get delta updates since sequenceNumber 35. This exception will be thrown to the client
org.apache.hadoop.hdds.utils.db.SequenceNumberNotFoundException: Unable to read data from RocksDB wal to get delta updates. It may have already been flushed to SSTs.
at org.apache.hadoop.hdds.utils.db.RDBStore.getUpdatesSince(RDBStore.java:359)
at org.apache.hadoop.ozone.om.OzoneManager.getDBUpdates(OzoneManager.java:3810)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.getOMDBUpdates(OzoneManagerRequestHandler.java:321)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleReadRequest(OzoneManagerRequestHandler.java:223)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:226)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:175)
at org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
at org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:466)
at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:574)
at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:552)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1035)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:963)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2966)
2022-10-25 05:56:02,604 [qtp1810327249-66] INFO utils.DBCheckpointServlet: Received request to obtain DB checkpoint snapshot
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000097.sst
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000092.sst
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000088.sst
2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000085.sst
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000083.sst
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000098.sst,
2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000098.sst:: 1
2022-10-25 05:56:02,626 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000097.sst:: 1
2022-10-25 05:56:02,626 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000092.sst:: 1
2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000088.sst:: 1
2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000085.sst:: 1
2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000088.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000097.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000098.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000085.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000083.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000088.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000097.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000092.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000085.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000092.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000088.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000097.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000083.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000085.sst
2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000092.sst
2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000099.sst
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000099.sst,
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000099.sst:: 1
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000088.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000097.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000099.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000098.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000085.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000083.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000088.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000097.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000092.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000085.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000092.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000088.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000097.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000099.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000083.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000085.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000092.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
2022-10-25 05:56:02,639 [qtp1810327249-66] INFO db.RDBCheckpointManager: Created checkpoint at /data/metadata/db.checkpoints/om.db_checkpoint_1666677362604 in 34 milliseconds
2022-10-25 05:56:02,654 [qtp1810327249-66] INFO utils.DBCheckpointServlet: Time taken to write the checkpoint to response output stream: 14 milliseconds
2022-10-25 05:56:02,654 [qtp1810327249-66] INFO db.RocksDBCheckpoint: Cleaning up RocksDB checkpoint at /data/metadata/db.checkpoints/om.db_checkpoint_1666677362604 {code}
> [snapshot] Creating snapshot with snapshot name starting with # successful with different snapshot name
> -------------------------------------------------------------------------------------------------------
>
> Key: HDDS-7404
> URL: https://issues.apache.org/jira/browse/HDDS-7404
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Manager
> Affects Versions: 1.3.0
> Environment: docker-compose on HDDS-6517-Snapshot
> Reporter: Jyotirmoy Sinha
> Priority: Major
>
> Steps :
> # Create snapshot with snapshot name starting with #
> # Snapshot created successfully with some different name
> Note - The output in docker logs display a lot of detailed data when snapshot names start with #. In other scenarios the docker logs data is not so verbose.
> Console output :
> {code:java}
> bash-4.2$ ozone sh snapshot create vol1/buck1 #snap3
> bash-4.2$ {code}
> Snapshot console info:
> {code:java}
> bash-4.2$ ozone fs -ls /vol1/buck1/.snapshot/s20221025-055549.226/
> Found 1 items
> -rw-rw-rw- 3 hadoop hadoop 17539 2022-10-21 10:45 /vol1/buck1/.snapshot/s20221025-055549.226/key11 {code}
> Docker logs:
> {code:java}
> 2022-10-25 05:55:49,226 [OM StateMachine ApplyTransaction Thread - 0] INFO snapshot.OMSnapshotCreateRequest: created snapshot: name s20221025-055549.226 in snapshotPath: vol1/buck1
> 2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
> 2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
> 2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000094.sst
> 2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
> 2022-10-25 05:55:49,256 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000094.sst,
> 2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000094.sst:: 1
> 2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
> 2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
> 2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
> 2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:55:49,257 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:55:49,258 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
> 2022-10-25 05:55:49,259 [Thread-289] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
> 2022-10-25 05:55:49,272 [OMDoubleBufferFlushThread] INFO db.RDBCheckpointManager: Created checkpoint at /data/metadata/db.snapshots/om.db-e8ed0cf0-bd40-4e33-a743-6458092ba3da in 42 milliseconds
> 2022-10-25 05:56:02,594 [IPC Server handler 13 on default port 9862] WARN db.RDBStore: Unable to get delta updates since sequenceNumber 35. This exception will be thrown to the client
> org.apache.hadoop.hdds.utils.db.SequenceNumberNotFoundException: Unable to read data from RocksDB wal to get delta updates. It may have already been flushed to SSTs.
> at org.apache.hadoop.hdds.utils.db.RDBStore.getUpdatesSince(RDBStore.java:359)
> at org.apache.hadoop.ozone.om.OzoneManager.getDBUpdates(OzoneManager.java:3810)
> at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.getOMDBUpdates(OzoneManagerRequestHandler.java:321)
> at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleReadRequest(OzoneManagerRequestHandler.java:223)
> at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:226)
> at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:175)
> at org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
> at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
> at org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:466)
> at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:574)
> at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:552)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1035)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:963)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2966)
> 2022-10-25 05:56:02,604 [qtp1810327249-66] INFO utils.DBCheckpointServlet: Received request to obtain DB checkpoint snapshot
> 2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
> 2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
> 2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000097.sst
> 2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000092.sst
> 2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000088.sst
> 2022-10-25 05:56:02,624 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000085.sst
> 2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000083.sst
> 2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
> 2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000098.sst,
> 2022-10-25 05:56:02,625 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000098.sst:: 1
> 2022-10-25 05:56:02,626 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000097.sst:: 1
> 2022-10-25 05:56:02,626 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000092.sst:: 1
> 2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000088.sst:: 1
> 2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000085.sst:: 1
> 2022-10-25 05:56:02,627 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000088.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,628 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000097.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,629 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
> 2022-10-25 05:56:02,630 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000098.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000085.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000083.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000088.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000097.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000092.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000085.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000092.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000088.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000097.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000083.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
> 2022-10-25 05:56:02,631 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000085.sst
> 2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000092.sst
> 2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
> 2022-10-25 05:56:02,632 [Thread-309] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
> 2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: kLevelL0FilesNum
> 2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: List of input files:
> 2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000099.sst
> 2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: List of output files:
> 2022-10-25 05:56:02,636 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: /data/metadata/om.db/000099.sst,
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: getSSTFileSummary 000099.sst:: 1
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Printing the Graph
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000078.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000088.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000060.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000072.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000079.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000097.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000099.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000050.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000083.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000078.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000075.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000072.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000081.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000094.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000086.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000089.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000075.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000066.sst
> 2022-10-25 05:56:02,637 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000053.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000081.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000098.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000085.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000083.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000088.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000097.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000092.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000085.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000082.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000092.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: No Children Node
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Parent Node :000053.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000066.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000050.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Children Node :000060.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: src snap:null
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: dest snap:null
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000078.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000088.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000072.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000060.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000079.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000097.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000099.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000050.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000083.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000094.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000086.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000089.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000075.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000066.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000081.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000085.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000092.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000082.sst
> 2022-10-25 05:56:02,638 [Thread-312] WARN rocksdiff.RocksDBCheckpointDiffer: Files are :000053.sst
> 2022-10-25 05:56:02,639 [qtp1810327249-66] INFO db.RDBCheckpointManager: Created checkpoint at /data/metadata/db.checkpoints/om.db_checkpoint_1666677362604 in 34 milliseconds
> 2022-10-25 05:56:02,654 [qtp1810327249-66] INFO utils.DBCheckpointServlet: Time taken to write the checkpoint to response output stream: 14 milliseconds
> 2022-10-25 05:56:02,654 [qtp1810327249-66] INFO db.RocksDBCheckpoint: Cleaning up RocksDB checkpoint at /data/metadata/db.checkpoints/om.db_checkpoint_1666677362604 {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org