You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/11/01 01:11:10 UTC
[GitHub] [ozone] GeorgeJahad commented on a diff in pull request #3824: HDDS-7281. [Snapshot] Handle RocksDB compaction DAG persistence and reconstruction
GeorgeJahad commented on code in PR #3824:
URL: https://github.com/apache/ozone/pull/3824#discussion_r1009990346
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -131,7 +132,30 @@ public static DBCheckpoint createOmSnapshotCheckpoint(
OMMetadataManager omMetadataManager, SnapshotInfo snapshotInfo)
throws IOException {
RDBStore store = (RDBStore) omMetadataManager.getStore();
- return store.getSnapshot(snapshotInfo.getCheckpointDirName());
+
+ final long dbLatestSequenceNumber =
+ ((RDBStore) omMetadataManager.getStore()).getDb()
+ .getLatestSequenceNumber();
+
+ final RocksDBCheckpointDiffer dbCpDiffer =
+ omMetadataManager.getStore().getRocksDBCheckpointDiffer();
+
+ final DBCheckpoint dbCheckpoint = store.getSnapshot(
+ snapshotInfo.getCheckpointDirName());
+
+ // Write snapshot generation (latest sequence number) to compaction log.
+ // This will be used for DAG reconstruction as snapshotGeneration.
+ dbCpDiffer.appendSequenceNumberToCompactionLog(dbLatestSequenceNumber);
+
+ // Set compaction log filename to the latest DB sequence number
+ // right after taking the RocksDB checkpoint for Ozone snapshot.
+ //
+ // Note it doesn't matter if sequence number hasn't increased (even though
+ // it shouldn't happen), since the writer always appends the file.
+ dbCpDiffer.setCompactionLogParentDir(store.getSnapshotsParentDir());
Review Comment:
Does the parent dir get set every time a snapshot is created?
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -131,7 +132,30 @@ public static DBCheckpoint createOmSnapshotCheckpoint(
OMMetadataManager omMetadataManager, SnapshotInfo snapshotInfo)
throws IOException {
RDBStore store = (RDBStore) omMetadataManager.getStore();
- return store.getSnapshot(snapshotInfo.getCheckpointDirName());
+
+ final long dbLatestSequenceNumber =
+ ((RDBStore) omMetadataManager.getStore()).getDb()
+ .getLatestSequenceNumber();
+
+ final RocksDBCheckpointDiffer dbCpDiffer =
+ omMetadataManager.getStore().getRocksDBCheckpointDiffer();
+
+ final DBCheckpoint dbCheckpoint = store.getSnapshot(
+ snapshotInfo.getCheckpointDirName());
+
+ // Write snapshot generation (latest sequence number) to compaction log.
+ // This will be used for DAG reconstruction as snapshotGeneration.
+ dbCpDiffer.appendSequenceNumberToCompactionLog(dbLatestSequenceNumber);
+
+ // Set compaction log filename to the latest DB sequence number
+ // right after taking the RocksDB checkpoint for Ozone snapshot.
+ //
+ // Note it doesn't matter if sequence number hasn't increased (even though
+ // it shouldn't happen), since the writer always appends the file.
+ dbCpDiffer.setCompactionLogParentDir(store.getSnapshotsParentDir());
Review Comment:
Does the parent dir need to get set every time a snapshot is created?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org