You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "sadanand48 (via GitHub)" <gi...@apache.org> on 2023/08/07 20:01:25 UTC
[GitHub] [ozone] sadanand48 commented on a diff in pull request #5155: HDDS-9126. [Snapshot] SstFilteringService should only process active snapshots.
sadanand48 commented on code in PR #5155:
URL: https://github.com/apache/ozone/pull/5155#discussion_r1286335242
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/SstFilteringService.java:
##########
@@ -149,49 +161,55 @@ public BackgroundTaskResult call() throws Exception {
Table.KeyValue<String, SnapshotInfo> keyValue = iterator.next();
String snapShotTableKey = keyValue.getKey();
SnapshotInfo snapshotInfo = keyValue.getValue();
- UUID snapshotId = snapshotInfo.getSnapshotId();
-
- File omMetadataDir =
- OMStorage.getOmDbDir(ozoneManager.getConfiguration());
- String snapshotDir = omMetadataDir + OM_KEY_PREFIX + OM_SNAPSHOT_DIR;
- Path filePath =
- Paths.get(snapshotDir + OM_KEY_PREFIX + FILTERED_SNAPSHOTS);
-
- // If entry for the snapshotID is present in this file,
- // it has already undergone filtering.
- if (Files.exists(filePath)) {
- List<String> processedSnapshotIds = Files.readAllLines(filePath);
- if (snapshotId != null &&
- processedSnapshotIds.contains(snapshotId.toString())) {
- continue;
+ if (snapshotInfo.getSnapshotStatus()
+ .equals(SnapshotInfo.SnapshotStatus.SNAPSHOT_ACTIVE)) {
+ UUID snapshotId = snapshotInfo.getSnapshotId();
+
+ File omMetadataDir =
+ OMStorage.getOmDbDir(ozoneManager.getConfiguration());
+ String snapshotDir =
+ omMetadataDir + OM_KEY_PREFIX + OM_SNAPSHOT_DIR;
+ Path filePath =
+ Paths.get(snapshotDir + OM_KEY_PREFIX + FILTERED_SNAPSHOTS);
+
+ // If entry for the snapshotID is present in this file,
+ // it has already undergone filtering.
+ if (Files.exists(filePath)) {
+ List<String> processedSnapshotIds = Files.readAllLines(filePath);
+ if (snapshotId != null && processedSnapshotIds.contains(
+ snapshotId.toString())) {
+ continue;
+ }
}
- }
- LOG.debug("Processing snapshot {} to filter relevant SST Files",
- snapShotTableKey);
-
- List<Pair<String, String>> prefixPairs =
- constructPrefixPairs(snapshotInfo);
-
- try (ReferenceCounted<IOmMetadataReader, SnapshotCache>
- snapshotMetadataReader = snapshotCache.get()
- .get(snapshotInfo.getTableKey())) {
- OmSnapshot omSnapshot = (OmSnapshot) snapshotMetadataReader.get();
- RDBStore rdbStore = (RDBStore) omSnapshot.getMetadataManager()
- .getStore();
- RocksDatabase db = rdbStore.getDb();
- try (BootstrapStateHandler.Lock lock =
- getBootstrapStateLock().lock()) {
- db.deleteFilesNotMatchingPrefix(prefixPairs, FILTER_FUNCTION);
+ LOG.debug("Processing snapshot {} to filter relevant SST Files",
+ snapShotTableKey);
+
+ List<Pair<String, String>> prefixPairs =
+ constructPrefixPairs(snapshotInfo);
+
+ try (
+ ReferenceCounted<IOmMetadataReader, SnapshotCache>
+ snapshotMetadataReader = snapshotCache.get().get(
+ snapshotInfo.getTableKey())) {
Review Comment:
There is no field in the snapshot table to check status of filtering complete. We rely on the 'filtered snapshots' file in the snapshots directory, Even if we update the file with the snapshot name as part of the snapshot delete , there is still a possibility that this file was opened by the filtering service before this update was made during delete and we could still get the same issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org