You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "swamirishi (via GitHub)" <gi...@apache.org> on 2025/11/20 16:54:59 UTC

[PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

swamirishi opened a new pull request, #9335:
URL: https://github.com/apache/ozone/pull/9335

   ## What changes were proposed in this pull request?
   OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/HDDS-13967
   
   ## How was this patch tested?
   Updated existing unit tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #9335:
URL: https://github.com/apache/ozone/pull/9335#discussion_r2563529550


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java:
##########
@@ -399,12 +436,12 @@ private OzoneConfiguration getConf() {
    * Collects paths to all snapshot databases from the OM DB.
    *
    * @param activeOMMetadataManager OMMetadataManager instance
-   * @return Set of paths to snapshot databases
+   * @return Map of paths to snapshot databases with snapshot IDs as keys.
    * @throws IOException if an I/O error occurs
    */
-  Set<Path> getSnapshotDirsFromDB(OMMetadataManager activeOMMetadataManager, OMMetadataManager omMetadataManager,
+  Map<UUID, Path> getSnapshotDirsFromDB(OMMetadataManager activeOMMetadataManager, OMMetadataManager omMetadataManager,
       OmSnapshotLocalDataManager localDataManager) throws IOException {
-    Set<Path> snapshotPaths = new HashSet<>();
+    Map<UUID, Path> snapshotPaths = new HashMap<>();
     try (TableIterator<String, ? extends Table.KeyValue<String, SnapshotInfo>> iter =
         omMetadataManager.getSnapshotInfoTable().iterator()) {
       while (iter.hasNext()) {

Review Comment:
   this is unrelated, but when transferring snapshots in the checkpoint, do we want to skip SNAPSHOT_DELETED snapshots?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9335:
URL: https://github.com/apache/ozone/pull/9335#discussion_r2564537380


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java:
##########
@@ -399,12 +436,12 @@ private OzoneConfiguration getConf() {
    * Collects paths to all snapshot databases from the OM DB.
    *
    * @param activeOMMetadataManager OMMetadataManager instance
-   * @return Set of paths to snapshot databases
+   * @return Map of paths to snapshot databases with snapshot IDs as keys.
    * @throws IOException if an I/O error occurs
    */
-  Set<Path> getSnapshotDirsFromDB(OMMetadataManager activeOMMetadataManager, OMMetadataManager omMetadataManager,
+  Map<UUID, Path> getSnapshotDirsFromDB(OMMetadataManager activeOMMetadataManager, OMMetadataManager omMetadataManager,
       OmSnapshotLocalDataManager localDataManager) throws IOException {
-    Set<Path> snapshotPaths = new HashSet<>();
+    Map<UUID, Path> snapshotPaths = new HashMap<>();
     try (TableIterator<String, ? extends Table.KeyValue<String, SnapshotInfo>> iter =
         omMetadataManager.getSnapshotInfoTable().iterator()) {
       while (iter.hasNext()) {

Review Comment:
   No during bootstrap even the deleted snapshots are copied. SNAPSHOT_DELETED is an intermediate state. When a snapshot is purged that is when the entry is completely removed from the system. Then the snapshotInfo table won't have the entries in checkpoint



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9335:
URL: https://github.com/apache/ozone/pull/9335#issuecomment-3583479147

   thank you @smengcl and @jojochuang  for reviewing the patch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9335:
URL: https://github.com/apache/ozone/pull/9335#discussion_r2564537380


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java:
##########
@@ -399,12 +436,12 @@ private OzoneConfiguration getConf() {
    * Collects paths to all snapshot databases from the OM DB.
    *
    * @param activeOMMetadataManager OMMetadataManager instance
-   * @return Set of paths to snapshot databases
+   * @return Map of paths to snapshot databases with snapshot IDs as keys.
    * @throws IOException if an I/O error occurs
    */
-  Set<Path> getSnapshotDirsFromDB(OMMetadataManager activeOMMetadataManager, OMMetadataManager omMetadataManager,
+  Map<UUID, Path> getSnapshotDirsFromDB(OMMetadataManager activeOMMetadataManager, OMMetadataManager omMetadataManager,
       OmSnapshotLocalDataManager localDataManager) throws IOException {
-    Set<Path> snapshotPaths = new HashSet<>();
+    Map<UUID, Path> snapshotPaths = new HashMap<>();
     try (TableIterator<String, ? extends Table.KeyValue<String, SnapshotInfo>> iter =
         omMetadataManager.getSnapshotInfoTable().iterator()) {
       while (iter.hasNext()) {

Review Comment:
   No during bootstrap even the deleted snapshots are copied. SNAPSHOT_DELETED is an intermediate state. When a snapshot is purged that is when the entry is completely removed from the system.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #9335:
URL: https://github.com/apache/ozone/pull/9335#discussion_r2566231743


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java:
##########
@@ -309,6 +317,36 @@ public void writeDbDataToStream(HttpServletRequest request, OutputStream destina
     }
   }
 
+  /**
+   * Retrieves the paths to the local property YAML files for the specified snapshot IDs.
+   * This method resolves the chain of previous snapshot references for each snapshot ID
+   * and gathers their corresponding local property YAML file paths.
+   *
+   * @param localDataManager The OmSnapshotLocalDataManager instance responsible for managing
+   *                         snapshot data and metadata.
+   * @param snapshotIds      A set of snapshot IDs for which the local property YAML file
+   *                         paths should be resolved.
+   * @return A collection of paths to the local property YAML files for the specified
+   *         snapshot IDs.
+   */
+  private Collection<Path> getSnapshotLocalDataPaths(OmSnapshotLocalDataManager localDataManager,
+      Set<UUID> snapshotIds) {
+    Set<UUID> snapshotLocalDataIds = new HashSet<>();
+    Map<UUID, OmSnapshotLocalDataManager.SnapshotVersionsMeta> versionNodeMap =
+        localDataManager.getVersionNodeMapUnmodifiable();
+    for (UUID snapshot : snapshotIds) {
+      UUID id = snapshot;
+      // Get the previous snapshot id for the current snapshot id until we reach null or the first snapshot id which
+      // is already in the snapshotLocalDataIds set.
+      while (id != null && !snapshotLocalDataIds.contains(id)) {
+        snapshotLocalDataIds.add(id);
+        id = versionNodeMap.get(id).getPreviousSnapshotId();
+      }
+    }
+    return snapshotLocalDataIds.stream().map(localDataManager::getSnapshotLocalPropertyYamlPath)
+        .map(Paths::get).collect(Collectors.toList());

Review Comment:
   Access of [element](1) annotated with VisibleForTesting found in production code.
   ```suggestion
       // Avoid using @VisibleForTesting method. Construct the path manually using public API.
       // Assume the local property YAML file is named "local_property.yaml" in the snapshot directory.
       return snapshotLocalDataIds.stream()
           .map(id -> Paths.get(localDataManager.getSnapshotDir(id).toString(), "local_property.yaml"))
           .collect(Collectors.toList());
   ```



##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServletInodeBasedXfer.java:
##########
@@ -309,6 +317,36 @@ public void writeDbDataToStream(HttpServletRequest request, OutputStream destina
     }
   }
 
+  /**
+   * Retrieves the paths to the local property YAML files for the specified snapshot IDs.
+   * This method resolves the chain of previous snapshot references for each snapshot ID
+   * and gathers their corresponding local property YAML file paths.
+   *
+   * @param localDataManager The OmSnapshotLocalDataManager instance responsible for managing
+   *                         snapshot data and metadata.
+   * @param snapshotIds      A set of snapshot IDs for which the local property YAML file
+   *                         paths should be resolved.
+   * @return A collection of paths to the local property YAML files for the specified
+   *         snapshot IDs.
+   */
+  private Collection<Path> getSnapshotLocalDataPaths(OmSnapshotLocalDataManager localDataManager,
+      Set<UUID> snapshotIds) {
+    Set<UUID> snapshotLocalDataIds = new HashSet<>();
+    Map<UUID, OmSnapshotLocalDataManager.SnapshotVersionsMeta> versionNodeMap =
+        localDataManager.getVersionNodeMapUnmodifiable();
+    for (UUID snapshot : snapshotIds) {
+      UUID id = snapshot;
+      // Get the previous snapshot id for the current snapshot id until we reach null or the first snapshot id which
+      // is already in the snapshotLocalDataIds set.
+      while (id != null && !snapshotLocalDataIds.contains(id)) {
+        snapshotLocalDataIds.add(id);
+        id = versionNodeMap.get(id).getPreviousSnapshotId();

Review Comment:
   Potential NullPointerException: `versionNodeMap.get(id)` can return null if the snapshot ID is not present in the version node map. This will cause a NPE when calling `.getPreviousSnapshotId()`. Consider adding a null check:
   ```java
   OmSnapshotLocalDataManager.SnapshotVersionsMeta meta = versionNodeMap.get(id);
   if (meta == null) {
     break; // or handle appropriately
   }
   id = meta.getPreviousSnapshotId();
   ```
   ```suggestion
           OmSnapshotLocalDataManager.SnapshotVersionsMeta meta = versionNodeMap.get(id);
           if (meta == null) {
             break;
           }
           id = meta.getPreviousSnapshotId();
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #9335:
URL: https://github.com/apache/ozone/pull/9335#discussion_r2566524487


##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMDbCheckpointServletInodeBasedXfer.java:
##########
@@ -767,27 +771,33 @@ public void testCheckpointIncludesSnapshotsFromFrozenState() throws Exception {
         .filter(snap -> snap.getName().equals("snapshot1"))
         .findFirst()
         .orElseThrow(() -> new RuntimeException("snapshot1 not found"));
-
     // Setup servlet mocks for checkpoint processing
     setupMocks();
     when(requestMock.getParameter(OZONE_DB_CHECKPOINT_INCLUDE_SNAPSHOT_DATA)).thenReturn("true");
 
     // Create a checkpoint that captures current state (S1)
-    DBStore dbStore = om.getMetadataManager().getStore();
-    DBStore spyDbStore = spy(dbStore);
-    AtomicReference<DBCheckpoint> capturedCheckpoint = new AtomicReference<>();
+    DBStore spyDbStore = spy(om.getMetadataManager().getStore());
 
-    when(spyDbStore.getCheckpoint(true)).thenAnswer(invocation -> {
-      // Purge snapshot2 before checkpoint
+    AtomicReference<DBCheckpoint> capturedCheckpoint = new AtomicReference<>();
+    SnapshotCache spySnapshotCache = spy(om.getOmSnapshotManager().getSnapshotCache());
+    OmSnapshotManager spySnapshotManager = spy(om.getOmSnapshotManager());
+    when(om.getOmSnapshotManager()).thenReturn(spySnapshotManager);
+    when(spySnapshotManager.getSnapshotCache()).thenReturn(spySnapshotCache);
+    // Mock the snapshot cache to create a snapshot2 just after taking a snapshot cache lock.
+    doAnswer(invocationOnMock -> {
+      Object ret = invocationOnMock.callRealMethod();
       // create snapshot 3 before checkpoint

Review Comment:
   This comment no longer applies
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-13967. OM bootstrap lock should acquire a lock on SnapshotLocalDataManager before copying the local data files [ozone]

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi merged PR #9335:
URL: https://github.com/apache/ozone/pull/9335


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org