You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "jojochuang (via GitHub)" <gi...@apache.org> on 2026/06/18 22:17:51 UTC
[PR] HDDS-14435. Fix snapshotUsedNamespace underflow when FSO directory is deleted and purged [ozone]
jojochuang opened a new pull request, #10545:
URL: https://github.com/apache/ozone/pull/10545
## What changes were proposed in this pull request?
HDDS-14435. Fix snapshotUsedNamespace underflow when FSO directory is deleted and purged
Please describe your PR in detail:
• The Problem: FSO directories have no blocks, so directory deletion incorrectly evaluated to isKeyNonEmpty = false . This decremented the active
bucket namespace usedNamespace but did not increment snapshotUsedNamespace , causing a mismatch when the directory was later purged (unconditionally
decrementing snapshotUsedNamespace by 1 and underflowing the count to -1 ).
• The Fix: Updated OMKeyDeleteRequestWithFSO.java to shift the namespace quota to the snapshot namespace on deletion if it is either a non-empty key or a
directory (
isKeyNonEmpty || keyStatus.isDirectory() ).
• Test Utils Alignment: Updated mock deleteDir helper in OMRequestTestUtils.java to align with actual deletion quota tracking.
• Regression Test: Added a test case testSnapshotUsedNamespaceAfterDirectoryDeleteAndPurge in TestOMKeyDeleteRequestWithFSO.java to verify that
directory deletion
followed by a purge does not cause negative snapshotUsedNamespace values.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14435
## How was this patch tested?
New unit test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15650. Fix snapshotUsedNamespace underflow when FSO directory is deleted and purged [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #10545:
URL: https://github.com/apache/ozone/pull/10545#discussion_r3462109525
##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyDeleteRequestWithFSO.java:
##########
@@ -333,4 +334,72 @@ public void testDeleteParentAfterChildDeleted() throws Exception {
assertEquals(OzoneManagerProtocolProtos.Status.OK, response.getOMResponse().getStatus(),
"Parent delete should succeed after children deleted");
}
+
+ @Test
+ public void testSnapshotUsedNamespaceAfterDirectoryDeleteAndPurge() throws Exception {
+ OMRequestTestUtils.addVolumeAndBucketToDB(volumeName, bucketName, omMetadataManager, getBucketLayout());
+
+ String dirName = "dir1";
+ String dirKeyPath = addKeyToDirTable(volumeName, bucketName, dirName);
+
+ long parentObjectID = 0L;
+ long dirObjectID = 12345L;
+ OmDirectoryInfo omDirectoryInfo = OMRequestTestUtils.createOmDirectoryInfo(dirName, dirObjectID, parentObjectID);
+ omMetadataManager.getDirectoryTable().put(dirKeyPath, omDirectoryInfo);
+
+ String bucketKey = omMetadataManager.getBucketKey(volumeName, bucketName);
+ OmBucketInfo omBucketInfo = omMetadataManager.getBucketTable().get(bucketKey);
+ assertNotNull(omBucketInfo);
+ // Initialize used namespace and snapshot used namespace for test predictability
+ omBucketInfo.incrUsedNamespace(1);
+ omMetadataManager.getBucketTable().put(bucketKey, omBucketInfo);
+
+ // Delete the directory
+ long txnId = 100L;
+ OMRequest deleteRequest = doPreExecute(createDeleteKeyRequest(dirName, false));
+ OMKeyDeleteRequest omKeyDeleteRequest = getOmKeyDeleteRequest(deleteRequest);
+ OMClientResponse deleteResponse = omKeyDeleteRequest.validateAndUpdateCache(ozoneManager, txnId++);
+ assertEquals(OzoneManagerProtocolProtos.Status.OK, deleteResponse.getOMResponse().getStatus());
+
+ OmBucketInfo bucketInfoAfterDelete = omMetadataManager.getBucketTable().get(bucketKey);
+
+ // Perform purge
+ OzoneManagerProtocolProtos.PurgeDirectoriesRequest.Builder purgeDirRequest =
+ OzoneManagerProtocolProtos.PurgeDirectoriesRequest.newBuilder();
+
+ long volumeId = omMetadataManager.getVolumeId(volumeName);
+ long bucketId = bucketInfoAfterDelete.getObjectID();
+
+ OzoneManagerProtocolProtos.PurgePathRequest purgePathRequest =
+ OzoneManagerProtocolProtos.PurgePathRequest.newBuilder()
+ .setVolumeId(volumeId)
+ .setBucketId(bucketId)
+ .setDeletedDir(dirKeyPath)
+ .build();
+
+ purgeDirRequest.addDeletedPath(purgePathRequest);
+ purgeDirRequest.addBucketNameInfos(
+ OzoneManagerProtocolProtos.BucketNameInfo.newBuilder()
+ .setVolumeName(volumeName)
+ .setBucketName(bucketName)
+ .setBucketId(bucketId)
+ .setVolumeId(volumeId)
+ .build());
+
+ OMRequest purgeRequest = OMRequest.newBuilder()
+ .setCmdType(OzoneManagerProtocolProtos.Type.PurgeDirectories)
+ .setPurgeDirectoriesRequest(purgeDirRequest)
+ .setClientId(UUID.randomUUID().toString())
+ .build();
+
+ OMDirectoriesPurgeRequestWithFSO omPurgeRequest = new OMDirectoriesPurgeRequestWithFSO(purgeRequest);
+ OMClientResponse purgeResponse = omPurgeRequest.validateAndUpdateCache(ozoneManager, txnId);
+ assertEquals(OzoneManagerProtocolProtos.Status.OK, purgeResponse.getOMResponse().getStatus());
+
+ OmBucketInfo bucketInfoAfterPurge = omMetadataManager.getBucketTable().get(bucketKey);
+
+ // We expect snapshotUsedNamespace to not go negative
+ assertTrue(bucketInfoAfterPurge.getSnapshotUsedNamespace() >= 0,
+ "SnapshotUsedNamespace went negative (" + bucketInfoAfterPurge.getSnapshotUsedNamespace() + ") due to bug.");
Review Comment:
ah good point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15650. Fix snapshotUsedNamespace underflow when FSO directory is deleted and purged [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #10545:
URL: https://github.com/apache/ozone/pull/10545
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15650. Fix snapshotUsedNamespace underflow when FSO directory is deleted and purged [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #10545:
URL: https://github.com/apache/ozone/pull/10545#discussion_r3462095780
##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyDeleteRequestWithFSO.java:
##########
@@ -333,4 +334,72 @@ public void testDeleteParentAfterChildDeleted() throws Exception {
assertEquals(OzoneManagerProtocolProtos.Status.OK, response.getOMResponse().getStatus(),
"Parent delete should succeed after children deleted");
}
+
+ @Test
+ public void testSnapshotUsedNamespaceAfterDirectoryDeleteAndPurge() throws Exception {
+ OMRequestTestUtils.addVolumeAndBucketToDB(volumeName, bucketName, omMetadataManager, getBucketLayout());
+
+ String dirName = "dir1";
+ String dirKeyPath = addKeyToDirTable(volumeName, bucketName, dirName);
+
+ long parentObjectID = 0L;
+ long dirObjectID = 12345L;
+ OmDirectoryInfo omDirectoryInfo = OMRequestTestUtils.createOmDirectoryInfo(dirName, dirObjectID, parentObjectID);
+ omMetadataManager.getDirectoryTable().put(dirKeyPath, omDirectoryInfo);
+
+ String bucketKey = omMetadataManager.getBucketKey(volumeName, bucketName);
+ OmBucketInfo omBucketInfo = omMetadataManager.getBucketTable().get(bucketKey);
+ assertNotNull(omBucketInfo);
+ // Initialize used namespace and snapshot used namespace for test predictability
+ omBucketInfo.incrUsedNamespace(1);
+ omMetadataManager.getBucketTable().put(bucketKey, omBucketInfo);
+
+ // Delete the directory
+ long txnId = 100L;
+ OMRequest deleteRequest = doPreExecute(createDeleteKeyRequest(dirName, false));
+ OMKeyDeleteRequest omKeyDeleteRequest = getOmKeyDeleteRequest(deleteRequest);
+ OMClientResponse deleteResponse = omKeyDeleteRequest.validateAndUpdateCache(ozoneManager, txnId++);
+ assertEquals(OzoneManagerProtocolProtos.Status.OK, deleteResponse.getOMResponse().getStatus());
+
+ OmBucketInfo bucketInfoAfterDelete = omMetadataManager.getBucketTable().get(bucketKey);
+
+ // Perform purge
+ OzoneManagerProtocolProtos.PurgeDirectoriesRequest.Builder purgeDirRequest =
+ OzoneManagerProtocolProtos.PurgeDirectoriesRequest.newBuilder();
+
+ long volumeId = omMetadataManager.getVolumeId(volumeName);
+ long bucketId = bucketInfoAfterDelete.getObjectID();
+
+ OzoneManagerProtocolProtos.PurgePathRequest purgePathRequest =
+ OzoneManagerProtocolProtos.PurgePathRequest.newBuilder()
+ .setVolumeId(volumeId)
+ .setBucketId(bucketId)
+ .setDeletedDir(dirKeyPath)
+ .build();
+
+ purgeDirRequest.addDeletedPath(purgePathRequest);
+ purgeDirRequest.addBucketNameInfos(
+ OzoneManagerProtocolProtos.BucketNameInfo.newBuilder()
+ .setVolumeName(volumeName)
+ .setBucketName(bucketName)
+ .setBucketId(bucketId)
+ .setVolumeId(volumeId)
+ .build());
+
+ OMRequest purgeRequest = OMRequest.newBuilder()
+ .setCmdType(OzoneManagerProtocolProtos.Type.PurgeDirectories)
+ .setPurgeDirectoriesRequest(purgeDirRequest)
+ .setClientId(UUID.randomUUID().toString())
+ .build();
+
+ OMDirectoriesPurgeRequestWithFSO omPurgeRequest = new OMDirectoriesPurgeRequestWithFSO(purgeRequest);
+ OMClientResponse purgeResponse = omPurgeRequest.validateAndUpdateCache(ozoneManager, txnId);
+ assertEquals(OzoneManagerProtocolProtos.Status.OK, purgeResponse.getOMResponse().getStatus());
+
+ OmBucketInfo bucketInfoAfterPurge = omMetadataManager.getBucketTable().get(bucketKey);
+
+ // We expect snapshotUsedNamespace to not go negative
+ assertTrue(bucketInfoAfterPurge.getSnapshotUsedNamespace() >= 0,
+ "SnapshotUsedNamespace went negative (" + bucketInfoAfterPurge.getSnapshotUsedNamespace() + ") due to bug.");
Review Comment:
nit: please use `assertThat(<number>).isNotNegative()` instead of `assertTrue(<number> >= 0)`, see HDDS-9951
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org