You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ashishkumar50 (via GitHub)" <gi...@apache.org> on 2024/02/21 06:56:05 UTC
[PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
ashishkumar50 opened a new pull request, #6244:
URL: https://github.com/apache/ozone/pull/6244
## What changes were proposed in this pull request?
When enableFileSystemPath is true and bucket layout is legacy, during key delete operation in normalizekey(prexecute) it removes trailing slash, but in DB key is present with trailing slash. And because of this KEY_NOT_FOUND error comes.
But fixing legacy bucket directory deletion using sh can lead to other issues. In this PR we are restricting legacy bucket directory deletion using sh command.
More discussion in [PR](https://github.com/apache/ozone/pull/6218).
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-10397
## How was this patch tested?
New Integration test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on code in PR #6244:
URL: https://github.com/apache/ozone/pull/6244#discussion_r1497611121
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java:
##########
@@ -1984,6 +1986,48 @@ public void testLinkedAndNonLinkedBucketMetaData()
out.reset();
}
+ @Test
+ public void testKeyDeleteLegacyWithEnableFileSystemPath() throws IOException {
+ String volumeName = "vol5";
+ String bucketName = "legacybucket";
+ String[] args = new String[] {"volume", "create", "o3://" + omServiceId + OZONE_URI_DELIMITER + volumeName};
+ execute(ozoneShell, args);
+
+ args = new String[] {"bucket", "create", "o3://" + omServiceId + OZONE_URI_DELIMITER +
+ volumeName + OZONE_URI_DELIMITER + bucketName, "--layout", BucketLayout.LEGACY.toString()};
+ execute(ozoneShell, args);
+
+ String dirPath = OZONE_URI_DELIMITER + volumeName + OZONE_URI_DELIMITER +
+ bucketName + OZONE_URI_DELIMITER + "dir/";
+ String keyPath = dirPath + "key1";
+
+ // Create key, it will generate two keys, one with dirPath other with keyPath
+ args = new String[] {"key", "put", "o3://" + omServiceId + keyPath, testFile.getPath()};
+ execute(ozoneShell, args);
+
+ // Enable fileSystem path for client config
+ String fileSystemEnable = generateSetConfString(OMConfigKeys.OZONE_OM_ENABLE_FILESYSTEM_PATHS, "true");
+ // Delete dirPath key, it should fail
+ args = new String[] {fileSystemEnable, "key", "delete", dirPath};
+ execute(ozoneShell, args);
+
+ // Check number of keys
+ args = new String[] {"key", "list", "o3://" + omServiceId + OZONE_URI_DELIMITER +
+ volumeName + OZONE_URI_DELIMITER + bucketName};
+ out.reset();
+ execute(ozoneShell, args);
Review Comment:
The above 3 lines are not needed as we are already checking key list size below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
Posted by "ashishkumar50 (via GitHub)" <gi...@apache.org>.
ashishkumar50 commented on code in PR #6244:
URL: https://github.com/apache/ozone/pull/6244#discussion_r1498645321
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java:
##########
@@ -1984,6 +1986,48 @@ public void testLinkedAndNonLinkedBucketMetaData()
out.reset();
}
+ @Test
+ public void testKeyDeleteLegacyWithEnableFileSystemPath() throws IOException {
+ String volumeName = "vol5";
+ String bucketName = "legacybucket";
+ String[] args = new String[] {"volume", "create", "o3://" + omServiceId + OZONE_URI_DELIMITER + volumeName};
+ execute(ozoneShell, args);
+
+ args = new String[] {"bucket", "create", "o3://" + omServiceId + OZONE_URI_DELIMITER +
+ volumeName + OZONE_URI_DELIMITER + bucketName, "--layout", BucketLayout.LEGACY.toString()};
+ execute(ozoneShell, args);
+
+ String dirPath = OZONE_URI_DELIMITER + volumeName + OZONE_URI_DELIMITER +
+ bucketName + OZONE_URI_DELIMITER + "dir/";
+ String keyPath = dirPath + "key1";
+
+ // Create key, it will generate two keys, one with dirPath other with keyPath
+ args = new String[] {"key", "put", "o3://" + omServiceId + keyPath, testFile.getPath()};
+ execute(ozoneShell, args);
+
+ // Enable fileSystem path for client config
+ String fileSystemEnable = generateSetConfString(OMConfigKeys.OZONE_OM_ENABLE_FILESYSTEM_PATHS, "true");
+ // Delete dirPath key, it should fail
+ args = new String[] {fileSystemEnable, "key", "delete", dirPath};
+ execute(ozoneShell, args);
+
+ // Check number of keys
+ args = new String[] {"key", "list", "o3://" + omServiceId + OZONE_URI_DELIMITER +
+ volumeName + OZONE_URI_DELIMITER + bucketName};
+ out.reset();
+ execute(ozoneShell, args);
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6244:
URL: https://github.com/apache/ozone/pull/6244#issuecomment-1956154475
If you want to get feedback on the code without running CI checks, or you know code is work-in-progress, you can create the PR as draft to skip most checks. Otherwise, if no big changes are expected, running CI checks before review is fine, no need to cancel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
Posted by "ashishkumar50 (via GitHub)" <gi...@apache.org>.
ashishkumar50 commented on PR #6244:
URL: https://github.com/apache/ozone/pull/6244#issuecomment-1956146194
> @ashishkumar50 was CI run cancelled intentionally?
Yes, I thought of after review let it run. It's already passed in my fork branch.
I have run it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
Posted by "ashishkumar50 (via GitHub)" <gi...@apache.org>.
ashishkumar50 commented on PR #6244:
URL: https://github.com/apache/ozone/pull/6244#issuecomment-1956008153
@sadanand48 Please help to review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #6244:
URL: https://github.com/apache/ozone/pull/6244#issuecomment-1956072101
@ashishkumar50 was CI run cancelled intentionally?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-10397. Restrict legacy bucket directory deletion through sh command. [ozone]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 merged PR #6244:
URL: https://github.com/apache/ozone/pull/6244
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org