You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "DaveTeng0 (via GitHub)" <gi...@apache.org> on 2023/05/12 17:26:15 UTC

[GitHub] [ozone] DaveTeng0 opened a new pull request, #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

DaveTeng0 opened a new pull request, #4708:
URL: https://github.com/apache/ozone/pull/4708

   ## What changes were proposed in this pull request?
   Prevent writes to keys that start with ".snapshot"
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/HDDS-7168
   
   ## How was this patch tested?
   - Unit tests
   - Manual test in a docker ozone cluster.
   I tried command "ozone sh key put vol1/bucket1/.snapshot/dir1/dir2/k1  tmpFile"
   The response is expected as follows
   """ INVALID_KEY_NAME Can not use key name starting with prefix: .snapshot """
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] hemantk-12 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1204562215


##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java:
##########
@@ -612,6 +612,21 @@ public static void validateKeyName(String keyName)
     }
   }
 
+  /**
+   * Verify if key name contains invalid snapshot reserved word or not.

Review Comment:
   nit:
   ```suggestion
      * Verify if key name contains snapshot reserved word.
   ```



##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java:
##########
@@ -612,6 +612,21 @@ public static void validateKeyName(String keyName)
     }
   }
 
+  /**
+   * Verify if key name contains invalid snapshot reserved word or not.
+   * This verification will run even when
+   * ozone.om.keyname.character.check.enabled sets to false
+   */
+  public static void verifyKeyNameWithSnapshotReservedWord(String keyName)
+          throws OMException {
+    if (keyName.startsWith(OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX)) {

Review Comment:
   1. Can `keyName` be null? 
   2. Should it be `String.contains()` or `String.startsWith()`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1200831000


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java:
##########
@@ -94,6 +96,12 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createFileRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName()

Review Comment:
   yup nice catch, moved to HddsClientUtils, and reuse!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1205876545


##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java:
##########
@@ -612,6 +612,21 @@ public static void validateKeyName(String keyName)
     }
   }
 
+  /**
+   * Verify if key name contains invalid snapshot reserved word or not.

Review Comment:
   yup thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1201176179


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java:
##########
@@ -71,6 +71,8 @@
 import static org.apache.hadoop.ozone.om.exceptions.OMException.ResultCodes.NOT_A_FILE;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.DIRECTORY_EXISTS;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.FILE_EXISTS_IN_GIVENPATH;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_KEY_PREFIX;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_SNAPSHOT_INDICATOR;

Review Comment:
   oh yes removed!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1200829119


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCreateRequest.java:
##########
@@ -629,6 +631,29 @@ public void testKeyCreateWithFileSystemPathsEnabled() throws Exception {
 
   }
 
+  @Test
+  public void testPreExecuteWithInvalidKeyPrefix() throws Exception {
+
+    KeyArgs.Builder keyArgs = KeyArgs.newBuilder()
+        .setVolumeName(volumeName).setBucketName(bucketName)

Review Comment:
   yup added!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1205999114


##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java:
##########
@@ -612,6 +612,21 @@ public static void validateKeyName(String keyName)
     }
   }
 
+  /**
+   * Verify if key name contains invalid snapshot reserved word or not.

Review Comment:
   updated!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] swamirishi commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1195823429


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequest.java:
##########
@@ -334,6 +337,30 @@ public void testValidateAndUpdateCacheWithNonRecursiveAndOverWrite()
     testNonRecursivePath(key, false, false, true);
   }
 
+  @Test
+  public void testPreExecuteWithInvalidKeyPrefix() throws Exception {
+    String[] invalidKeyNames = {OM_SNAPSHOT_INDICATOR + "/" + keyName,
+        OM_SNAPSHOT_INDICATOR + "/a/" + keyName,

Review Comment:
   Can you also add a test case with .snapshotabc/a/b/c, this should pass. This should be a valid key



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] hemantk-12 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1200883797


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java:
##########
@@ -71,6 +71,8 @@
 import static org.apache.hadoop.ozone.om.exceptions.OMException.ResultCodes.NOT_A_FILE;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.DIRECTORY_EXISTS;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.FILE_EXISTS_IN_GIVENPATH;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_KEY_PREFIX;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_SNAPSHOT_INDICATOR;

Review Comment:
   Same as above.



##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java:
##########
@@ -72,6 +72,8 @@
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.DIRECTORY_EXISTS;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.FILE_EXISTS_IN_GIVENPATH;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.FILE_EXISTS;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_KEY_PREFIX;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_SNAPSHOT_INDICATOR;

Review Comment:
   Unused imports.



##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -223,6 +226,11 @@ public static void verifyKeyName(String keyName) {
     if (keyName == null) {
       throw new IllegalArgumentException("Key name is null");
     }
+    if (keyName.startsWith(OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX)) {

Review Comment:
   I think you have to move out this to separate check because [validateKeyName](https://github.com/apache/ozone/blob/3dd6e29f2deb1d27aef2e82fcf9ef962e996cc03/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java#L596) only runs when [checkKeyNameEnabled](https://github.com/apache/ozone/blob/e9592d2654e86ebc64648195e1578578eda3f389/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java#L100).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] hemantk-12 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1198365304


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java:
##########
@@ -94,6 +96,12 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createFileRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName()

Review Comment:
   nit: this can be moved to `OmUtils` similar to [validateKeyName](https://github.com/apache/ozone/blob/3dd6e29f2deb1d27aef2e82fcf9ef962e996cc03/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java#L596) and reused it here and in `OMKeyCreateRequest`?



##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequest.java:
##########
@@ -334,6 +337,30 @@ public void testValidateAndUpdateCacheWithNonRecursiveAndOverWrite()
     testNonRecursivePath(key, false, false, true);
   }
 
+  @Test
+  public void testPreExecuteWithInvalidKeyPrefix() throws Exception {
+    String[] invalidKeyNames = {OM_SNAPSHOT_INDICATOR + "/" + keyName,
+        OM_SNAPSHOT_INDICATOR + "/a/" + keyName,

Review Comment:
   Also `.snapshot` as middle dir.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1200839878


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCreateRequest.java:
##########
@@ -138,81 +140,94 @@ public void preExecuteRejectsInvalidReplication() {
   public void testValidateAndUpdateCache() throws Exception {
     when(ozoneManager.getOzoneLockProvider()).thenReturn(
         new OzoneLockProvider(keyPathLockEnabled, enableFileSystemPaths));
-    OMRequest modifiedOmRequest =
-        doPreExecute(createKeyRequest(false, 0));
 
-    OMKeyCreateRequest omKeyCreateRequest =
-            getOMKeyCreateRequest(modifiedOmRequest);
+    String[] validKeyNames = {
+        keyName,
+        OM_SNAPSHOT_INDICATOR + "abc/" + keyName,
+        "a/" + OM_SNAPSHOT_INDICATOR + "/b/c/" + keyName
+    };
+    for (String validKeyName : validKeyNames) {

Review Comment:
   The only changes I added in this tests is just to create a String array of validKeyNames, then iterate them through as keyName in original test. (also add indent to original code block due to the new for-loop)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] smengcl commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1192807929


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java:
##########
@@ -93,6 +94,10 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createKeyRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName().startsWith(OM_SNAPSHOT_INDICATOR)) {
+      throw new OMException("Can not use key name starting with prefix: "
+          + OM_SNAPSHOT_INDICATOR, OMException.ResultCodes.INVALID_KEY_NAME);

Review Comment:
   Yup. Need `OM_KEY_PREFIX` as well.
   
   ```suggestion
       if (keyArgs.getKeyName().startsWith(OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX)) {
         throw new OMException("Cannot create key under path reserved for snapshot path: "
             + OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX, OMException.ResultCodes.INVALID_KEY_NAME);
   ```



##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java:
##########
@@ -93,6 +94,10 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createKeyRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName().startsWith(OM_SNAPSHOT_INDICATOR)) {
+      throw new OMException("Can not use key name starting with prefix: "
+          + OM_SNAPSHOT_INDICATOR, OMException.ResultCodes.INVALID_KEY_NAME);

Review Comment:
   Yup. Need `OM_KEY_PREFIX` as well.
   
   ```suggestion
       if (keyArgs.getKeyName().startsWith(OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX)) {
         throw new OMException("Cannot create key under path reserved for snapshot: "
             + OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX, OMException.ResultCodes.INVALID_KEY_NAME);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1194101724


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java:
##########
@@ -93,6 +94,10 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createKeyRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName().startsWith(OM_SNAPSHOT_INDICATOR)) {

Review Comment:
   oh sure!



##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java:
##########
@@ -93,6 +94,10 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createKeyRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName().startsWith(OM_SNAPSHOT_INDICATOR)) {
+      throw new OMException("Can not use key name starting with prefix: "
+          + OM_SNAPSHOT_INDICATOR, OMException.ResultCodes.INVALID_KEY_NAME);

Review Comment:
   yes sure!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] prashantpogde merged pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "prashantpogde (via GitHub)" <gi...@apache.org>.
prashantpogde merged PR #4708:
URL: https://github.com/apache/ozone/pull/4708


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] swamirishi commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1195823680


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyCreateRequest.java:
##########
@@ -629,6 +631,29 @@ public void testKeyCreateWithFileSystemPathsEnabled() throws Exception {
 
   }
 
+  @Test
+  public void testPreExecuteWithInvalidKeyPrefix() throws Exception {
+
+    KeyArgs.Builder keyArgs = KeyArgs.newBuilder()
+        .setVolumeName(volumeName).setBucketName(bucketName)

Review Comment:
   Similarly add a test case for valid keys as well



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1200840515


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequest.java:
##########
@@ -115,57 +118,69 @@ public void testPreExecuteWithBlankKey() throws Exception {
 
   @Test
   public void testValidateAndUpdateCache() throws Exception {
-    OMRequest omRequest = createFileRequest(volumeName, bucketName, keyName,
-        HddsProtos.ReplicationFactor.ONE, HddsProtos.ReplicationType.RATIS,
-        false, true);
 
-    OMRequestTestUtils.addVolumeAndBucketToDB(volumeName, bucketName,
-        omMetadataManager, getBucketLayout());
-    OMFileCreateRequest omFileCreateRequest = getOMFileCreateRequest(omRequest);
+    String[] validKeyNames = {
+        keyName,
+        OM_SNAPSHOT_INDICATOR + "abc/" + keyName,
+        "a/" + OM_SNAPSHOT_INDICATOR + "/b/c/" + keyName
+    };
 
-    OMRequest modifiedOmRequest = omFileCreateRequest.preExecute(ozoneManager);
+    for (String validKeyName : validKeyNames) {
+      keyName = validKeyName;

Review Comment:
   The only changes I added in this tests is just to create a String array of validKeyNames, then iterate them through as keyName in original test. (also add indent to original code block due to the new for-loop)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1201170859


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java:
##########
@@ -72,6 +72,8 @@
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.DIRECTORY_EXISTS;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.FILE_EXISTS_IN_GIVENPATH;
 import static org.apache.hadoop.ozone.om.request.file.OMFileRequest.OMDirectoryResult.FILE_EXISTS;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_KEY_PREFIX;
+import static org.apache.hadoop.ozone.OzoneConsts.OM_SNAPSHOT_INDICATOR;

Review Comment:
   oh yes removed!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1200828831


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequest.java:
##########
@@ -334,6 +337,30 @@ public void testValidateAndUpdateCacheWithNonRecursiveAndOverWrite()
     testNonRecursivePath(key, false, false, true);
   }
 
+  @Test
+  public void testPreExecuteWithInvalidKeyPrefix() throws Exception {
+    String[] invalidKeyNames = {OM_SNAPSHOT_INDICATOR + "/" + keyName,
+        OM_SNAPSHOT_INDICATOR + "/a/" + keyName,

Review Comment:
   yup added!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] hemantk-12 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "hemantk-12 (via GitHub)" <gi...@apache.org>.
hemantk-12 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1198365304


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/file/OMFileCreateRequest.java:
##########
@@ -94,6 +96,12 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createFileRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName()

Review Comment:
   nit: this can be moved to `OmUtils` similar to [validateKeyName](https://github.com/apache/ozone/blob/3dd6e29f2deb1d27aef2e82fcf9ef962e996cc03/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java#L596) and reused it here and in `OMKeyCreateRequest`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] swamirishi commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1192669362


##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCreateRequest.java:
##########
@@ -93,6 +94,10 @@ public OMRequest preExecute(OzoneManager ozoneManager) throws IOException {
     KeyArgs keyArgs = createKeyRequest.getKeyArgs();
 
     // Verify key name
+    if (keyArgs.getKeyName().startsWith(OM_SNAPSHOT_INDICATOR)) {

Review Comment:
   shouldn't it be OM_SNAPSHOT_INDICATOR + "/". I guess we should allow keynames like ".snapshotabc/key1blah...blah...blah" @smengcl @prashantpogde correct me if I am wrong.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1201250889


##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -223,6 +226,11 @@ public static void verifyKeyName(String keyName) {
     if (keyName == null) {
       throw new IllegalArgumentException("Key name is null");
     }
+    if (keyName.startsWith(OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX)) {

Review Comment:
   yup! moved to another separated method in OmUtils.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1201539685


##########
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/file/TestOMFileCreateRequestWithFSO.java:
##########
@@ -156,6 +157,18 @@ public void testValidateAndUpdateCacheWithNonRecursiveAndOverWrite()
     testNonRecursivePath(key, false, false, true);
   }
 
+  @Test
+  public void testValidateAndUpdateCacheWithSnapshotReservedWord()
+      throws Exception {
+    String[] validPaths = {
+        "a/b/" + OM_SNAPSHOT_INDICATOR + "c/d",
+        "a/b/c/" + OM_SNAPSHOT_INDICATOR + "/d"

Review Comment:
   For the first two levels of path ("a/b/c/d"), ozone assumes the first level (a) as volume, the second level (b) as bucket, rest of levels are directory and the last level is file name.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on a diff in pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "DaveTeng0 (via GitHub)" <gi...@apache.org>.
DaveTeng0 commented on code in PR #4708:
URL: https://github.com/apache/ozone/pull/4708#discussion_r1205880365


##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java:
##########
@@ -612,6 +612,21 @@ public static void validateKeyName(String keyName)
     }
   }
 
+  /**
+   * Verify if key name contains invalid snapshot reserved word or not.
+   * This verification will run even when
+   * ozone.om.keyname.character.check.enabled sets to false
+   */
+  public static void verifyKeyNameWithSnapshotReservedWord(String keyName)
+          throws OMException {
+    if (keyName.startsWith(OM_SNAPSHOT_INDICATOR + OM_KEY_PREFIX)) {

Review Comment:
   1. yeah it's possible, it's better to check null here.
   2. Hmm should be startsWith(). Since we might have valid key name like 'abc.snapshot/'..



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] prashantpogde commented on pull request #4708: HDDS-7168. Prevent writes to keys that start with .snapshot

Posted by "prashantpogde (via GitHub)" <gi...@apache.org>.
prashantpogde commented on PR #4708:
URL: https://github.com/apache/ozone/pull/4708#issuecomment-1570880581

   Thank you @DaveTeng0 for this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org