You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "octachoron (via GitHub)" <gi...@apache.org> on 2025/04/22 20:45:31 UTC

[PR] HDDS-12649. Include name in length validation error [ozone]

octachoron opened a new pull request, #8322:
URL: https://github.com/apache/ozone/pull/8322

   ## What changes were proposed in this pull request?
   
   In the client, in most cases when the specified volume or bucket name is shorter or longer than the allowed range, the resulting error does not include the name itself. This can make troubleshooting difficult in situations where the name is not implied by the context.
   
   This PR addresses the difficulty by adding the name itself, and the resource type (volume or bucket), to the message of the innermost `IllegalArgumentException`.
   
   In the case of buckets, also adding the volume name seems to have a disproportionate cost in terms of clarity (at each usage). Accordingly, this PR proposes to accept a bucket name as unique enough for debugging purposes. A user of the client may still choose to add this information as desired.
   
   Since adding the name to the message could theoretically be used as an attack vector through logging unsanitized user input verbatim, the length check is moved to after the character check as a precautionary measure.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-12649
   
   ## How was this patch tested?
   
   - Unit test for the new expectations about the message: includes the resource type and name, does not include the name if it fails the character check.
   - Manually through the CLI.
   - CI on fork: https://github.com/octachoron/ozone/actions/runs/14590841497


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #8322:
URL: https://github.com/apache/ozone/pull/8322


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "octachoron (via GitHub)" <gi...@apache.org>.
octachoron commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2064320301


##########
hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/client/TestHddsClientUtils.java:
##########
@@ -225,8 +232,38 @@ public void testVerifyResourceName() {
 
     for (String name : invalidNames) {
       assertThrows(IllegalArgumentException.class, () -> HddsClientUtils.verifyResourceName(name),
-          "Did not reject invalid string [" + name + "] as a name");
+          getInvalidNameMessage(name));
     }
+
+    // Message should include the name and resource type, if the name is of
+    // illegal length.
+    List<String> resourceTypes = ImmutableList.of("bucket", "volume");
+    for (int i = 0; i < 2; i++) {
+      String resType = resourceTypes.get(i);
+      String otherResType = resourceTypes.get(1 - i);
+      Throwable thrown = assertThrows(
+          IllegalArgumentException.class,
+          () -> HddsClientUtils.verifyResourceName(tooShort, resType, true),
+          getInvalidNameMessage(tooShort));
+
+      String message = thrown.getMessage();
+      assertNotNull(message);
+      assertTrue(message.contains(resType));
+      assertFalse(message.contains(otherResType));
+      assertTrue(thrown.getMessage().contains(tooShort));

Review Comment:
   Sounds like a significant improvement indeed. Thank you for including the ticket link too. Updated.
   
   _Edit:_ I will keep this in mind for the future as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "octachoron (via GitHub)" <gi...@apache.org>.
octachoron commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2064320301


##########
hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/client/TestHddsClientUtils.java:
##########
@@ -225,8 +232,38 @@ public void testVerifyResourceName() {
 
     for (String name : invalidNames) {
       assertThrows(IllegalArgumentException.class, () -> HddsClientUtils.verifyResourceName(name),
-          "Did not reject invalid string [" + name + "] as a name");
+          getInvalidNameMessage(name));
     }
+
+    // Message should include the name and resource type, if the name is of
+    // illegal length.
+    List<String> resourceTypes = ImmutableList.of("bucket", "volume");
+    for (int i = 0; i < 2; i++) {
+      String resType = resourceTypes.get(i);
+      String otherResType = resourceTypes.get(1 - i);
+      Throwable thrown = assertThrows(
+          IllegalArgumentException.class,
+          () -> HddsClientUtils.verifyResourceName(tooShort, resType, true),
+          getInvalidNameMessage(tooShort));
+
+      String message = thrown.getMessage();
+      assertNotNull(message);
+      assertTrue(message.contains(resType));
+      assertFalse(message.contains(otherResType));
+      assertTrue(thrown.getMessage().contains(tooShort));

Review Comment:
   Sounds like a significant improvement indeed. Thank you for including the ticket link too. Updated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "octachoron (via GitHub)" <gi...@apache.org>.
octachoron commented on PR #8322:
URL: https://github.com/apache/ozone/pull/8322#issuecomment-2836995149

   Thank you @adoroszlai for the detailed review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2061191390


##########
hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/client/TestHddsClientUtils.java:
##########
@@ -225,8 +232,38 @@ public void testVerifyResourceName() {
 
     for (String name : invalidNames) {
       assertThrows(IllegalArgumentException.class, () -> HddsClientUtils.verifyResourceName(name),
-          "Did not reject invalid string [" + name + "] as a name");
+          getInvalidNameMessage(name));
     }
+
+    // Message should include the name and resource type, if the name is of
+    // illegal length.
+    List<String> resourceTypes = ImmutableList.of("bucket", "volume");
+    for (int i = 0; i < 2; i++) {
+      String resType = resourceTypes.get(i);
+      String otherResType = resourceTypes.get(1 - i);
+      Throwable thrown = assertThrows(
+          IllegalArgumentException.class,
+          () -> HddsClientUtils.verifyResourceName(tooShort, resType, true),
+          getInvalidNameMessage(tooShort));
+
+      String message = thrown.getMessage();
+      assertNotNull(message);
+      assertTrue(message.contains(resType));
+      assertFalse(message.contains(otherResType));
+      assertTrue(thrown.getMessage().contains(tooShort));
+    }
+
+    // However, such names also containing unsupported characters should not be
+    // logged verbatim. This is to avoid vulnerabilities like Log4Shell.
+    final String tooShortInvalidChar = "$a";
+    Throwable thrown = assertThrows(
+        IllegalArgumentException.class,
+        () -> HddsClientUtils.verifyResourceName(tooShortInvalidChar),
+        getInvalidNameMessage(tooShortInvalidChar));
+
+    String message = thrown.getMessage();
+    assertNotNull(message);
+    assertFalse(message.contains(tooShortInvalidChar));

Review Comment:
   Let's add these new test cases as separate two test methods.  I think `tooShort` can be moved, no need to test it twice.



##########
hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/client/TestHddsClientUtils.java:
##########
@@ -225,8 +232,38 @@ public void testVerifyResourceName() {
 
     for (String name : invalidNames) {
       assertThrows(IllegalArgumentException.class, () -> HddsClientUtils.verifyResourceName(name),
-          "Did not reject invalid string [" + name + "] as a name");
+          getInvalidNameMessage(name));
     }
+
+    // Message should include the name and resource type, if the name is of
+    // illegal length.
+    List<String> resourceTypes = ImmutableList.of("bucket", "volume");
+    for (int i = 0; i < 2; i++) {
+      String resType = resourceTypes.get(i);
+      String otherResType = resourceTypes.get(1 - i);
+      Throwable thrown = assertThrows(
+          IllegalArgumentException.class,
+          () -> HddsClientUtils.verifyResourceName(tooShort, resType, true),
+          getInvalidNameMessage(tooShort));
+
+      String message = thrown.getMessage();
+      assertNotNull(message);
+      assertTrue(message.contains(resType));
+      assertFalse(message.contains(otherResType));
+      assertTrue(thrown.getMessage().contains(tooShort));

Review Comment:
   Please use `assertThat` instead of `assertTrue`/`assertFalse` for better failure message.  (See HDDS-9951 for details.)



##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -71,7 +71,7 @@ private static void doNameChecks(String resName, String resType) {
     if (resName.length() < OzoneConsts.OZONE_MIN_BUCKET_NAME_LENGTH ||
         resName.length() > OzoneConsts.OZONE_MAX_BUCKET_NAME_LENGTH) {
       throw new IllegalArgumentException(resType +
-          " length is illegal, " + "valid length is 3-63 characters");
+          " name '" + resName + "' is of illegal length, " + "valid length is 3-63 characters");
     }

Review Comment:
   - I think we could further improve it by checking and rejecting too short and too long values separately.
   - I suggest changing "illegal" to "invalid".
   - Should we cap too long invalid value in the message at some reasonable length?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "octachoron (via GitHub)" <gi...@apache.org>.
octachoron commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2064314620


##########
hadoop-hdds/client/src/test/java/org/apache/hadoop/hdds/scm/client/TestHddsClientUtils.java:
##########
@@ -225,8 +232,38 @@ public void testVerifyResourceName() {
 
     for (String name : invalidNames) {
       assertThrows(IllegalArgumentException.class, () -> HddsClientUtils.verifyResourceName(name),
-          "Did not reject invalid string [" + name + "] as a name");
+          getInvalidNameMessage(name));
     }
+
+    // Message should include the name and resource type, if the name is of
+    // illegal length.
+    List<String> resourceTypes = ImmutableList.of("bucket", "volume");
+    for (int i = 0; i < 2; i++) {
+      String resType = resourceTypes.get(i);
+      String otherResType = resourceTypes.get(1 - i);
+      Throwable thrown = assertThrows(
+          IllegalArgumentException.class,
+          () -> HddsClientUtils.verifyResourceName(tooShort, resType, true),
+          getInvalidNameMessage(tooShort));
+
+      String message = thrown.getMessage();
+      assertNotNull(message);
+      assertTrue(message.contains(resType));
+      assertFalse(message.contains(otherResType));
+      assertTrue(thrown.getMessage().contains(tooShort));
+    }
+
+    // However, such names also containing unsupported characters should not be
+    // logged verbatim. This is to avoid vulnerabilities like Log4Shell.
+    final String tooShortInvalidChar = "$a";
+    Throwable thrown = assertThrows(
+        IllegalArgumentException.class,
+        () -> HddsClientUtils.verifyResourceName(tooShortInvalidChar),
+        getInvalidNameMessage(tooShortInvalidChar));
+
+    String message = thrown.getMessage();
+    assertNotNull(message);
+    assertFalse(message.contains(tooShortInvalidChar));

Review Comment:
   It crossed my mind the first time around. Sounds like I made the wrong call then. :slightly_smiling_face: Sure, it's split up now, thank you.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "octachoron (via GitHub)" <gi...@apache.org>.
octachoron commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2064325887


##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -71,7 +71,7 @@ private static void doNameChecks(String resName, String resType) {
     if (resName.length() < OzoneConsts.OZONE_MIN_BUCKET_NAME_LENGTH ||
         resName.length() > OzoneConsts.OZONE_MAX_BUCKET_NAME_LENGTH) {
       throw new IllegalArgumentException(resType +
-          " length is illegal, " + "valid length is 3-63 characters");
+          " name '" + resName + "' is of illegal length, " + "valid length is 3-63 characters");
     }

Review Comment:
   - Indeed! That would be better.
   - Yes, I think so too. If the special character check made sense, then this does as well.
   
   Thank you, added both.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2064103653


##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -48,6 +49,10 @@
 @InterfaceAudience.Public
 @InterfaceStability.Unstable
 public final class HddsClientUtils {
+  @VisibleForTesting
+  static final int MAX_BUCKET_NAME_LENGTH_IN_LOG =

Review Comment:
   ```suggestion
     static final int MAX_BUCKET_NAME_LENGTH_IN_LOG =
   ```
   
   I think we can omit the annotation.



##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -17,6 +17,7 @@
 
 package org.apache.hadoop.hdds.scm.client;
 
+import com.google.common.annotations.VisibleForTesting;

Review Comment:
   remove unused import
   
   ```suggestion
   ```



##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -68,10 +73,26 @@ private static void doNameChecks(String resName, String resType) {
       throw new IllegalArgumentException(resType + " name is null");
     }
 
-    if (resName.length() < OzoneConsts.OZONE_MIN_BUCKET_NAME_LENGTH ||
-        resName.length() > OzoneConsts.OZONE_MAX_BUCKET_NAME_LENGTH) {
+    if (resName.length() < OzoneConsts.OZONE_MIN_BUCKET_NAME_LENGTH) {
       throw new IllegalArgumentException(resType +
-          " length is illegal, " + "valid length is 3-63 characters");
+          " name '" + resName + "' is too short, " +
+          "valid length is 3-63 characters");

Review Comment:
   Let's create a String constant for `"valid length is 3-63 characters"`, using actual `OZONE_MIN_BUCKET_NAME_LENGTH` and `OZONE_MAX_BUCKET_NAME_LENGTH`, and reuse it for both checks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "octachoron (via GitHub)" <gi...@apache.org>.
octachoron commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2064308867


##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -68,10 +73,26 @@ private static void doNameChecks(String resName, String resType) {
       throw new IllegalArgumentException(resType + " name is null");
     }
 
-    if (resName.length() < OzoneConsts.OZONE_MIN_BUCKET_NAME_LENGTH ||
-        resName.length() > OzoneConsts.OZONE_MAX_BUCKET_NAME_LENGTH) {
+    if (resName.length() < OzoneConsts.OZONE_MIN_BUCKET_NAME_LENGTH) {
       throw new IllegalArgumentException(resType +
-          " length is illegal, " + "valid length is 3-63 characters");
+          " name '" + resName + "' is too short, " +
+          "valid length is 3-63 characters");

Review Comment:
   Good point, thank you! Added in the latest commit. That also made `doTestBadResourceNameReported()` specific to length-related assertions, so I adjusted its name accordingly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "octachoron (via GitHub)" <gi...@apache.org>.
octachoron commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2064167538


##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -48,6 +49,10 @@
 @InterfaceAudience.Public
 @InterfaceStability.Unstable
 public final class HddsClientUtils {
+  @VisibleForTesting
+  static final int MAX_BUCKET_NAME_LENGTH_IN_LOG =

Review Comment:
   Makes sense that way too (especially if we are going to keep using the package for utilities), no strong opinion either way.



##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -17,6 +17,7 @@
 
 package org.apache.hadoop.hdds.scm.client;
 
+import com.google.common.annotations.VisibleForTesting;

Review Comment:
   If we remove the annotation, then it does become unused indeed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-12649. Include name in length validation error [ozone]

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8322:
URL: https://github.com/apache/ozone/pull/8322#discussion_r2061189823


##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/client/HddsClientUtils.java:
##########
@@ -71,7 +71,7 @@ private static void doNameChecks(String resName, String resType) {
     if (resName.length() < OzoneConsts.OZONE_MIN_BUCKET_NAME_LENGTH ||
         resName.length() > OzoneConsts.OZONE_MAX_BUCKET_NAME_LENGTH) {
       throw new IllegalArgumentException(resType +
-          " length is illegal, " + "valid length is 3-63 characters");
+          " name '" + resName + "' is of illegal length, " + "valid length is 3-63 characters");
     }

Review Comment:
   - I think we could further improve it by checking and rejecting too short and too long values separately.  Then we can state "too short" and "too long" instead of "illegal length".
   - Should we cap too long invalid value in the message at some reasonable length?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org