You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "frankvicky (via GitHub)" <gi...@apache.org> on 2024/06/13 02:51:29 UTC

[PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

frankvicky opened a new pull request, #16319:
URL: https://github.com/apache/kafka/pull/16319

   In previous PR(#16048), I mistakenly excluded the underscore (_) from the set of valid characters for the protocol, resulting in the inability to correctly parse the connection string for SASL_PLAINTEXT. This bug fix addresses the issue and includes corresponding tests.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on PR #16319:
URL: https://github.com/apache/kafka/pull/16319#issuecomment-2165708156

   Hi @AndrewJSchofield 
   I have kept `0-9a-zA-Z\-%._` from the original regex and made changes based on it, PTAL 😄 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on PR #16319:
URL: https://github.com/apache/kafka/pull/16319#issuecomment-2164935759

   Hi @showuon, I have do a little change based on your comment, PTAL


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on PR #16319:
URL: https://github.com/apache/kafka/pull/16319#issuecomment-2165742067

   Thank you for your patience and review 😃 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on PR #16319:
URL: https://github.com/apache/kafka/pull/16319#issuecomment-2165495911

   Hi @AndrewJSchofield 
   I have rewrite the regex based on your feedback, PTAL 😄 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1638175128


##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -94,7 +94,7 @@ private Utils() {}
 
     // This matches URIs of formats: host:port and protocol://host:port
     // IPv6 is supported with [ip] pattern
-    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+-.]*://)?\\[?([0-9a-zA-Z\\-._%:]+)\\]?:([0-9]+)$");
+    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z0-9+_.%-]*://)?\\[?([0-9a-zA-Z\\-._%:]+)]?:([0-9]+)$");

Review Comment:
   I see, I think I miss understand before. 🤔 
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1637590625


##########
clients/src/test/java/org/apache/kafka/common/utils/UtilsTest.java:
##########
@@ -113,16 +113,26 @@ public void testGetHost() {
         assertEquals("mydomain.com", getHost("PLAINTEXT://mydomain.com:8080"));
         assertEquals("MyDomain.com", getHost("PLAINTEXT://MyDomain.com:8080"));
         assertEquals("My_Domain.com", getHost("PLAINTEXT://My_Domain.com:8080"));
+        assertEquals("mydomain.com", getHost("SASL_PLAINTEXT://mydomain.com:8080"));
+        assertEquals("MyDomain.com", getHost("SASL_PLAINTEXT://MyDomain.com:8080"));
+        assertEquals("My_Domain.com", getHost("SASL_PLAINTEXT://My_Domain.com:8080"));
         assertEquals("::1", getHost("[::1]:1234"));
         assertEquals("2001:db8:85a3:8d3:1319:8a2e:370:7348", getHost("PLAINTEXT://[2001:db8:85a3:8d3:1319:8a2e:370:7348]:5678"));
         assertEquals("2001:DB8:85A3:8D3:1319:8A2E:370:7348", getHost("PLAINTEXT://[2001:DB8:85A3:8D3:1319:8A2E:370:7348]:5678"));
         assertEquals("fe80::b1da:69ca:57f7:63d8%3", getHost("PLAINTEXT://[fe80::b1da:69ca:57f7:63d8%3]:5678"));
+        assertEquals("2001:db8:85a3:8d3:1319:8a2e:370:7348", getHost("SASL_PLAINTEXT://[2001:db8:85a3:8d3:1319:8a2e:370:7348]:5678"));

Review Comment:
   @frankvicky , we have 4 kinds of security protocol, let's test all of them to make sure we don't break anything. I think we can use `ParameterizedTest` here:
   
   ```
       @ParameterizedTest
       @CsvSource(value = {"PLANTEXT", "SASL_PLAINTEXT", ...)
   ```
   
   ref: https://kafka.apache.org/documentation/#brokerconfigs_security.inter.broker.protocol



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on PR #16319:
URL: https://github.com/apache/kafka/pull/16319#issuecomment-2164695371

   Hi @showuon, I have refactor the test case by utilizing `@CsvSource`, PTAL


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1637598927


##########
clients/src/test/java/org/apache/kafka/common/utils/UtilsTest.java:
##########
@@ -113,16 +113,26 @@ public void testGetHost() {
         assertEquals("mydomain.com", getHost("PLAINTEXT://mydomain.com:8080"));
         assertEquals("MyDomain.com", getHost("PLAINTEXT://MyDomain.com:8080"));
         assertEquals("My_Domain.com", getHost("PLAINTEXT://My_Domain.com:8080"));
+        assertEquals("mydomain.com", getHost("SASL_PLAINTEXT://mydomain.com:8080"));
+        assertEquals("MyDomain.com", getHost("SASL_PLAINTEXT://MyDomain.com:8080"));
+        assertEquals("My_Domain.com", getHost("SASL_PLAINTEXT://My_Domain.com:8080"));
         assertEquals("::1", getHost("[::1]:1234"));
         assertEquals("2001:db8:85a3:8d3:1319:8a2e:370:7348", getHost("PLAINTEXT://[2001:db8:85a3:8d3:1319:8a2e:370:7348]:5678"));
         assertEquals("2001:DB8:85A3:8D3:1319:8A2E:370:7348", getHost("PLAINTEXT://[2001:DB8:85A3:8D3:1319:8A2E:370:7348]:5678"));
         assertEquals("fe80::b1da:69ca:57f7:63d8%3", getHost("PLAINTEXT://[fe80::b1da:69ca:57f7:63d8%3]:5678"));
+        assertEquals("2001:db8:85a3:8d3:1319:8a2e:370:7348", getHost("SASL_PLAINTEXT://[2001:db8:85a3:8d3:1319:8a2e:370:7348]:5678"));

Review Comment:
   I see, I will refactor it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1637980599


##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -94,7 +94,7 @@ private Utils() {}
 
     // This matches URIs of formats: host:port and protocol://host:port
     // IPv6 is supported with [ip] pattern
-    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+-.]*://)?\\[?([0-9a-zA-Z\\-._%:]+)\\]?:([0-9]+)$");
+    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+_.-]*://)?\\[?([0-9a-zA-Z\\-._%:]+)]?:([0-9]+)$");

Review Comment:
   Reviewing the pattern compared with the original that the previous PR broke, there are some subtle differences. Because the patterns are so hard to read, I think it's good to keep the change as simple as possible.
   
   * Formerly, the majority of matching was done using `0-9a-zA-Z\-%._`. I think it would be a good idea to use this as a starting point for all of the matching, making minimal changes to this as required for the desired effect. This PR is needed because `_` was omitted, which is easy to miss.
   * `0-9` was used to match digits in all cases, except now there's a change to `\d`. Not sure why it was necessary to switch.
   * The initial character of the protocol name is now required to be alphabetic, where previously any of the permitted characters in the protocol could occupy any position.
   * The protocol name was previously able to include %. I'm sure it's unlikely this is a problem, but it's a change.



##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -94,7 +94,7 @@ private Utils() {}
 
     // This matches URIs of formats: host:port and protocol://host:port
     // IPv6 is supported with [ip] pattern
-    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+-.]*://)?\\[?([0-9a-zA-Z\\-._%:]+)\\]?:([0-9]+)$");
+    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+_.-]*://)?\\[?([0-9a-zA-Z\\-._%:]+)]?:([0-9]+)$");

Review Comment:
   Also, the removal of the final `\\` is surprising. Can you please confirm this was intentional?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "cmccabe (via GitHub)" <gi...@apache.org>.
cmccabe merged PR #16319:
URL: https://github.com/apache/kafka/pull/16319


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1637590625


##########
clients/src/test/java/org/apache/kafka/common/utils/UtilsTest.java:
##########
@@ -113,16 +113,26 @@ public void testGetHost() {
         assertEquals("mydomain.com", getHost("PLAINTEXT://mydomain.com:8080"));
         assertEquals("MyDomain.com", getHost("PLAINTEXT://MyDomain.com:8080"));
         assertEquals("My_Domain.com", getHost("PLAINTEXT://My_Domain.com:8080"));
+        assertEquals("mydomain.com", getHost("SASL_PLAINTEXT://mydomain.com:8080"));
+        assertEquals("MyDomain.com", getHost("SASL_PLAINTEXT://MyDomain.com:8080"));
+        assertEquals("My_Domain.com", getHost("SASL_PLAINTEXT://My_Domain.com:8080"));
         assertEquals("::1", getHost("[::1]:1234"));
         assertEquals("2001:db8:85a3:8d3:1319:8a2e:370:7348", getHost("PLAINTEXT://[2001:db8:85a3:8d3:1319:8a2e:370:7348]:5678"));
         assertEquals("2001:DB8:85A3:8D3:1319:8A2E:370:7348", getHost("PLAINTEXT://[2001:DB8:85A3:8D3:1319:8A2E:370:7348]:5678"));
         assertEquals("fe80::b1da:69ca:57f7:63d8%3", getHost("PLAINTEXT://[fe80::b1da:69ca:57f7:63d8%3]:5678"));
+        assertEquals("2001:db8:85a3:8d3:1319:8a2e:370:7348", getHost("SASL_PLAINTEXT://[2001:db8:85a3:8d3:1319:8a2e:370:7348]:5678"));

Review Comment:
   @frankvicky , we have 4 kinds of security protocol, let's test all of them. I think we can use `ParameterizedTest` here:
   
   ```
       @ParameterizedTest
       @CsvSource(value = {"PLANTEXT", "SASL_PLAINTEXT", ...)
   ```
   
   ref: https://kafka.apache.org/documentation/#brokerconfigs_security.inter.broker.protocol



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on PR #16319:
URL: https://github.com/apache/kafka/pull/16319#issuecomment-2164259901

   Hi @zhaochun-ma, @showuon 
   I have fix the bug that regex can't correctly parse that connection string starting with `SASL_PLAINTEXT`. PTAL 😄 
   Special thanks to @zhaochun-ma finding this issue. 🙏🏼 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1638040220


##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -94,7 +94,7 @@ private Utils() {}
 
     // This matches URIs of formats: host:port and protocol://host:port
     // IPv6 is supported with [ip] pattern
-    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+-.]*://)?\\[?([0-9a-zA-Z\\-._%:]+)\\]?:([0-9]+)$");
+    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+_.-]*://)?\\[?([0-9a-zA-Z\\-._%:]+)]?:([0-9]+)$");

Review Comment:
   Hi @AndrewJSchofield 
   Thanks for your your comment. 😄 
   I agree that we should keep regex easy to read instead of figuring out what each character mean. I will do a change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1638151790


##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -94,7 +94,7 @@ private Utils() {}
 
     // This matches URIs of formats: host:port and protocol://host:port
     // IPv6 is supported with [ip] pattern
-    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+-.]*://)?\\[?([0-9a-zA-Z\\-._%:]+)\\]?:([0-9]+)$");
+    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z0-9+_.%-]*://)?\\[?([0-9a-zA-Z\\-._%:]+)]?:([0-9]+)$");

Review Comment:
   I did mean `0-9a-zA-Z\\-%._` because that's what we had before. That's also what you have for the hostname part of the pattern, and I suggest the protocol part should use the same form and the same order of matching characters. Because it's not identical, people wanting to understand the change have a bit more work to do in order to check for equivalence.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1638054563


##########
clients/src/main/java/org/apache/kafka/common/utils/Utils.java:
##########
@@ -94,7 +94,7 @@ private Utils() {}
 
     // This matches URIs of formats: host:port and protocol://host:port
     // IPv6 is supported with [ip] pattern
-    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+-.]*://)?\\[?([0-9a-zA-Z\\-._%:]+)\\]?:([0-9]+)$");
+    private static final Pattern HOST_PORT_PATTERN = Pattern.compile("^(?:[a-zA-Z][a-zA-Z\\d+_.-]*://)?\\[?([0-9a-zA-Z\\-._%:]+)]?:([0-9]+)$");

Review Comment:
   Yes, I delete it because I have receive an alert from my idea, it said that this escape is redundant.
   I have test it in [regex101](https://regex101.com/), I think it work as expected.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #16319:
URL: https://github.com/apache/kafka/pull/16319#discussion_r1637736221


##########
clients/src/test/java/org/apache/kafka/common/utils/UtilsTest.java:
##########
@@ -106,31 +108,33 @@ public void testMurmur2() {
         }
     }
 
-    @Test
-    public void testGetHost() {
-        // valid
-        assertEquals("127.0.0.1", getHost("127.0.0.1:8000"));

Review Comment:
   Let's keep this test in `testGetHost`.



##########
clients/src/test/java/org/apache/kafka/common/utils/UtilsTest.java:
##########
@@ -106,31 +108,33 @@ public void testMurmur2() {
         }
     }
 
-    @Test
-    public void testGetHost() {
-        // valid
-        assertEquals("127.0.0.1", getHost("127.0.0.1:8000"));
-        assertEquals("mydomain.com", getHost("PLAINTEXT://mydomain.com:8080"));
-        assertEquals("MyDomain.com", getHost("PLAINTEXT://MyDomain.com:8080"));
-        assertEquals("My_Domain.com", getHost("PLAINTEXT://My_Domain.com:8080"));
-        assertEquals("::1", getHost("[::1]:1234"));

Review Comment:
   And this one.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16946: Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port [kafka]

Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on PR #16319:
URL: https://github.com/apache/kafka/pull/16319#issuecomment-2167331023

   Retriggering CI build: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-16319/6/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org