You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "chickenchickenlove (via GitHub)" <gi...@apache.org> on 2026/01/10 13:11:37 UTC
[PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
chickenchickenlove opened a new pull request, #21280:
URL: https://github.com/apache/kafka/pull/21280
This PR fixes a flaky failure in `ConfigCommandIntegrationTest.testUpdateInvalidBrokerConfigs`.
The test updates an unknown broker dynamic config `(invalid=2)` and immediately runs `kafka-configs --describe` to assert that the unknown config is treated as sensitive `(sensitive=true)` and returned as `invalid=null`.
In KRaft, the alter request can return before the change is fully visible to `--describe` because `dynamic config` updates are applied asynchronously and published via `DynamicConfigPublisher` as metadata updates are processed.
As a result, the first describe may read stale state and miss the expected fields, causing intermittent failures.
To make the test robust, we now poll `--describe` for a short period and proceed once the expected output becomes visible (or fail with the last observed output on timeout).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2714139432
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,37 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ boolean hasInvalidNull = describeResult.contains("invalid=null");
+ boolean hasSensitiveTrue = describeResult.contains("sensitive=true");
+
+ if (hasInvalidNull && !hasSensitiveTrue) {
+ throw new AssertionError("Expected sensitive=true when invalid=null is visible, but it was not present.\n" +
Review Comment:
will this truly make it interrupt the `waitForCondition`? or is it maybe going to continue retrying? (from a quick look at the code path seems it ends up using `retryOnExceptionWithTimeout`?)
Maybe we can just `waitForCondition` checking `describeResult.contains("invalid=null") `, and then, outside of the wait, assert that `describeResult.contains("sensitive=true");` wdyt?
nit: the message seems a bit confusing to me, maybe something along the lines of "Description of the invalid config did not include sensive=true as expected"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "chickenchickenlove (via GitHub)" <gi...@apache.org>.
chickenchickenlove commented on PR #21280:
URL: https://github.com/apache/kafka/pull/21280#issuecomment-3778479924
@lianetm
Thanks for taking time to review this PR.
I made new commits baseㅇ on your comment!
When you get a chance, Please take another look. 🙇♂️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "chickenchickenlove (via GitHub)" <gi...@apache.org>.
chickenchickenlove commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2716969375
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,32 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ return describeResult.contains("invalid=null");
+ }, 5000, () -> "Dynamic broker config was not visible within 5s (missing 'invalid=null').\n" +
Review Comment:
@lianetm , Hi!
Thanks for taking time to review this.
IMHO, the current message is actually correct in this context.
Inside `waitForCondition`, we are specifically waiting for `invalid=null` to appear.
If this times out, it means `invalid=null` was indeed missing from the output.
The check for `sensitive=true` happens after this block, so it doesn't affect this timeout message.
Please let me know if I misunderstood anything. I want to make sure we're on the same page. 🙇♂️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "chickenchickenlove (via GitHub)" <gi...@apache.org>.
chickenchickenlove commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2714851541
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,37 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ boolean hasInvalidNull = describeResult.contains("invalid=null");
+ boolean hasSensitiveTrue = describeResult.contains("sensitive=true");
+
+ if (hasInvalidNull && !hasSensitiveTrue) {
+ throw new AssertionError("Expected sensitive=true when invalid=null is visible, but it was not present.\n" +
Review Comment:
@lianetm
Thanks for the detailed review!!! 🙇♂️ — good point about waitForCondition catching exceptions (via `retryOnExceptionWithTimeout`), which could lead to unnecessary retries.
I refactored the test to wait for `invalid=null` first, and then assert `sensitive=true` outside the wait loop.
I also removed the custom error message since the assertion logic is now explicit and self-explanatory.
When you have bandwidth, please take another look. 🙇♂️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm merged PR #21280:
URL: https://github.com/apache/kafka/pull/21280
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2716530410
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,32 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ return describeResult.contains("invalid=null");
+ }, 5000, () -> "Dynamic broker config was not visible within 5s (missing 'invalid=null').\n" +
Review Comment:
we need to update this msg after the latest changes right? (remove the "missing 'invalid=null'"?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2717265693
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,32 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ return describeResult.contains("invalid=null");
+ }, 5000, () -> "Dynamic broker config was not visible within 5s (missing 'invalid=null').\n" +
Review Comment:
you're right! I probably read too fast and though it referred to the property we now assert outside, my bad, all good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "chickenchickenlove (via GitHub)" <gi...@apache.org>.
chickenchickenlove commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2714851541
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,37 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ boolean hasInvalidNull = describeResult.contains("invalid=null");
+ boolean hasSensitiveTrue = describeResult.contains("sensitive=true");
+
+ if (hasInvalidNull && !hasSensitiveTrue) {
+ throw new AssertionError("Expected sensitive=true when invalid=null is visible, but it was not present.\n" +
Review Comment:
Thanks for the detailed review!!! 🙇♂️ — good point about waitForCondition catching exceptions (via `retryOnExceptionWithTimeout`), which could lead to unnecessary retries.
I refactored the test to wait for `invalid=null` first, and then assert `sensitive=true` outside the wait loop.
I also removed the custom error message since the assertion logic is now explicit and self-explanatory.
When you have bandwidth, please take another look. 🙇♂️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21280:
URL: https://github.com/apache/kafka/pull/21280#issuecomment-3764864907
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "chickenchickenlove (via GitHub)" <gi...@apache.org>.
chickenchickenlove commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2714851541
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,37 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ boolean hasInvalidNull = describeResult.contains("invalid=null");
+ boolean hasSensitiveTrue = describeResult.contains("sensitive=true");
+
+ if (hasInvalidNull && !hasSensitiveTrue) {
+ throw new AssertionError("Expected sensitive=true when invalid=null is visible, but it was not present.\n" +
Review Comment:
Thanks for the detailed review!!! 🙇♂️ — good point about waitForCondition catching exceptions (via `retryOnExceptionWithTimeout`), which could lead to unnecessary retries.
I refactored the test to wait for `invalid=null` first, and then assert `sensitive=true` outside the wait loop.
I also removed the custom error message since the assertion logic is now explicit and self-explanatory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "chickenchickenlove (via GitHub)" <gi...@apache.org>.
chickenchickenlove commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2712765990
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,34 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ // We will treat unknown config as sensitive
+ // For 'describeResult.contains("sensitive=true")'
+
+ // Sensitive config will not return
+ // For 'describeResult.contains("invalid=null")'
Review Comment:
Agree!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "lianetm (via GitHub)" <gi...@apache.org>.
lianetm commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2709022323
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,34 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ // We will treat unknown config as sensitive
+ // For 'describeResult.contains("sensitive=true")'
+
+ // Sensitive config will not return
+ // For 'describeResult.contains("invalid=null")'
Review Comment:
also not clear on the value of these, the return line seems pretty explicit to me, wdyt?
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,34 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ // We will treat unknown config as sensitive
+ // For 'describeResult.contains("sensitive=true")'
+
+ // Sensitive config will not return
+ // For 'describeResult.contains("invalid=null")'
+ return describeResult.contains("sensitive=true") && describeResult.contains("invalid=null");
Review Comment:
I guess that it makes sense to wait until we get `invalid=null` in the output, but once that happens we do expect to have `sensitive=true` right away, right?
If my expectation is right, then we should fix this to ensure we fail fast if sensitive is false for our "invalid" config (without waiting for sensitive to become true)
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,34 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ // We will treat unknown config as sensitive
+ // For 'describeResult.contains("sensitive=true")'
Review Comment:
don't quite get this comment. Needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20059: Fix flaky ConfigCommandIntegrationTest testUpdateInvalidBrokerConfigs [kafka]
Posted by "chickenchickenlove (via GitHub)" <gi...@apache.org>.
chickenchickenlove commented on code in PR #21280:
URL: https://github.com/apache/kafka/pull/21280#discussion_r2712849441
##########
tools/src/test/java/org/apache/kafka/tools/ConfigCommandIntegrationTest.java:
##########
@@ -461,27 +461,34 @@ public void testUpdatePerBrokerConfigInKRaftThenShouldFail() {
}
@ClusterTest
- public void testUpdateInvalidBrokerConfigs() {
+ public void testUpdateInvalidBrokerConfigs() throws InterruptedException {
updateAndCheckInvalidBrokerConfig(Optional.empty());
updateAndCheckInvalidBrokerConfig(Optional.of(String.valueOf((cluster.brokers().entrySet().iterator().next().getKey()))));
}
- private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) {
+ private void updateAndCheckInvalidBrokerConfig(Optional<String> brokerIdOrDefault) throws InterruptedException {
List<String> alterOpts = generateDefaultAlterOpts(cluster.bootstrapServers());
try (Admin client = cluster.admin()) {
alterConfigWithAdmin(client, brokerIdOrDefault, Map.of("invalid", "2"), alterOpts);
+ AtomicReference<String> last = new AtomicReference<>("");
- Stream<String> describeCommand = Stream.concat(
- Stream.concat(
- Stream.of("--bootstrap-server", cluster.bootstrapServers()),
- Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
- Stream.of("--entity-type", "brokers", "--describe"));
- String describeResult = captureStandardOut(run(describeCommand));
-
- // We will treat unknown config as sensitive
- assertTrue(describeResult.contains("sensitive=true"), describeResult);
- // Sensitive config will not return
- assertTrue(describeResult.contains("invalid=null"), describeResult);
+ TestUtils.waitForCondition(() -> {
+ Stream<String> describeCommand = Stream.concat(
+ Stream.concat(
+ Stream.of("--bootstrap-server", cluster.bootstrapServers()),
+ Stream.of(entityOp(brokerIdOrDefault).toArray(new String[0]))),
+ Stream.of("--entity-type", "brokers", "--describe")
+ );
+ String describeResult = captureStandardOut(run(describeCommand));
+ last.set(describeResult);
+
+ // We will treat unknown config as sensitive
+ // For 'describeResult.contains("sensitive=true")'
+
+ // Sensitive config will not return
+ // For 'describeResult.contains("invalid=null")'
+ return describeResult.contains("sensitive=true") && describeResult.contains("invalid=null");
Review Comment:
Ah, I see. I totally agree with you.
Addressed it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org