You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "FrankChen021 (via GitHub)" <gi...@apache.org> on 2026/08/10 03:50:02 UTC
[PR] fix(test): retry Kafka partition readiness (druid)
FrankChen021 opened a new pull request, #19948:
URL: https://github.com/apache/druid/pull/19948
### Description
This is a follow-up to [#19922](https://github.com/apache/druid/pull/19922) that hardens the embedded Kafka test resource against the topic metadata propagation race observed during that PR's CI run.
`KafkaResource.createTopicWithPartitions()` waits for the topic-creation future, but Kafka can acknowledge creation before the topic metadata and partition leaders are ready for a subsequent `listOffsets` request. The existing `waitForPartitionsToBeReady()` performed that request only once, so the test setup could fail with `UnknownTopicOrPartitionException` during the propagation window.
This PR:
- retries the partition latest-offset readiness probe up to five times with Druid's retry backoff;
- retries only Kafka `RetriableException` instances, including exceptions nested in `ExecutionException`;
- keeps the readiness check shared by both topic creation and partition expansion.
There is no end-user behavior change; this only stabilizes embedded test setup.
### Evidence
- PR #19922 run [31319365598](https://github.com/apache/druid/actions/runs/31319365598), failed job [93259746963](https://github.com/apache/druid/actions/runs/31319365598/job/93259746963), reported one error in `KafkaIndexFaultToleranceTest`.
- The stack trace was `KafkaFutureImpl.get` -> `KafkaResource.waitForPartitionsToBeReady` -> `KafkaResource.createTopicWithPartitions`, with `UnknownTopicOrPartitionException` as the cause.
- The same topic-readiness error occurred in an unrelated historical job (job `93262496826`), while nearby runs passed, confirming a timing-dependent failure.
- Before this change, the focused fault-tolerance test passed locally twice but did not eliminate the one-shot readiness race. After this change, `KafkaResourceTest` passed (1 test), and the focused fault-tolerance test passed twice, including both parameterizations: 9 tests, 0 failures, 0 errors on each run.
### Validation
```text
mvn -B -ntp -Pskip-static-checks -pl extensions-core/kafka-indexing-service -am test \
-Dtest=org.apache.druid.indexing.kafka.simulate.KafkaResourceTest \
-Dsurefire.failIfNoSpecifiedTests=false -Dweb.console.skip=true -T1C
mvn -B -ntp -Pskip-static-checks -pl embedded-tests -am test \
-Dtest=org.apache.druid.testing.embedded.indexing.KafkaIndexFaultToleranceTest \
-Dsurefire.failIfNoSpecifiedTests=false -Dweb.console.skip=true -T1C
mvn -B -ntp -pl extensions-core/kafka-indexing-service -am -DskipTests \
-Dweb.console.skip=true checkstyle:check pmd:check forbiddenapis:check forbiddenapis:testCheck
```
All commands passed locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] fix(test): retry Kafka partition readiness (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 merged PR #19948:
URL: https://github.com/apache/druid/pull/19948
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org