You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "TaiJuWu (via GitHub)" <gi...@apache.org> on 2024/07/18 16:01:35 UTC
[PR] MINOR: Verify topic deletion for new test infra [kafka]
TaiJuWu opened a new pull request, #16627:
URL: https://github.com/apache/kafka/pull/16627
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1689010781
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica manager's should have deleted all of this topic's partitions"
+ );
+
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
Review Comment:
It already check [here](https://github.com/apache/kafka/blob/63fccc4f4cd37a6d205630b3b0b31ff70cf73317/core/src/test/java/kafka/test/ClusterInstance.java#L214-L220)
Or I miss something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1690631302
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,53 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
Review Comment:
> Maybe we don't need this so complicated method. It can be implemented by `waitForTopic("xxx", 0)` WDYT?
This is a good suggestion.
I have same idea before, but this change make `testTopicWithCollidingCharDeletionAndCreateAgain` on `ZK mode` failed at that moment.
But this method can work at this moment, I still not sure the reason.
Maybe I have these checking ?
```
org.apache.kafka.test.TestUtils.waitForCondition(
() -> !clusterReference.get().zkClient().isTopicMarkedForDeletion(topic),
String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
);
org.apache.kafka.test.TestUtils.waitForCondition(
() -> !clusterReference.get().zkClient().topicExists(topic),
String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1690631302
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,53 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
Review Comment:
> Maybe we don't need this so complicated method. It can be implemented by `waitForTopic("xxx", 0)` WDYT?
This is a good suggestion.
I have same idea before, but this change make `testTopicWithCollidingCharDeletionAndCreateAgain` on `ZK mode` failed at that moment.
And this method can work at this moment, I still not sure the reason.
Maybe I have these checking ?
```
org.apache.kafka.test.TestUtils.waitForCondition(
() -> !clusterReference.get().zkClient().isTopicMarkedForDeletion(topic),
String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
);
org.apache.kafka.test.TestUtils.waitForCondition(
() -> !clusterReference.get().zkClient().topicExists(topic),
String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1701776780
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +183,10 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ default void verifyTopicDeletion(String topic) throws InterruptedException {
Review Comment:
Updated. Thanks for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1697416103
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +183,10 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ default void verifyTopicDeletion(String topic) throws InterruptedException {
Review Comment:
It is used to wait for deletion, so maybe it can be renamed to `waitTopicDeletion`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #16627:
URL: https://github.com/apache/kafka/pull/16627
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1700043528
##########
core/src/test/java/kafka/test/junit/ZkClusterInvocationContext.java:
##########
@@ -206,6 +206,22 @@ public void startBroker(int brokerId) {
findBrokerOrThrow(brokerId).startup();
}
+ @Override
Review Comment:
> I guess that we don't need those extra waiting. Or could you please loop the tests on your local to check the race condition?
Hi @chia7712 ,
As discussion offline, this is normal flow to deal `ZK` mode so we have to keep it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1689010781
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica manager's should have deleted all of this topic's partitions"
+ );
+
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
Review Comment:
Update to the newest commit, hope my understanding is correct :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1695285793
##########
core/src/test/java/kafka/test/junit/ZkClusterInvocationContext.java:
##########
@@ -206,6 +206,22 @@ public void startBroker(int brokerId) {
findBrokerOrThrow(brokerId).startup();
}
+ @Override
Review Comment:
I guess that we don't need those extra waiting. Or could you please loop the tests on your local to check the race condition?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1685705213
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
Review Comment:
This should be moved into zk instance, since it is not a part of default implementation. FOr example:
```java
@Override
public void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
org.apache.kafka.test.TestUtils.waitForCondition(
() -> !getUnderlying().zkClient().isTopicMarkedForDeletion(topic),
String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
);
org.apache.kafka.test.TestUtils.waitForCondition(
() -> !getUnderlying().zkClient().topicExists(topic),
String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
);
ClusterInstance.super.verifyTopicDeletion(topic, partitions);
}
```
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica manager's should have deleted all of this topic's partitions"
+ );
+
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica logs not deleted after delete topic is complete"
+ );
+
+
+ TestUtils.waitForCondition(() -> brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ JavaConverters.seqAsJavaList(broker.logManager().liveLogDirs()).stream()
+ .map(logDir -> JavaConverters.mapAsJavaMap(new OffsetCheckpointFile(new File(logDir, "cleaner-offset-checkpoint"), null).read()))
+ .collect(Collectors.toList())
Review Comment:
Do we need to re-collect it?
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
Review Comment:
please fix the indent.
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica manager's should have deleted all of this topic's partitions"
+ );
+
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica logs not deleted after delete topic is complete"
+ );
+
+
+ TestUtils.waitForCondition(() -> brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ JavaConverters.seqAsJavaList(broker.logManager().liveLogDirs()).stream()
+ .map(logDir -> JavaConverters.mapAsJavaMap(new OffsetCheckpointFile(new File(logDir, "cleaner-offset-checkpoint"), null).read()))
+ .collect(Collectors.toList())
+ .stream()
+ .noneMatch(checkpointsPerLogDir -> checkpointsPerLogDir.containsKey(tp))
+ )
+ ), "Cleaner offset for deleted partition should have been removed");
+
+ TestUtils.waitForCondition(() -> brokers().values().stream().allMatch(
+ broker -> broker.config().logDirs().forall(
+ logDir -> topicPartitions.stream().noneMatch(
+ tp -> new File(logDir, tp.topic() + "-" + tp.partition()).exists()))),
+ "Failed to soft-delete the data to a delete directory"
+ );
+
+ TestUtils.waitForCondition(() -> brokers().values().stream().allMatch(
+ broker -> broker.config().logDirs().forall(
+ logDir -> topicPartitions.stream().noneMatch(
+ tp -> Arrays.asList(new File(logDir).list()).stream().allMatch(
Review Comment:
please add null check
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica manager's should have deleted all of this topic's partitions"
+ );
+
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
Review Comment:
We should check `logManager`, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1689010781
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica manager's should have deleted all of this topic's partitions"
+ );
+
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
Review Comment:
Update to the newest commit, hope my understanding is correct :)
Update:
Oh, I find the fool mistake....
Sorry for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1690011240
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,53 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
Review Comment:
Maybe we don't need this so complicated method. It can be implemented by `waitForTopic("xxx", 0)` WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Verify topic deletion for new test infra [kafka]
Posted by "TaiJuWu (via GitHub)" <gi...@apache.org>.
TaiJuWu commented on code in PR #16627:
URL: https://github.com/apache/kafka/pull/16627#discussion_r1689009256
##########
core/src/test/java/kafka/test/ClusterInstance.java:
##########
@@ -183,6 +193,67 @@ default Set<GroupProtocol> supportedGroupProtocols() {
//---------------------------[wait]---------------------------//
+ @SuppressWarnings("deprecation")
+ default void verifyTopicDeletion(String topic, int partitions) throws InterruptedException {
+ if (!isKRaftTest()) {
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().isTopicMarkedForDeletion(topic),
+ String.format("Admin path /admin/delete_topics/%s path not deleted even after a replica is restarted", topic)
+ );
+
+ TestUtils.waitForCondition(
+ () -> !((IntegrationTestHarness) getUnderlying()).zkClient().topicExists(topic),
+ String.format("Topic path /brokers/topics/%s not deleted after /admin/delete_topics/%s path is deleted", topic, topic)
+ );
+ }
+
+ List<TopicPartition> topicPartitions = IntStream.range(0, partitions).mapToObj(i -> new TopicPartition(topic, i)).collect(Collectors.toList());
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica manager's should have deleted all of this topic's partitions"
+ );
+
+ TestUtils.waitForCondition(
+ () ->
+ brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ broker.replicaManager().onlinePartition(tp).isEmpty())
+ ), "Replica logs not deleted after delete topic is complete"
+ );
+
+
+ TestUtils.waitForCondition(() -> brokers().values().stream().allMatch(broker ->
+ topicPartitions.stream().allMatch(tp ->
+ JavaConverters.seqAsJavaList(broker.logManager().liveLogDirs()).stream()
+ .map(logDir -> JavaConverters.mapAsJavaMap(new OffsetCheckpointFile(new File(logDir, "cleaner-offset-checkpoint"), null).read()))
+ .collect(Collectors.toList())
Review Comment:
Remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org