You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/12/02 07:45:34 UTC
[GitHub] [kafka] showuon opened a new pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
showuon opened a new pull request #9674:
URL: https://github.com/apache/kafka/pull/9674
The flaky tests are because we forgot to close the kafkaStreams before purgeLocalStreamsState, so that sometimes there will be some tmp files be created during streams running/removing (ex: `checkpoint.tmp`), and caused the `DirectoryNotEmptyException` or `NoSuchFileException` be thrown.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bbejeck commented on pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
bbejeck commented on pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#issuecomment-738292594
Java 11 failed with
```
> Task :streams:streams-scala:integrationTest FAILED
05:26:47 org.gradle.internal.remote.internal.ConnectException: Could not connect to server [704081bf-4ce4-4a82-ae43-69c7e855ca11 port:38063, addresses:[/127.0.0.1]]. Tried addresses: [/127.0.0.1].
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a change in pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#discussion_r533956676
##########
File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamTableJoinTopologyOptimizationIntegrationTest.java
##########
@@ -149,9 +153,7 @@ public void shouldDoStreamTableJoinWithDifferentNumberOfPartitions() throws Exce
sendEvents(inputTopic, timestamp, expectedRecords);
sendEvents(outputTopic, timestamp, expectedRecords);
- startStreams(streamsBuilder);
Review comment:
I think we don't need to start 2 streams for this test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bbejeck commented on pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
bbejeck commented on pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#issuecomment-738799496
Failure on Java 11 unrelated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
showuon commented on pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#issuecomment-737056995
@bbejeck @vvcephei @lkokhreidze , please help review this PR. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bbejeck commented on pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
bbejeck commented on pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#issuecomment-738810006
cherry-picked to 2.7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bbejeck commented on pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
bbejeck commented on pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#issuecomment-738801930
Thanks for the fix @showuon!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a change in pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
showuon commented on a change in pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#discussion_r533956768
##########
File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamTableJoinTopologyOptimizationIntegrationTest.java
##########
@@ -221,7 +223,7 @@ private boolean topicExists(final String topic) {
);
}
- private <K, V> void validateReceivedMessages(final String topic,
+ private <K, V> void validateMessagesReceived(final String topic,
Review comment:
Rename the method name
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bbejeck commented on pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
bbejeck commented on pull request #9674:
URL: https://github.com/apache/kafka/pull/9674#issuecomment-738801798
merged #9674 into trunk
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] bbejeck merged pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState
Posted by GitBox <gi...@apache.org>.
bbejeck merged pull request #9674:
URL: https://github.com/apache/kafka/pull/9674
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org