You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/04/27 21:22:38 UTC
[GitHub] [kafka] guozhangwang opened a new pull request #8565: KAFKA-9798: Send one record before starting the async producer
guozhangwang opened a new pull request #8565:
URL: https://github.com/apache/kafka/pull/8565
Comparing all other test cases, the `shouldAllowConcurrentAccesses` starts an async producer sending records throughout the test other than just synchronously sent and acked a few records before we start the streams application. Right after the streams app is started, we check that at least one record is sent to the output topic (i.e. completed processing). However since only this test starts the producer async and did not wait for it to complete, it is possible that the async producer gets too longer to produce some records and causing it to fail.
To follow what other tests did, I let this test to first send one round of records synchronously before starting the async producing.
Also encountered some new scala warnings that I fixed along with this PR.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on pull request #8565: KAFKA-9798: Send one record before starting the async producer
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on pull request #8565:
URL: https://github.com/apache/kafka/pull/8565#issuecomment-620242776
@mjsax @cadonna to take a look.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on pull request #8565: KAFKA-9798: Send one round synchronously before starting the async producer
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on pull request #8565:
URL: https://github.com/apache/kafka/pull/8565#issuecomment-622602829
@mjsax thanks for the review! Addressed your comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mjsax commented on a change in pull request #8565: KAFKA-9798: Send one round synchronously before starting the async producer
Posted by GitBox <gi...@apache.org>.
mjsax commented on a change in pull request #8565:
URL: https://github.com/apache/kafka/pull/8565#discussion_r418678556
##########
File path: streams/src/test/java/org/apache/kafka/streams/integration/QueryableStateIntegrationTest.java
##########
@@ -1262,6 +1266,7 @@ public void run() {
}
incrementIteration();
}
+ producer.flush();
Review comment:
The producer is create using try-with-resource clause thus, the producer should be flushed on the implicit `close()` anyway? Thus, I don't think explicit flushing would help?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on a change in pull request #8565: KAFKA-9798: Send one round synchronously before starting the async producer
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on a change in pull request #8565:
URL: https://github.com/apache/kafka/pull/8565#discussion_r418768692
##########
File path: streams/src/test/java/org/apache/kafka/streams/integration/QueryableStateIntegrationTest.java
##########
@@ -1262,6 +1266,7 @@ public void run() {
}
incrementIteration();
}
+ producer.flush();
Review comment:
Makes sense, will fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mjsax commented on a change in pull request #8565: KAFKA-9798: Send one round synchronously before starting the async producer
Posted by GitBox <gi...@apache.org>.
mjsax commented on a change in pull request #8565:
URL: https://github.com/apache/kafka/pull/8565#discussion_r418679194
##########
File path: streams/src/test/java/org/apache/kafka/streams/integration/QueryableStateIntegrationTest.java
##########
@@ -631,7 +631,11 @@ public void shouldAllowConcurrentAccesses() throws Exception {
final String storeName = "word-count-store";
final String windowStoreName = "windowed-word-count-store";
- final ProducerRunnable producerRunnable = new ProducerRunnable(streamConcurrent, inputValues, numIterations);
+ // send one round of records first to populate the stores
+ ProducerRunnable producerRunnable = new ProducerRunnable(streamThree, inputValues, 1);
+ producerRunnable.run();
+
+ producerRunnable = new ProducerRunnable(streamConcurrent, inputValues, numIterations);
Review comment:
Should this be `numIterations - 1`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org