You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by pnowojski <gi...@git.apache.org> on 2017/09/29 13:38:06 UTC
[GitHub] flink pull request #4751: [FLINK-7739][kafka-tests] Throttle down data produ...
GitHub user pnowojski opened a pull request:
https://github.com/apache/flink/pull/4751
[FLINK-7739][kafka-tests] Throttle down data producing thread
Minor tests improvement in tests to avoid busy loop
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pnowojski/flink custom-partitioning
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4751.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4751
----
commit 77cefb87214a4b7462c5b65fafe86352fe3a7047
Author: Piotr Nowojski <pi...@gmail.com>
Date: 2017-09-28T14:59:09Z
[FLINK-7739][kafka-tests] Throttle down data producing thread
----
---
[GitHub] flink issue #4751: [FLINK-7739][kafka-tests] Throttle down data producing th...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/4751
I assume that the `Thread.sleep(1)` should not do much harm if the Kafka Producer already blocks due to buffering and I/O. If it should not block, then the `Thread.sleep(1)` will give other threads the chance to execute when having only scarce resources (e.g. on Travis).
I agree with @pnowojski that this is not a stress test and we have seen in the past that tests became unstable due to resource scarcity. Therefore I would like to merge this PR if Travis gives green light and @StephanEwen does not object.
---
[GitHub] flink issue #4751: [FLINK-7739][kafka-tests] Throttle down data producing th...
Posted by pnowojski <gi...@git.apache.org>.
Github user pnowojski commented on the issue:
https://github.com/apache/flink/pull/4751
But this is not supposed to be a stress test, so why make it more computation demanding/longer to execute?
---
[GitHub] flink pull request #4751: [FLINK-7739][kafka-tests] Throttle down data produ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4751
---
[GitHub] flink issue #4751: [FLINK-7739][kafka-tests] Throttle down data producing th...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/4751
@pnowojski I am wondering whether this is really necessary. This is not quite a busy loop after all, because the `collect()` call emits to the Kafka Producer and goes through buffering and I/O.
Unless this is really necessary for stability, I would prefer to not do such throttling changes. The system should be stable even with threads being aggressively intensive.
---