You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by rangadi <gi...@git.apache.org> on 2017/07/04 07:25:16 UTC
[GitHub] beam pull request #3492: [BEAM-2551] KafkaIO reader blocks indefinitely in c...
GitHub user rangadi opened a pull request:
https://github.com/apache/beam/pull/3492
[BEAM-2551] KafkaIO reader blocks indefinitely in case of network issues
If the KafaIO source reader on the worker can't reach the server, Kafka consumer blocks forever inside UnboundedReader.start(). Users have no indication of the error. It is better if start() fails with an error.
It is easy to reproduce with Kafka command line. I reported it on Kafka users list here : https://lists.apache.org/thread.html/98cebefacbd65b0d6c6817fe0b5197e26bc60252e72d05fced91e628@%3Cusers.kafka.apache.org%3E
It blocks inside Kafka client. Fortunately it can be unblocked with KafkaConsumer.wakeup(). This PR initializes partition on a separate thread and cancels it if it does not complete in one minute. Added a unit test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rangadi/beam timeout
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3492.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3492
----
commit 9238c36263b4c885f53b43dc26ff5c5f820ed908
Author: Raghu Angadi <ra...@google.com>
Date: 2017-07-04T06:54:10Z
intialize partition on a separate thread.
commit 65a4bc15d48d25f2729bb6a6b8ac42bca5ea9999
Author: Raghu Angadi <ra...@google.com>
Date: 2017-07-04T06:00:12Z
Add a unit test
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] beam pull request #3492: [BEAM-2551] KafkaIO reader blocks indefinitely in c...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/3492
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---