You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by aaltay <gi...@git.apache.org> on 2017/01/10 21:49:37 UTC
[GitHub] beam pull request #1762: Implement wait_until_finish method for existing run...
GitHub user aaltay opened a pull request:
https://github.com/apache/beam/pull/1762
Implement wait_until_finish method for existing runners.
Implement wait_until_finish method for existing runners.
Also defines the not implemented cancel() method and updates existing
usages to use wait_until_finish() instead of blocking runners.
Main changes are in the runners/ folder
runner.py - has the APIs
dataflow_runner.py, direct_runner.py modified to implement the API (moving the existing blocking code around.)
Rest of the changes are mechanical to mainly convert
p.run() to p.run().wait_until_finish() in tests and examples. Changed tests because they run validation after the run and need to block until completion. We may revert the changes in examples. I converted the because in the instructions we directed users to blocking runners before and this change keeps the behavior same.
I have started a local post commit run (not completed yet) and it was successful with the first few tests so far and the changes are same for all tests.
Remaining work after this PR:
- Removing BlockingDataflowRunner. It is not used after this change with the SDK code/examples/tests.
- Support for duration argument in wait_until_finish is missing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aaltay/incubator-beam expand
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/1762.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1762
----
commit c6c78a3fe15910acf2d1cc0bf244af06794dea1f
Author: Ahmet Altay <al...@google.com>
Date: 2017-01-10T02:23:20Z
Implement wait_until_finish method for existing runners.
Also defines the not implemented cancel() method and updates existing
usages to use wait_until_finish() instead of blocking runners.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] beam pull request #1762: [BEAM-759] Implement wait_until_finish method for e...
Posted by aaltay <gi...@git.apache.org>.
GitHub user aaltay reopened a pull request:
https://github.com/apache/beam/pull/1762
[BEAM-759] Implement wait_until_finish method for existing runners.
Implement wait_until_finish method for existing runners.
Also defines the not implemented cancel() method and updates existing
usages to use wait_until_finish() instead of blocking runners.
Main changes are in the runners/ folder
runner.py - has the APIs
dataflow_runner.py, direct_runner.py modified to implement the API (moving the existing blocking code around.)
Rest of the changes are mechanical to mainly convert
p.run() to p.run().wait_until_finish() in tests and examples. Changed tests because they run validation after the run and need to block until completion. We may revert the changes in examples. I converted the because in the instructions we directed users to blocking runners before and this change keeps the behavior same.
I have started a local post commit run (not completed yet) and it was successful with the first few tests so far and the changes are same for all tests.
Remaining work after this PR:
- Removing BlockingDataflowRunner. It is not used after this change with the SDK code/examples/tests.
- Support for duration argument in wait_until_finish is missing.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aaltay/incubator-beam expand
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/1762.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1762
----
commit c6c78a3fe15910acf2d1cc0bf244af06794dea1f
Author: Ahmet Altay <al...@google.com>
Date: 2017-01-10T02:23:20Z
Implement wait_until_finish method for existing runners.
Also defines the not implemented cancel() method and updates existing
usages to use wait_until_finish() instead of blocking runners.
commit a27773088d1ecad371725fdd5d27c5a1dd58a5cb
Author: Ahmet Altay <al...@google.com>
Date: 2017-01-10T22:15:36Z
Fix wrong use of wait_until_complete in examples.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] beam pull request #1762: [BEAM-759] Implement wait_until_finish method for e...
Posted by aaltay <gi...@git.apache.org>.
Github user aaltay closed the pull request at:
https://github.com/apache/beam/pull/1762
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] beam pull request #1762: [BEAM-759] Implement wait_until_finish method for e...
Posted by aaltay <gi...@git.apache.org>.
Github user aaltay closed the pull request at:
https://github.com/apache/beam/pull/1762
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---