You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "becketqin (via GitHub)" <gi...@apache.org> on 2023/03/07 02:23:30 UTC
[GitHub] [beam] becketqin opened a new pull request, #25742: [Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API.
becketqin opened a new pull request, #25742:
URL: https://github.com/apache/beam/pull/25742
This is the second patch of migrating Flink runner to execute batch jobs with DataStream API.
This patch does the following:
1. Add a new option of `UseDataStreamForBatch`. When it is set to true, the batch jobs will be executed with DataStream API. Otherwise, the jobs are executed with DataSet API.
2. Modify the pipeline translation path to honor the configuration.
3. Modify the tests to cover the DataStream execution path.
This patch should be merged after #25741 is resolved, so the streaming jobs metrics are still backwards compatible.
fix #25740
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [x] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25742: [Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API.
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25742:
URL: https://github.com/apache/beam/pull/25742#issuecomment-1591118143
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25742: [Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API.
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25742:
URL: https://github.com/apache/beam/pull/25742#issuecomment-1578697808
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25742: [Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API.
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25742:
URL: https://github.com/apache/beam/pull/25742#issuecomment-1457448089
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] closed pull request #25742: [Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API.
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #25742: [Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API.
URL: https://github.com/apache/beam/pull/25742
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] becketqin commented on pull request #25742: [Flink Runner] Add UseDataStreamForBatch option to Flink runner to enable batch execution on DataStream API.
Posted by "becketqin (via GitHub)" <gi...@apache.org>.
becketqin commented on PR #25742:
URL: https://github.com/apache/beam/pull/25742#issuecomment-1459141255
@xinyuiscool Would you help take a look? Thanks!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org