You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/03/15 21:14:41 UTC
[GitHub] [beam] dpcollins-google opened a new pull request #17095: [BEAM-11934] Remove DataflowRunner override for runner-determined sharding
dpcollins-google opened a new pull request #17095:
URL: https://github.com/apache/beam/pull/17095
Since https://github.com/apache/beam/pull/14450, this override artificially constrains dataflow's ability to use autosharding.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit merged pull request #17095: [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner users to opt-in to dynamic sharding logic
Posted by GitBox <gi...@apache.org>.
TheNeuralBit merged pull request #17095:
URL: https://github.com/apache/beam/pull/17095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] scwhittle commented on pull request #17095: [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner users to opt-in to dynamic sharding logic
Posted by GitBox <gi...@apache.org>.
scwhittle commented on pull request #17095:
URL: https://github.com/apache/beam/pull/17095#issuecomment-1070974494
LGTM, no concerns with update since this keeps existing behavior without an experiment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit merged pull request #17095: [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner users to opt-in to dynamic sharding logic
Posted by GitBox <gi...@apache.org>.
TheNeuralBit merged pull request #17095:
URL: https://github.com/apache/beam/pull/17095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] dpcollins-google commented on pull request #17095: [BEAM-11934] Remove DataflowRunner override for runner-determined sharding
Posted by GitBox <gi...@apache.org>.
dpcollins-google commented on pull request #17095:
URL: https://github.com/apache/beam/pull/17095#issuecomment-1068479872
R: @scwhittle
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] scwhittle commented on pull request #17095: [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner users to opt-in to dynamic sharding logic
Posted by GitBox <gi...@apache.org>.
scwhittle commented on pull request #17095:
URL: https://github.com/apache/beam/pull/17095#issuecomment-1070974494
LGTM, no concerns with update since this keeps existing behavior without an experiment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] dpcollins-google commented on pull request #17095: [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner users to opt-in to dynamic sharding logic
Posted by GitBox <gi...@apache.org>.
dpcollins-google commented on pull request #17095:
URL: https://github.com/apache/beam/pull/17095#issuecomment-1070972948
Run Java PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] scwhittle commented on pull request #17095: [BEAM-11934] Remove DataflowRunner override for runner-determined sharding
Posted by GitBox <gi...@apache.org>.
scwhittle commented on pull request #17095:
URL: https://github.com/apache/beam/pull/17095#issuecomment-1068856790
There may be a potential concern for pipeline update compatiblity here. Perhaps this should just be changed so that if the user explicitly requests the new autosharding it occurs. (Or this could be left as is and portable job submission addresses it by not using this path.)
@reuvenlax for thoughts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] dpcollins-google commented on pull request #17095: [BEAM-11934] Add enable_file_dynamic_sharding to allow DataflowRunner users to opt-in to dynamic sharding logic
Posted by GitBox <gi...@apache.org>.
dpcollins-google commented on pull request #17095:
URL: https://github.com/apache/beam/pull/17095#issuecomment-1070972948
Run Java PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org