You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/09 22:20:03 UTC
[GitHub] [beam] ihji opened a new pull request, #21782: Add withResources to Java External transform
ihji opened a new pull request, #21782:
URL: https://github.com/apache/beam/pull/21782
#21780
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Mention the appropriate issue in your description (for example: "addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment "fixes #<ISSUE NUMBER>" instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ihji commented on pull request #21782: Add withResources to Java External transform
Posted by GitBox <gi...@apache.org>.
ihji commented on PR #21782:
URL: https://github.com/apache/beam/pull/21782#issuecomment-1152083762
I'm not sure resource hint is the right tool for this. From what I've read from the proto definition, resource hint is advisory (nice to have) and can be ignored by runners. However, the resource here is considered as one of the inputs of the expanded transform (must have) and should not be ignored (otherwise the transform will certainly fail). The staged resources are more like runtime dependencies not just hints.
I agree that it would be great if we could look up staged resources by identifiers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ihji commented on pull request #21782: Add withResources to Java External transform
Posted by GitBox <gi...@apache.org>.
ihji commented on PR #21782:
URL: https://github.com/apache/beam/pull/21782#issuecomment-1151677587
R: @chamikaramj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on pull request #21782: Add withResources to Java External transform
Posted by GitBox <gi...@apache.org>.
robertwb commented on PR #21782:
URL: https://github.com/apache/beam/pull/21782#issuecomment-1151756221
I don't think this is the right direction either. If an external transform needs a certain set of files, that transform should be responsible for making the declaration, not the callee (and especially, it seems, the callee from another language).
I think the correct solution is to us resource hints (e.g. https://beam.apache.org/documentation/runtime/resource-hints/). There's also the issue of finding the resources at their new locations (on the worker). Presumably one would ask for a resource (e.g. file) to be staged against some kind of identifier, and on the worker one could use this identifier to look up the copied resource in its final destination.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ihji closed pull request #21782: Add withResources to Java External transform
Posted by GitBox <gi...@apache.org>.
ihji closed pull request #21782: Add withResources to Java External transform
URL: https://github.com/apache/beam/pull/21782
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org