You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/10 00:17:49 UTC
[GitHub] [beam] ananvay opened a new pull request #11370: Update
environments.py to add a method to specify container image
ananvay opened a new pull request #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ananvay commented on issue #11370: Update environments.py to
add a method to specify container image
Posted by GitBox <gi...@apache.org>.
ananvay commented on issue #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370#issuecomment-611814439
R @robertwb
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] pabloem commented on issue #11370: Update environments.py to
add a method to specify container image
Posted by GitBox <gi...@apache.org>.
pabloem commented on issue #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370#issuecomment-612177881
I figured i'd cherry pick too. https://github.com/apache/beam/pull/11384
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] robertwb commented on a change in pull request #11370:
Update environments.py to add a method to specify container image
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370#discussion_r406545577
##########
File path: sdks/python/apache_beam/transforms/environments.py
##########
@@ -252,11 +252,18 @@ def from_runner_api_parameter(payload, capabilities, artifacts, context):
@classmethod
def from_options(cls, options):
# type: (PipelineOptions) -> DockerEnvironment
- return cls(
+ return from_container_image(
container_image=options.environment_config,
- capabilities=python_sdk_capabilities(),
artifacts=python_sdk_dependencies(options))
+ @classmethod
+ def from_container_image(cls, container_image, artifacts=()):
+ # type: (ContainerImage) -> DockerEnvironment
Review comment:
```suggestion
# type: (str) -> DockerEnvironment
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] kamilwu commented on issue #11370: Update environments.py to
add a method to specify container image
Posted by GitBox <gi...@apache.org>.
kamilwu commented on issue #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370#issuecomment-611962579
I'm getting the following error while executing Python pipeline with PortableRunner and --environment_type set to DOCKER:
```
File "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_ParDo_Flink_Batch_PR/src/sdks/python/apache_beam/runners/portability/portable_runner.py", line 282, in _create_environment
11:37:41 return env_class.from_options(portable_options)
11:37:41 File "/home/jenkins/jenkins-slave/workspace/beam_LoadTests_Python_ParDo_Flink_Batch_PR/src/sdks/python/apache_beam/transforms/environments.py", line 255, in from_options
11:37:41 return from_container_image(
11:37:41 NameError: name 'from_container_image' is not defined
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] robertwb commented on issue #11370: Update environments.py
to add a method to specify container image
Posted by GitBox <gi...@apache.org>.
robertwb commented on issue #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370#issuecomment-612167768
Sorry. I merged to be able to run tests of the PR that sat on top of it (they should have been one).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] pabloem commented on issue #11370: Update environments.py to
add a method to specify container image
Posted by GitBox <gi...@apache.org>.
pabloem commented on issue #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370#issuecomment-612176044
Created https://github.com/apache/beam/pull/11383 to fix if nobody else has something out
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] robertwb merged pull request #11370: Update environments.py
to add a method to specify container image
Posted by GitBox <gi...@apache.org>.
robertwb merged pull request #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] ibzib commented on issue #11370: Update environments.py to
add a method to specify container image
Posted by GitBox <gi...@apache.org>.
ibzib commented on issue #11370: Update environments.py to add a method to specify container image
URL: https://github.com/apache/beam/pull/11370#issuecomment-612083293
> NameError: name 'from_container_image' is not defined
I encountered this issue as well. Filed BEAM-9736 to track. Please fix and cherry pick onto 2.21 release branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services