You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/03/07 18:47:40 UTC
[GitHub] [beam] emilymye commented on a change in pull request #16938: [BEAM-13314]Revise recommendations to manage Python pipeline dependencies.
emilymye commented on a change in pull request #16938:
URL: https://github.com/apache/beam/pull/16938#discussion_r820993456
##########
File path: website/www/site/content/en/documentation/runtime/environments.md
##########
@@ -171,6 +171,49 @@ creates a Java 8 SDK image with appropriate licenses in `/opt/apache/beam/third_
By default, no licenses/notices are added to the docker images.
+#### Build an existing container image to make it compatible with Apache Beam Runners {#modify-existing-base-image}
Review comment:
```suggestion
#### Build a container image based on an existing image compatible with Apache Beam Runners {#modify-existing-base-image}
```
##########
File path: website/www/site/content/en/documentation/runtime/environments.md
##########
@@ -171,6 +171,49 @@ creates a Java 8 SDK image with appropriate licenses in `/opt/apache/beam/third_
By default, no licenses/notices are added to the docker images.
+#### Build an existing container image to make it compatible with Apache Beam Runners {#modify-existing-base-image}
+Beam offers a way to take a Beam container image and customize it. But if you have an existing base image to be compatible with Apache Beam Runners, use a [multi-stage build](https://docs.docker.com/develop/develop-images/multistage-build/) process to copy over the necessary artifacts from a default Apache Beam base image and provide your custom container image.
Review comment:
```suggestion
Beam offers a way to provide your own custom Beam container image. The easiest way to build a new custom image that is compatible with Apache Beam Runners is to use a [multi-stage build](https://docs.docker.com/develop/develop-images/multistage-build/) process. This copies over the necessary artifacts from a default Apache Beam base image to build your custom container image.
```
##########
File path: website/www/site/content/en/documentation/runtime/environments.md
##########
@@ -171,6 +171,49 @@ creates a Java 8 SDK image with appropriate licenses in `/opt/apache/beam/third_
By default, no licenses/notices are added to the docker images.
+#### Build an existing container image to make it compatible with Apache Beam Runners {#modify-existing-base-image}
+Beam offers a way to take a Beam container image and customize it. But if you have an existing base image to be compatible with Apache Beam Runners, use a [multi-stage build](https://docs.docker.com/develop/develop-images/multistage-build/) process to copy over the necessary artifacts from a default Apache Beam base image and provide your custom container image.
+
+
+1. Copy necessary artifacts from Apache Beam base image to your image.
+ ```
+ # This can be any container image,
+ FROM python:3.8-slim
+
+ # Install SDK. (needed for Python SDK)
+ RUN pip install --no-cache-dir apache-beam[gcp]==2.25.0
+
+ # Copy files from official SDK image, including script/dependencies.
+ COPY --from=apache/beam_python3.7_sdk:2.25.0 /opt/apache/beam /opt/apache/beam
+
+ # Perform any addtional customizations if desired
Review comment:
```suggestion
# Perform any additional customizations if desired
```
##########
File path: website/www/site/content/en/documentation/runtime/environments.md
##########
@@ -171,6 +171,49 @@ creates a Java 8 SDK image with appropriate licenses in `/opt/apache/beam/third_
By default, no licenses/notices are added to the docker images.
+#### Build an existing container image to make it compatible with Apache Beam Runners {#modify-existing-base-image}
+Beam offers a way to take a Beam container image and customize it. But if you have an existing base image to be compatible with Apache Beam Runners, use a [multi-stage build](https://docs.docker.com/develop/develop-images/multistage-build/) process to copy over the necessary artifacts from a default Apache Beam base image and provide your custom container image.
+
+
+1. Copy necessary artifacts from Apache Beam base image to your image.
+ ```
+ # This can be any container image,
+ FROM python:3.8-slim
+
+ # Install SDK. (needed for Python SDK)
+ RUN pip install --no-cache-dir apache-beam[gcp]==2.25.0
Review comment:
Can make the version here a build arg https://docs.docker.com/engine/reference/builder/#arg
e.g.
```
FROM python:3.8-slim
ARG beam_ver=2.25.0
...
RUN pip install --no-cache-dir apache-beam[gcp]==$beam_ver
COPY --from=apache/beam_python3.7_sdk:$beam_ver /opt/apache/beam /opt/apache/beam
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org