You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/12/01 16:20:02 UTC
[GitHub] [beam] tysonjh commented on a change in pull request #13420: Rewrite container docs for custom containers
tysonjh commented on a change in pull request #13420:
URL: https://github.com/apache/beam/pull/13420#discussion_r533542428
##########
File path: website/www/site/content/en/documentation/runtime/environments.md
##########
@@ -87,77 +202,28 @@ python -m apache_beam.examples.wordcount \
--output=path/to/write/counts \
--runner=PortableRunner \
--job_endpoint=localhost:8099 \
---environment_config=path/to/container/image
+--environment_config="${IMAGE}:${TAG}"
{{< /highlight >}}
-## Building container images
-
-To build Beam SDK container images:
-
-1. Navigate to the root directory of the local copy of your Apache Beam.
-2. Run Gradle with the `docker` target. If you're [building a child image](#writing-new-dockerfiles), set the optional `--file` flag to the new Dockerfile. If you're [building an image from an original Dockerfile](#modifying-dockerfiles), ignore the `--file` flag:
-
-```
-# The default repository of each SDK
-./gradlew [--file=path/to/new/Dockerfile] :sdks:java:container:java8:docker
-./gradlew [--file=path/to/new/Dockerfile] :sdks:java:container:java11:docker
-./gradlew [--file=path/to/new/Dockerfile] :sdks:go:container:docker
-./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py2:docker
-./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py35:docker
-./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py36:docker
-./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container:py37:docker
-
-# Shortcut for building all four Python SDKs
-./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container buildAll
-```
-
-From 2.21.0, `docker-pull-licenses` tag was introduced. Licenses/notices of third party dependencies will be added to the docker images when `docker-pull-licenses` was set.
-For example, `./gradlew :sdks:java:container:java8:docker -Pdocker-pull-licenses`. The files are added to `/opt/apache/beam/third_party_licenses/`.
-By default, no licenses/notices are added to the docker images.
-
-To examine the containers that you built, run `docker images` from anywhere in the command line. If you successfully built all of the container images, the command prints a table like the following:
-```
-REPOSITORY TAG IMAGE ID CREATED SIZE
-apache/beam_java8_sdk latest ... 2 weeks ago ...
-apache/beam_java11_sdk latest ... 2 weeks ago ...
-apache/beam_python2.7_sdk latest ... 2 weeks ago ...
-apache/beam_python3.5_sdk latest ... 2 weeks ago ...
-apache/beam_python3.6_sdk latest ... 2 weeks ago ...
-apache/beam_python3.7_sdk latest ... 2 weeks ago ...
-apache/beam_go_sdk latest ... 2 weeks ago ...
-```
-
-### Overriding default Docker targets
-
-The default [tag](https://docs.docker.com/engine/reference/commandline/tag/) is sdk_version defined at [gradle.properties](https://github.com/apache/beam/blob/master/gradle.properties) and the default repositories are in the Docker Hub `apache` namespace.
-The `docker` command-line tool implicitly [pushes container images](#pushing-container-images) to this location.
+{{< highlight class="runner-dataflow" >}}
+export IMAGE="my-repo/beam_python_sdk_custom"
+export TAG="X.Y.Z"
-To tag a local image, set the `docker-tag` option when building the container. The following command tags a Python SDK image with a date.
-```
-./gradlew :sdks:python:container:py36:docker -Pdocker-tag=2019-10-04
-```
-
-To change the repository, set the `docker-repository-root` option to a new location. The following command sets the `docker-repository-root`
-to a repository named `example-repo` on Docker Hub.
-```
-./gradlew :sdks:python:container:py36:docker -Pdocker-repository-root=example-repo
-```
+export GCS_PATH="gs://my-gcs-bucket"
+export GCP_PROJECT="my-gcp-project"
+export REGION="us-central1"
-## Pushing container images
-
-After [building a container image](#building-container-images), you can store it in a remote Docker repository.
-
-The following steps push a Python3.6 SDK image to the [`docker-root-repository` value](#overriding-default-docker-targets).
-Please log in to the destination repository as needed.
-
-Upload it to the remote repository:
-```
-docker push example-repo/beam_python3.6_sdk
-```
-
-To download the image again, run `docker pull`:
-```
-docker pull example-repo/beam_python3.6_sdk
-```
+# Run a pipeline on Dataflow.
+# This is a Python batch pipeline, so to run on Dataflow Runner V2
+# you must specify the experiment "use_runner_v2"
Review comment:
Got it. I went to 'view file' in GitHub and it only showed the rendered markdown which made each of these H1 titles. Is there a way to preview?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org