(beam) branch master updated: Add instructions for Go custom container builds, link from 2.55.0 release notes. (#30290)

[lo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

lostluck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
     new 9a4e1144532 Add instructions for Go custom container builds, link from 2.55.0 release notes. (#30290)
9a4e1144532 is described below

commit 9a4e1144532291244cfe6fde7f99655de1e9c88d
Author: Robert Burke <lo...@users.noreply.github.com>
AuthorDate: Tue Feb 13 13:23:26 2024 -0800

    Add instructions for Go custom container builds, link from 2.55.0 release notes. (#30290)
    
    * Stopgap message in release notes.
    
    * Recommend 2.54.0 not 2.53.0.
    
    * Add from scratch instructions.
    
    * remove specific go version
    
    * ws
    
    * redundant line
    
    * rm more ws
    
    ---------
    
    Co-authored-by: lostluck <13...@users.noreply.github.com>
---
 CHANGES.md                                         |  3 ++
 .../en/documentation/runtime/environments.md       | 40 +++++++++++++++++++++-
 2 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/CHANGES.md b/CHANGES.md
index bda5937a93d..607f1a1185f 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -72,6 +72,9 @@
 
 * X behavior was changed ([#X](https://github.com/apache/beam/issues/X)).
 * Arrow version was bumped to 15.0.0 from 5.0.0 ([#30181](https://github.com/apache/beam/pull/30181)).
+* Go SDK users who build custom worker containers may run into issues with the move to distroless containers as a base (see Security Fixes).
+  * The issue stems from distroless containers lacking additional tools, which current custom container processes may rely on.
+  * See https://beam.apache.org/documentation/runtime/environments/#from-scratch-go for instructions on building and using a custom container.
 
 ## Deprecations
 
diff --git a/website/www/site/content/en/documentation/runtime/environments.md b/website/www/site/content/en/documentation/runtime/environments.md
index c860816e300..d9a42db29e2 100644
--- a/website/www/site/content/en/documentation/runtime/environments.md
+++ b/website/www/site/content/en/documentation/runtime/environments.md
@@ -29,7 +29,7 @@ You may want to customize container images for many reasons, including:
 * Launching third-party software in the worker environment
 * Further customizing the execution environment
 
- This guide describes how to create and use customized containers for the Beam SDK.
+ This guide describes how to create and use customized containers for the Beam SDKs.
 
 ### Prerequisites
 
@@ -218,6 +218,44 @@ Beam offers a way to provide your own custom container image. The easiest way to
   docker push "${IMAGE_NAME}:${TAG}"
   ```
 
+#### Building a compatible container image from scratch (Go) {#from-scratch-go}
+
+From the 2.55.0 release, the Beam Go SDK has moved to using [distroless images](https://github.com/GoogleContainerTools/distroless) as a base.
+These images have a reduced security attack surface by not including common tools and utilities.
+This may cause difficulties customizing the image with using one of the above approaches.
+As a fallback, it's possible to build a custom image from scratch, by building a matching boot loader, and setting
+that as the container's entry point.
+
+For example, if it's preferable to use alpine as the container OS your multi-stage docker file might
+look like the following:
+
+```
+FROM golang:latest-alpine AS build_base
+
+# Set the Current Working Directory inside the container
+WORKDIR /tmp/beam
+
+# Build the Beam Go bootloader, to the local directory, matching your Beam version.
+# Similar go targets exist for other SDK languages.
+RUN GOBIN=`pwd` go install github.com/apache/beam/sdks/v2/go/container@v2.53.0
+
+# Set the real base image.
+FROM alpine:3.9
+RUN apk add ca-certificates
+
+# The following are required for the container to operate correctly.
+# Copy the boot loader `container` to the image.
+COPY --from=build_base /tmp/beam/container /opt/apache/beam/boot
+
+# Set the container to use the newly built boot loader.
+ENTRYPOINT ["/opt/apache/beam/boot"]
+```
+
+Build and push the new image as when [modifying an existing base image](#modify-existing-base-image) above.
+
+>**NOTE**: Java and Python require additional dependencies, such as their runtimes, and SDK packages for
+> a valid container image. The bootloader isn't sufficient for creating a custom container for these SDKs.
+
 ## Running pipelines with custom container images {#running-pipelines}
 
 The common method for providing a container image requires using the