You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/03/20 17:53:00 UTC
(spark-connect-swift) branch main updated: [SPARK-56110] Use `Distroless` images for `Examples`
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new bea5e45 [SPARK-56110] Use `Distroless` images for `Examples`
bea5e45 is described below
commit bea5e459b854dc645a507fd530a28aebdb1e28fe
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Fri Mar 20 10:52:52 2026 -0700
[SPARK-56110] Use `Distroless` images for `Examples`
### What changes were proposed in this pull request?
This PR uses Google `Distress` images as the runtime base image for all Swift example Dockerfiles (`app`, `pi`, `spark-sql`, `stream`, `web`).
Key changes:
- Use `gcr.io/distroless/cc-debian13:nonroot` instead of `ubuntu:24.04` for runtime stage
- Remove manual user/group creation (`groupadd`/`useradd`) — distroless `nonroot` tag provides a built-in nonroot user (UID `65532`)
- Remove unnecessary `ENV`, `WORKDIR`, and shell invocations
- Use direct `ENTRYPOINT` with exec form instead of `/bin/sh -c` wrapper
- Update README image sizes accordingly.
### Why are the changes needed?
1. **Smaller image size**: Distroless images contain only the application and its runtime dependencies, reducing image size by ~21% (e.g., 85.1MB → 67.2MB).
```diff
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
-apache/spark-connect-swift:app 909c6511dbcc 373MB 85.1MB
+apache/spark-connect-swift:app 6a7bef590619 287MB 67.2MB
-apache/spark-connect-swift:pi c29b333727ad 373MB 85.1MB
+apache/spark-connect-swift:pi 10a2c9888186 287MB 67.2MB
-apache/spark-connect-swift:spark-sql f4ee9e7df9eb 373MB 85.1MB
+apache/spark-connect-swift:spark-sql 9a5b55dee3cc 287MB 67.2MB
-apache/spark-connect-swift:stream bd754ab46f92 373MB 85.1MB
+apache/spark-connect-swift:stream 152df15d747d 287MB 67.2MB
-apache/spark-connect-swift:web 6fd1ee541fc6 428MB 96.7MB
+apache/spark-connect-swift:web 95b8f92cbd3b 342MB 78.7MB
```
2. **Improved security**: Distroless images have a minimal attack surface — no shell, no package manager, no unnecessary OS utilities — reducing the risk of CVEs.
3. **Simplified Dockerfile**: Removing manual user management and shell wrappers makes the Dockerfiles cleaner and easier to maintain.
5. **Best practice**: Using `cc-debian13` variant (instead of `base`) provides `libstdc++.so.6` required by Swift binaries, while the `nonroot` tag ensures the container runs as a non-root user by default.
### Does this PR introduce _any_ user-facing change?
No. The example applications behave identically. The only observable difference is smaller Docker image sizes.
### How was this patch tested?
Docker images were built and verified to run correctly with the new distroless base images. Image sizes were measured and documented in the updated READMEs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-code/1.0)
Closes #332 from dongjoon-hyun/SPARK-56110.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
Examples/app/Dockerfile | 19 +++----------------
Examples/app/README.md | 2 +-
Examples/pi/Dockerfile | 19 +++----------------
Examples/pi/README.md | 2 +-
Examples/spark-sql/Dockerfile | 19 +++----------------
Examples/spark-sql/README.md | 2 +-
Examples/stream/Dockerfile | 19 +++----------------
Examples/stream/README.md | 2 +-
Examples/web/Dockerfile | 19 +++----------------
Examples/web/README.md | 2 +-
10 files changed, 20 insertions(+), 85 deletions(-)
diff --git a/Examples/app/Dockerfile b/Examples/app/Dockerfile
index 442dfab..bf2b0d1 100644
--- a/Examples/app/Dockerfile
+++ b/Examples/app/Dockerfile
@@ -22,25 +22,12 @@ COPY . .
RUN swift build -c release --static-swift-stdlib
-FROM ubuntu:24.04
-
-ARG SPARK_UID=185
+FROM gcr.io/distroless/cc-debian13:nonroot
LABEL org.opencontainers.image.authors="Apache Spark project <de...@spark.apache.org>"
LABEL org.opencontainers.image.licenses="Apache-2.0"
LABEL org.opencontainers.image.ref.name="Apache Spark Connect for Swift"
-ENV SPARK_APP_HOME=/opt/spark-app
-ENV SPARK_APP=SparkConnectSwiftApp
-
-WORKDIR $SPARK_APP_HOME
-
-RUN groupadd --system --gid=$SPARK_UID spark && \
- useradd --system --home-dir $SPARK_APP_HOME --uid=$SPARK_UID --gid=spark spark && \
- chown -R spark:spark $SPARK_APP_HOME
-
-COPY --from=builder --chown=spark:spark /app/.build/*-unknown-linux-gnu/release/$SPARK_APP .
-
-USER spark
+COPY --from=builder --chown=65532:65532 /app/.build/*-unknown-linux-gnu/release/SparkConnectSwiftApp /opt/spark-app/
-ENTRYPOINT ["/bin/sh", "-c", "$SPARK_APP_HOME/$SPARK_APP"]
+ENTRYPOINT ["/opt/spark-app/SparkConnectSwiftApp"]
diff --git a/Examples/app/README.md b/Examples/app/README.md
index ae3c6e6..c7d579c 100644
--- a/Examples/app/README.md
+++ b/Examples/app/README.md
@@ -16,7 +16,7 @@ Build an application Docker image.
$ docker build -t apache/spark-connect-swift:app .
$ docker images apache/spark-connect-swift:app
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
-apache/spark-connect-swift:app 909c6511dbcc 373MB 85.1MB
+apache/spark-connect-swift:app 6a7bef590619 287MB 67.2MB
```
Run `app` docker image.
diff --git a/Examples/pi/Dockerfile b/Examples/pi/Dockerfile
index 223de8f..811b431 100644
--- a/Examples/pi/Dockerfile
+++ b/Examples/pi/Dockerfile
@@ -22,25 +22,12 @@ COPY . .
RUN swift build -c release --static-swift-stdlib
-FROM ubuntu:24.04
-
-ARG SPARK_UID=185
+FROM gcr.io/distroless/cc-debian13:nonroot
LABEL org.opencontainers.image.authors="Apache Spark project <de...@spark.apache.org>"
LABEL org.opencontainers.image.licenses="Apache-2.0"
LABEL org.opencontainers.image.ref.name="Apache Spark Connect for Swift"
-ENV SPARK_APP_HOME=/opt/spark-app
-ENV SPARK_APP=SparkConnectSwiftPi
-
-WORKDIR $SPARK_APP_HOME
-
-RUN groupadd --system --gid=$SPARK_UID spark && \
- useradd --system --home-dir $SPARK_APP_HOME --uid=$SPARK_UID --gid=spark spark && \
- chown -R spark:spark $SPARK_APP_HOME
-
-COPY --from=builder --chown=spark:spark /app/.build/*-unknown-linux-gnu/release/$SPARK_APP .
-
-USER spark
+COPY --from=builder --chown=65532:65532 /app/.build/*-unknown-linux-gnu/release/SparkConnectSwiftPi /opt/spark-app/
-ENTRYPOINT ["/bin/sh", "-c", "$SPARK_APP_HOME/$SPARK_APP"]
+ENTRYPOINT ["/opt/spark-app/SparkConnectSwiftPi"]
diff --git a/Examples/pi/README.md b/Examples/pi/README.md
index 8f61d83..cc6ee00 100644
--- a/Examples/pi/README.md
+++ b/Examples/pi/README.md
@@ -16,7 +16,7 @@ Build an application Docker image.
$ docker build -t apache/spark-connect-swift:pi .
$ docker images apache/spark-connect-swift:pi
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
-apache/spark-connect-swift:pi c29b333727ad 373MB 85.1MB
+apache/spark-connect-swift:pi 10a2c9888186 287MB 67.2MB
```
Run `pi` docker image.
diff --git a/Examples/spark-sql/Dockerfile b/Examples/spark-sql/Dockerfile
index 2fab87e..e15a713 100644
--- a/Examples/spark-sql/Dockerfile
+++ b/Examples/spark-sql/Dockerfile
@@ -22,25 +22,12 @@ COPY . .
RUN swift build -c release --static-swift-stdlib
-FROM ubuntu:24.04
-
-ARG SPARK_UID=185
+FROM gcr.io/distroless/cc-debian13:nonroot
LABEL org.opencontainers.image.authors="Apache Spark project <de...@spark.apache.org>"
LABEL org.opencontainers.image.licenses="Apache-2.0"
LABEL org.opencontainers.image.ref.name="Apache Spark Connect for Swift"
-ENV SPARK_APP_HOME=/opt/spark-app
-ENV SPARK_APP=SparkConnectSwiftSQLRepl
-
-WORKDIR $SPARK_APP_HOME
-
-RUN groupadd --system --gid=$SPARK_UID spark && \
- useradd --system --home-dir $SPARK_APP_HOME --uid=$SPARK_UID --gid=spark spark && \
- chown -R spark:spark $SPARK_APP_HOME
-
-COPY --from=builder --chown=spark:spark /app/.build/*-unknown-linux-gnu/release/$SPARK_APP .
-
-USER spark
+COPY --from=builder --chown=65532:65532 /app/.build/*-unknown-linux-gnu/release/SparkConnectSwiftSQLRepl /opt/spark-app/
-ENTRYPOINT ["/bin/sh", "-c", "$SPARK_APP_HOME/$SPARK_APP"]
+ENTRYPOINT ["/opt/spark-app/SparkConnectSwiftSQLRepl"]
diff --git a/Examples/spark-sql/README.md b/Examples/spark-sql/README.md
index da056c6..3d85540 100644
--- a/Examples/spark-sql/README.md
+++ b/Examples/spark-sql/README.md
@@ -16,7 +16,7 @@ Build an application Docker image.
$ docker build -t apache/spark-connect-swift:spark-sql .
$ docker images apache/spark-connect-swift:spark-sql
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
-apache/spark-connect-swift:spark-sql f4ee9e7df9eb 373MB 85.1MB
+apache/spark-connect-swift:spark-sql 9a5b55dee3cc 287MB 67.2MB
```
Run `spark-sql` docker image.
diff --git a/Examples/stream/Dockerfile b/Examples/stream/Dockerfile
index 03100c3..8750b94 100644
--- a/Examples/stream/Dockerfile
+++ b/Examples/stream/Dockerfile
@@ -22,25 +22,12 @@ COPY . .
RUN swift build -c release --static-swift-stdlib
-FROM ubuntu:24.04
-
-ARG SPARK_UID=185
+FROM gcr.io/distroless/cc-debian13:nonroot
LABEL org.opencontainers.image.authors="Apache Spark project <de...@spark.apache.org>"
LABEL org.opencontainers.image.licenses="Apache-2.0"
LABEL org.opencontainers.image.ref.name="Apache Spark Connect for Swift"
-ENV SPARK_APP_HOME=/opt/spark-app
-ENV SPARK_APP=SparkConnectSwiftNetworkWordCount
-
-WORKDIR $SPARK_APP_HOME
-
-RUN groupadd --system --gid=$SPARK_UID spark && \
- useradd --system --home-dir $SPARK_APP_HOME --uid=$SPARK_UID --gid=spark spark && \
- chown -R spark:spark $SPARK_APP_HOME
-
-COPY --from=builder --chown=spark:spark /app/.build/*-unknown-linux-gnu/release/$SPARK_APP .
-
-USER spark
+COPY --from=builder --chown=65532:65532 /app/.build/*-unknown-linux-gnu/release/SparkConnectSwiftNetworkWordCount /opt/spark-app/
-ENTRYPOINT ["/bin/sh", "-c", "$SPARK_APP_HOME/$SPARK_APP"]
+ENTRYPOINT ["/opt/spark-app/SparkConnectSwiftNetworkWordCount"]
diff --git a/Examples/stream/README.md b/Examples/stream/README.md
index f59eac6..8e1cd4b 100644
--- a/Examples/stream/README.md
+++ b/Examples/stream/README.md
@@ -24,7 +24,7 @@ Build an application Docker image.
$ docker build -t apache/spark-connect-swift:stream .
$ docker images apache/spark-connect-swift:stream
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
-apache/spark-connect-swift:stream bd754ab46f92 373MB 85.1MB
+apache/spark-connect-swift:stream 152df15d747d 287MB 67.2MB
```
Run `stream` docker image.
diff --git a/Examples/web/Dockerfile b/Examples/web/Dockerfile
index 29acb2c..2dd09c9 100644
--- a/Examples/web/Dockerfile
+++ b/Examples/web/Dockerfile
@@ -22,27 +22,14 @@ COPY . .
RUN swift build -c release --static-swift-stdlib
-FROM ubuntu:24.04
-
-ARG SPARK_UID=185
+FROM gcr.io/distroless/cc-debian13:nonroot
LABEL org.opencontainers.image.authors="Apache Spark project <de...@spark.apache.org>"
LABEL org.opencontainers.image.licenses="Apache-2.0"
LABEL org.opencontainers.image.ref.name="Apache Spark Connect for Swift"
-ENV SPARK_APP_HOME=/opt/spark-app
-ENV SPARK_APP=SparkConnectSwiftWeb
-
-WORKDIR $SPARK_APP_HOME
-
-RUN groupadd --system --gid=$SPARK_UID spark && \
- useradd --system --home-dir $SPARK_APP_HOME --uid=$SPARK_UID --gid=spark spark && \
- chown -R spark:spark $SPARK_APP_HOME
-
-COPY --from=builder --chown=spark:spark /app/.build/*-unknown-linux-gnu/release/$SPARK_APP .
-
-USER spark
+COPY --from=builder --chown=65532:65532 /app/.build/*-unknown-linux-gnu/release/SparkConnectSwiftWeb /opt/spark-app/
EXPOSE 8080
-ENTRYPOINT ["/bin/sh", "-c", "$SPARK_APP_HOME/$SPARK_APP"]
+ENTRYPOINT ["/opt/spark-app/SparkConnectSwiftWeb"]
diff --git a/Examples/web/README.md b/Examples/web/README.md
index 9a6e3f7..304b15d 100644
--- a/Examples/web/README.md
+++ b/Examples/web/README.md
@@ -99,7 +99,7 @@ Build an application Docker image.
$ docker build -t apache/spark-connect-swift:web .
$ docker images apache/spark-connect-swift:web
IMAGE ID DISK USAGE CONTENT SIZE EXTRA
-apache/spark-connect-swift:web 6fd1ee541fc6 428MB 96.7MB
+apache/spark-connect-swift:web 95b8f92cbd3b 342MB 78.7MB
```
Run `web` docker image
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org