You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "merlimat (via GitHub)" <gi...@apache.org> on 2024/03/04 20:25:51 UTC
Re: [PR] [improve][pip] PIP-324: Alpine image [pulsar]
merlimat commented on code in PR #22054:
URL: https://github.com/apache/pulsar/pull/22054#discussion_r1511750645
##########
docker/pulsar/Dockerfile:
##########
@@ -26,82 +26,83 @@ ADD ${PULSAR_TARBALL} /
RUN mv /apache-pulsar-* /pulsar
RUN rm -rf /pulsar/bin/*.cmd
-COPY scripts/apply-config-from-env.py /pulsar/bin
-COPY scripts/apply-config-from-env-with-prefix.py /pulsar/bin
-COPY scripts/gen-yml-from-env.py /pulsar/bin
-COPY scripts/generate-zookeeper-config.sh /pulsar/bin
-COPY scripts/pulsar-zookeeper-ruok.sh /pulsar/bin
-COPY scripts/watch-znode.py /pulsar/bin
-COPY scripts/install-pulsar-client.sh /pulsar/bin
+COPY scripts/* /pulsar/bin/
# The final image needs to give the root group sufficient permission for Pulsar components
# to write to specific directories within /pulsar
# The file permissions are preserved when copying files from this builder image to the target image.
-RUN for SUBDIRECTORY in conf data download logs; do \
+RUN for SUBDIRECTORY in conf data download logs instances/deps; do \
[ -d /pulsar/$SUBDIRECTORY ] || mkdir /pulsar/$SUBDIRECTORY; \
- chmod -R g+w /pulsar/$SUBDIRECTORY; \
+ chmod -R g+rwx /pulsar/$SUBDIRECTORY; \
done
-### Create 2nd stage from Ubuntu image
-### and add OpenJDK and Python dependencies (for Pulsar functions)
+RUN chmod -R g+rx /pulsar/bin
+RUN chmod -R o+rx /pulsar
-FROM ubuntu:22.04
+## Create 2nd stage to build the Python dependencies
+## Since it needs to have GCC available, we're doing it in a different layer
+FROM alpine:3.19 AS python-deps
+
+RUN apk add --no-cache \
+ bash \
+ python3-dev \
+ g++ \
+ musl-dev \
+ libffi-dev \
+ py3-pip \
+ py3-grpcio \
+ py3-yaml
+
+RUN pip3 install --break-system-packages \
+ kazoo
+
+ARG PULSAR_CLIENT_PYTHON_VERSION
+RUN pip3 install --break-system-packages \
+ pulsar-client[all]==${PULSAR_CLIENT_PYTHON_VERSION}
-ARG DEBIAN_FRONTEND=noninteractive
-ARG UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/
-ARG UBUNTU_SECURITY_MIRROR=http://security.ubuntu.com/ubuntu/
-ARG DEFAULT_USERNAME=pulsar
-ARG JDK_MAJOR_VERSION=17
+
+## Create final stage from Alpine image
+## and add OpenJDK and Python dependencies (for Pulsar functions)
+FROM alpine:3.19.1
+ENV LANG C.UTF-8
# Install some utilities
-RUN sed -i -e "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}|g" \
- -e "s|http://security\.ubuntu\.com/ubuntu/|${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}|g" /etc/apt/sources.list \
- && echo 'Acquire::http::Timeout "30";\nAcquire::http::ConnectionAttemptDelayMsec "2000";\nAcquire::https::Timeout "30";\nAcquire::https::ConnectionAttemptDelayMsec "2000";\nAcquire::ftp::Timeout "30";\nAcquire::ftp::ConnectionAttemptDelayMsec "2000";\nAcquire::Retries "15";' > /etc/apt/apt.conf.d/99timeout_and_retries \
- && apt-get update \
- && apt-get -y dist-upgrade \
- && apt-get -y install netcat dnsutils less procps iputils-ping \
- curl ca-certificates wget apt-transport-https \
- && apt-get -y install --no-install-recommends python3 python3-kazoo python3-pip
-
-# Install Eclipse Temurin Package
-RUN mkdir -p /etc/apt/keyrings \
- && wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc \
- && echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list \
- && apt-get update \
- && apt-get -y dist-upgrade \
- && apt-get -y install temurin-${JDK_MAJOR_VERSION:-17}-jdk \
- && export ARCH=$(uname -m | sed -r 's/aarch64/arm64/g' | awk '!/arm64/{$0="amd64"}1') \
- && echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/temurin-${JDK_MAJOR_VERSION:-17}-jdk-$ARCH/conf/security/java.security \
- && echo networkaddress.cache.negative.ttl=1 >> /usr/lib/jvm/temurin-${JDK_MAJOR_VERSION:-17}-jdk-$ARCH/conf/security/java.security \
-
-# Cleanup apt
-RUN apt-get -y --purge autoremove \
- && apt-get autoclean \
- && apt-get clean \
- && rm -rf /var/lib/apt/lists/*
-
-RUN pip3 install pyyaml==6.0.1
-
-# Pulsar currently writes to the below directories, assuming the default configuration.
-# Note that number 4 is the reason that pulsar components need write access to the /pulsar directory.
-# 1. /pulsar/data - both bookkeepers and zookeepers use this directory
-# 2. /pulsar/logs - function workers write to this directory and pulsar-admin initializes this directory
-# 3. /pulsar/download - functions write to this directory
-# 4. /pulsar - hadoop writes to this directory
-RUN mkdir /pulsar && chmod g+w /pulsar
+RUN apk add --no-cache \
+ bash \
+ python3 \
+ py3-pip \
+ ca-certificates
+
+RUN wget -O /etc/apk/keys/amazoncorretto.rsa.pub https://apk.corretto.aws/amazoncorretto.rsa.pub && \
+ echo "https://apk.corretto.aws" >> /etc/apk/repositories && \
+ apk add --no-cache amazon-corretto-21 binutils && \
+ # Use JLink to create a slimmer JDK distribution (see: https://adoptium.net/blog/2021/10/jlink-to-produce-own-runtime/)
+ # This still includes all JDK modules, though in the future we could compile a list of required modules
+ /usr/lib/jvm/default-jvm/bin/jlink --add-modules ALL-MODULE-PATH --compress zip-9 --no-man-pages --no-header-files --strip-debug --output /opt/corretto-slim && \
+ apk del binutils amazon-corretto-21 && \
+ mkdir -p /usr/lib/jvm/ && \
+ mv /opt/corretto-slim /usr/lib/jvm/java-21-amazon-corretto && \
+ ln -sfn /usr/lib/jvm/java-21-amazon-corretto /usr/lib/jvm/default-jvm
+
+ENV JAVA_HOME=/usr/lib/jvm/default-jvm
+
+RUN echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/default-jvm/conf/security/java.security
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org