You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ma...@apache.org on 2023/03/23 19:06:11 UTC
[flink-docker] branch master updated (c975488 -> 6b2e9e2)
This is an automated email from the ASF dual-hosted git repository.
mapohl pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink-docker.git
from c975488 Update Dockerfiles for 1.15.4 release
new 1cb66d3 [FLINK-31566] Update Dockerfiles for 1.17.0 release
new 07dab5c [FLINK-31566] Updates .github/workflows/snapshot.yml to point to make all supported versions point to the right dev-* branches
new 6b2e9e2 [hotfix][docs] Updates README.md to match the current process
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/snapshot.yml | 4 +++-
{1.15 => 1.17}/scala_2.12-java11-ubuntu/Dockerfile | 6 +++---
.../scala_2.12-java11-ubuntu/docker-entrypoint.sh | 0
1.17/scala_2.12-java11-ubuntu/release.metadata | 2 ++
{1.16 => 1.17}/scala_2.12-java8-ubuntu/Dockerfile | 6 +++---
.../scala_2.12-java8-ubuntu}/docker-entrypoint.sh | 0
1.17/scala_2.12-java8-ubuntu/release.metadata | 2 ++
README.md | 24 ++++++++++------------
8 files changed, 24 insertions(+), 20 deletions(-)
copy {1.15 => 1.17}/scala_2.12-java11-ubuntu/Dockerfile (94%)
copy {1.15 => 1.17}/scala_2.12-java11-ubuntu/docker-entrypoint.sh (100%)
create mode 100644 1.17/scala_2.12-java11-ubuntu/release.metadata
copy {1.16 => 1.17}/scala_2.12-java8-ubuntu/Dockerfile (94%)
copy {1.15/scala_2.12-java11-ubuntu => 1.17/scala_2.12-java8-ubuntu}/docker-entrypoint.sh (100%)
create mode 100644 1.17/scala_2.12-java8-ubuntu/release.metadata
[flink-docker] 03/03: [hotfix][docs] Updates README.md to match the current process
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mapohl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-docker.git
commit 6b2e9e2d6f4c0c79bc6cb32464ee730681fb5171
Author: Matthias Pohl <ma...@aiven.io>
AuthorDate: Thu Mar 23 11:53:57 2023 +0100
[hotfix][docs] Updates README.md to match the current process
- Adds item check for `.github/workflows/snapshot.yml`
- Adds item updating the .github/workflows/snapshot.yml
- Updated the instructions on how to create a branch (following what's already documented in the Flink release docs).
- Removed step in checklist to update the aliases array in generate-stackbrew-library.sh: This became obsolete with the changes that happened in e47a802f
- Moved step describing the Flink minor version branch creation and updated it slightly to match the release documentation
- Removed step for updating the base version in testing/run_travis_tests.sh: This is obsolete with the changes that were done in FLINK-27488
- Removed step asking to update the latest tag in the script: This is not necessary anymore after the changes that have been committed with e47a802f.
---
README.md | 24 +++++++++++-------------
1 file changed, 11 insertions(+), 13 deletions(-)
diff --git a/README.md b/README.md
index 1d3cb46..ae52863 100644
--- a/README.md
+++ b/README.md
@@ -34,6 +34,14 @@ Development happens on the various `dev-X` branches.
Pull requests for a specific version should be opened against the respective `dev-<version>` branch.
Pull requests for all versions, or for the next major Flink release, should be opened against the `dev-master` branch.
+For new major Flink releases, once the release branch is created in the [apache/flink](https://github.com/apache/flink/) repo, the corresponding `dev-x.y`
+branch must be created in `apache/flink-docker`:
+1. Create the branch based on `dev-master`
+2. Update `.github/workflows/ci.yml` in the corresponding `dev-*` branches: Make sure that the correct
+ snapshot version is stated; for `dev-x.y` it should point to `x.y-SNAPSHOT`, while for `dev-master`
+ it should point to the most recent snapshot version (i.e. `x.(y+1)-SNAPSHOT`).
+3. Update `.github/workflows/snapshot.yml` in `master` mapping each (supported) version to the right branch.
+
### CI
The `dev-master` branch is tested against nightly Flink snapshots for the next major Flink version. This allows us to
@@ -51,9 +59,6 @@ There are additional steps required when a new Flink major version (x.y.0) is re
* Since only the current and previous major versions of Flink are supported, the Dockerfiles for
older versions must be removed when adding the new version to this repo
-* The new images should be given the `latest` tag, so the `aliases` array in
- `generate-stackbrew-library.sh` must be updated
-
### Release workflow
@@ -72,8 +77,7 @@ Updating the Dockerfiles involves the following steps:
* Be sure to use the full fingerprint of the GPG key (as shown in the example below), as the official images require this.
* Commit this change with message `Add GPG key for x.y.z release` <sup>\[[example](
https://github.com/apache/flink-docker/commit/94845f46c0f0f2de80d4a5ce309db49aff4655d0)]</sup>
- * (minor only) Update `testing/run_travis_tests.sh` to test against the new minor version.
- * Create a pull request against the `dev-x.y`/`dev-master` branch containing these commits.
+ * Create a pull request against the `dev-x.y`/`dev-master` branch containing this commit.
* Run `add-version.sh` with the appropriate arguments (`-r flink-major-version -f flink-full-version`)
* e.g. `./add-version.sh -r 1.2 -f 1.2.1`
2. Update Dockerfiles on the the `master` branch
@@ -111,18 +115,14 @@ https://github.com/docker-library/official-images/pull/10665)]</sup>
Once the pull request has been merged (often within 1 business day), the new images will be
available shortly thereafter.
-For new major Flink releases, once the new image is available, the `dev-x.y` branch must be created:
-1. Create the branch based on `dev-master`
-2. update `testing/run_travis_tests.sh`:
- * replace usage of `./add-custom.sh` with `./add-version.sh -r x.y -f x.y.0`
- * replace references to `dev-master` with `dev-x.y`
-
### Release checklist
Checklist for the `dev` branch:
- [ ] The GPG key ID of the key used to sign the release has been added to `add-version.sh` and
committed with the message `Add GPG key for x.y.z release`
- [ ] `./add-version.sh -r x.y -f x.y.z` has been run on the respective dev branch
+- [ ] `.github/workflows/snapshot.yml` in `master` has each supported version being mapped to the right branch.
+ Deprecated versions should have been removed.
Checklist for the `master` branch:
- [ ] _(new major releases only)_ any unsupported Flink major version Dockerfiles have been removed
@@ -130,8 +130,6 @@ Checklist for the `master` branch:
- [ ] _(new minor releases only)_ any existing generated files for the same major version have been
removed
- [ ] The updated Dockerfiles have been committed with the message `Update Dockerfiles for x.y.z release`
-- [ ] _(new major releases only)_ the `aliases` array in `generate-stackbrew-library.sh` has been
- updated with `[x.y]='latest'` and committed with the message `Update latest image tag to x.y`
- [ ] A pull request with the above changes has been opened on this repo and merged
- [ ] The new library manifest has been generated with `generate-stackbrew-library.sh` and a pull
request opened on the `official-images` repo with commit message `Update to Flink x.y.z`
[flink-docker] 01/03: [FLINK-31566] Update Dockerfiles for 1.17.0 release
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mapohl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-docker.git
commit 1cb66d334bd1a1a9a7627c47478a31afed542aab
Author: Matthias Pohl <ma...@aiven.io>
AuthorDate: Thu Mar 23 12:40:07 2023 +0100
[FLINK-31566] Update Dockerfiles for 1.17.0 release
---
1.17/scala_2.12-java11-ubuntu/Dockerfile | 94 +++++++++++++
1.17/scala_2.12-java11-ubuntu/docker-entrypoint.sh | 152 +++++++++++++++++++++
1.17/scala_2.12-java11-ubuntu/release.metadata | 2 +
1.17/scala_2.12-java8-ubuntu/Dockerfile | 94 +++++++++++++
1.17/scala_2.12-java8-ubuntu/docker-entrypoint.sh | 152 +++++++++++++++++++++
1.17/scala_2.12-java8-ubuntu/release.metadata | 2 +
6 files changed, 496 insertions(+)
diff --git a/1.17/scala_2.12-java11-ubuntu/Dockerfile b/1.17/scala_2.12-java11-ubuntu/Dockerfile
new file mode 100644
index 0000000..b43dd26
--- /dev/null
+++ b/1.17/scala_2.12-java11-ubuntu/Dockerfile
@@ -0,0 +1,94 @@
+###############################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+###############################################################################
+
+FROM eclipse-temurin:11-jre-jammy
+
+# Install dependencies
+RUN set -ex; \
+ apt-get update; \
+ apt-get -y install gpg libsnappy1v5 gettext-base libjemalloc-dev; \
+ rm -rf /var/lib/apt/lists/*
+
+# Grab gosu for easy step-down from root
+ENV GOSU_VERSION 1.11
+RUN set -ex; \
+ wget -nv -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)"; \
+ wget -nv -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc"; \
+ export GNUPGHOME="$(mktemp -d)"; \
+ for server in ha.pool.sks-keyservers.net $(shuf -e \
+ hkp://p80.pool.sks-keyservers.net:80 \
+ keyserver.ubuntu.com \
+ hkp://keyserver.ubuntu.com:80 \
+ pgp.mit.edu) ; do \
+ gpg --batch --keyserver "$server" --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && break || : ; \
+ done && \
+ gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
+ gpgconf --kill all; \
+ rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
+ chmod +x /usr/local/bin/gosu; \
+ gosu nobody true
+
+# Configure Flink version
+ENV FLINK_TGZ_URL=https://www.apache.org/dyn/closer.cgi?action=download&filename=flink/flink-1.17.0/flink-1.17.0-bin-scala_2.12.tgz \
+ FLINK_ASC_URL=https://www.apache.org/dist/flink/flink-1.17.0/flink-1.17.0-bin-scala_2.12.tgz.asc \
+ GPG_KEY=A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B \
+ CHECK_GPG=true
+
+# Prepare environment
+ENV FLINK_HOME=/opt/flink
+ENV PATH=$FLINK_HOME/bin:$PATH
+RUN groupadd --system --gid=9999 flink && \
+ useradd --system --home-dir $FLINK_HOME --uid=9999 --gid=flink flink
+WORKDIR $FLINK_HOME
+
+# Install Flink
+RUN set -ex; \
+ wget -nv -O flink.tgz "$FLINK_TGZ_URL"; \
+ \
+ if [ "$CHECK_GPG" = "true" ]; then \
+ wget -nv -O flink.tgz.asc "$FLINK_ASC_URL"; \
+ export GNUPGHOME="$(mktemp -d)"; \
+ for server in ha.pool.sks-keyservers.net $(shuf -e \
+ hkp://p80.pool.sks-keyservers.net:80 \
+ keyserver.ubuntu.com \
+ hkp://keyserver.ubuntu.com:80 \
+ pgp.mit.edu) ; do \
+ gpg --batch --keyserver "$server" --recv-keys "$GPG_KEY" && break || : ; \
+ done && \
+ gpg --batch --verify flink.tgz.asc flink.tgz; \
+ gpgconf --kill all; \
+ rm -rf "$GNUPGHOME" flink.tgz.asc; \
+ fi; \
+ \
+ tar -xf flink.tgz --strip-components=1; \
+ rm flink.tgz; \
+ \
+ chown -R flink:flink .; \
+ \
+ # Replace default REST/RPC endpoint bind address to use the container's network interface \
+ sed -i 's/rest.address: localhost/rest.address: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i 's/rest.bind-address: localhost/rest.bind-address: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i 's/jobmanager.bind-host: localhost/jobmanager.bind-host: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i 's/taskmanager.bind-host: localhost/taskmanager.bind-host: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i '/taskmanager.host: localhost/d' $FLINK_HOME/conf/flink-conf.yaml;
+
+# Configure container
+COPY docker-entrypoint.sh /
+ENTRYPOINT ["/docker-entrypoint.sh"]
+EXPOSE 6123 8081
+CMD ["help"]
diff --git a/1.17/scala_2.12-java11-ubuntu/docker-entrypoint.sh b/1.17/scala_2.12-java11-ubuntu/docker-entrypoint.sh
new file mode 100755
index 0000000..8b0350e
--- /dev/null
+++ b/1.17/scala_2.12-java11-ubuntu/docker-entrypoint.sh
@@ -0,0 +1,152 @@
+#!/usr/bin/env bash
+
+###############################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+###############################################################################
+
+COMMAND_STANDALONE="standalone-job"
+COMMAND_HISTORY_SERVER="history-server"
+
+# If unspecified, the hostname of the container is taken as the JobManager address
+JOB_MANAGER_RPC_ADDRESS=${JOB_MANAGER_RPC_ADDRESS:-$(hostname -f)}
+CONF_FILE="${FLINK_HOME}/conf/flink-conf.yaml"
+
+drop_privs_cmd() {
+ if [ $(id -u) != 0 ]; then
+ # Don't need to drop privs if EUID != 0
+ return
+ elif [ -x /sbin/su-exec ]; then
+ # Alpine
+ echo su-exec flink
+ else
+ # Others
+ echo gosu flink
+ fi
+}
+
+copy_plugins_if_required() {
+ if [ -z "$ENABLE_BUILT_IN_PLUGINS" ]; then
+ return 0
+ fi
+
+ echo "Enabling required built-in plugins"
+ for target_plugin in $(echo "$ENABLE_BUILT_IN_PLUGINS" | tr ';' ' '); do
+ echo "Linking ${target_plugin} to plugin directory"
+ plugin_name=${target_plugin%.jar}
+
+ mkdir -p "${FLINK_HOME}/plugins/${plugin_name}"
+ if [ ! -e "${FLINK_HOME}/opt/${target_plugin}" ]; then
+ echo "Plugin ${target_plugin} does not exist. Exiting."
+ exit 1
+ else
+ ln -fs "${FLINK_HOME}/opt/${target_plugin}" "${FLINK_HOME}/plugins/${plugin_name}"
+ echo "Successfully enabled ${target_plugin}"
+ fi
+ done
+}
+
+set_config_option() {
+ local option=$1
+ local value=$2
+
+ # escape periods for usage in regular expressions
+ local escaped_option=$(echo ${option} | sed -e "s/\./\\\./g")
+
+ # either override an existing entry, or append a new one
+ if grep -E "^${escaped_option}:.*" "${CONF_FILE}" > /dev/null; then
+ sed -i -e "s/${escaped_option}:.*/$option: $value/g" "${CONF_FILE}"
+ else
+ echo "${option}: ${value}" >> "${CONF_FILE}"
+ fi
+}
+
+prepare_configuration() {
+ set_config_option jobmanager.rpc.address ${JOB_MANAGER_RPC_ADDRESS}
+ set_config_option blob.server.port 6124
+ set_config_option query.server.port 6125
+
+ if [ -n "${TASK_MANAGER_NUMBER_OF_TASK_SLOTS}" ]; then
+ set_config_option taskmanager.numberOfTaskSlots ${TASK_MANAGER_NUMBER_OF_TASK_SLOTS}
+ fi
+
+ if [ -n "${FLINK_PROPERTIES}" ]; then
+ echo "${FLINK_PROPERTIES}" >> "${CONF_FILE}"
+ fi
+ envsubst < "${CONF_FILE}" > "${CONF_FILE}.tmp" && mv "${CONF_FILE}.tmp" "${CONF_FILE}"
+}
+
+maybe_enable_jemalloc() {
+ if [ "${DISABLE_JEMALLOC:-false}" == "false" ]; then
+ JEMALLOC_PATH="/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so"
+ JEMALLOC_FALLBACK="/usr/lib/x86_64-linux-gnu/libjemalloc.so"
+ if [ -f "$JEMALLOC_PATH" ]; then
+ export LD_PRELOAD=$LD_PRELOAD:$JEMALLOC_PATH
+ elif [ -f "$JEMALLOC_FALLBACK" ]; then
+ export LD_PRELOAD=$LD_PRELOAD:$JEMALLOC_FALLBACK
+ else
+ if [ "$JEMALLOC_PATH" = "$JEMALLOC_FALLBACK" ]; then
+ MSG_PATH=$JEMALLOC_PATH
+ else
+ MSG_PATH="$JEMALLOC_PATH and $JEMALLOC_FALLBACK"
+ fi
+ echo "WARNING: attempted to load jemalloc from $MSG_PATH but the library couldn't be found. glibc will be used instead."
+ fi
+ fi
+}
+
+maybe_enable_jemalloc
+
+copy_plugins_if_required
+
+prepare_configuration
+
+args=("$@")
+if [ "$1" = "help" ]; then
+ printf "Usage: $(basename "$0") (jobmanager|${COMMAND_STANDALONE}|taskmanager|${COMMAND_HISTORY_SERVER})\n"
+ printf " Or $(basename "$0") help\n\n"
+ printf "By default, Flink image adopts jemalloc as default memory allocator. This behavior can be disabled by setting the 'DISABLE_JEMALLOC' environment variable to 'true'.\n"
+ exit 0
+elif [ "$1" = "jobmanager" ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting Job Manager"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/jobmanager.sh" start-foreground "${args[@]}"
+elif [ "$1" = ${COMMAND_STANDALONE} ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting Job Manager"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/standalone-job.sh" start-foreground "${args[@]}"
+elif [ "$1" = ${COMMAND_HISTORY_SERVER} ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting History Server"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/historyserver.sh" start-foreground "${args[@]}"
+elif [ "$1" = "taskmanager" ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting Task Manager"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/taskmanager.sh" start-foreground "${args[@]}"
+fi
+
+args=("${args[@]}")
+
+# Running command in pass-through mode
+exec $(drop_privs_cmd) "${args[@]}"
diff --git a/1.17/scala_2.12-java11-ubuntu/release.metadata b/1.17/scala_2.12-java11-ubuntu/release.metadata
new file mode 100644
index 0000000..770b58e
--- /dev/null
+++ b/1.17/scala_2.12-java11-ubuntu/release.metadata
@@ -0,0 +1,2 @@
+Tags: 1.17.0-scala_2.12-java11, 1.17-scala_2.12-java11, scala_2.12-java11, 1.17.0-scala_2.12, 1.17-scala_2.12, scala_2.12, 1.17.0-java11, 1.17-java11, java11, 1.17.0, 1.17, latest
+Architectures: amd64,arm64v8
diff --git a/1.17/scala_2.12-java8-ubuntu/Dockerfile b/1.17/scala_2.12-java8-ubuntu/Dockerfile
new file mode 100644
index 0000000..b26e633
--- /dev/null
+++ b/1.17/scala_2.12-java8-ubuntu/Dockerfile
@@ -0,0 +1,94 @@
+###############################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+###############################################################################
+
+FROM eclipse-temurin:8-jre-jammy
+
+# Install dependencies
+RUN set -ex; \
+ apt-get update; \
+ apt-get -y install gpg libsnappy1v5 gettext-base libjemalloc-dev; \
+ rm -rf /var/lib/apt/lists/*
+
+# Grab gosu for easy step-down from root
+ENV GOSU_VERSION 1.11
+RUN set -ex; \
+ wget -nv -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)"; \
+ wget -nv -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc"; \
+ export GNUPGHOME="$(mktemp -d)"; \
+ for server in ha.pool.sks-keyservers.net $(shuf -e \
+ hkp://p80.pool.sks-keyservers.net:80 \
+ keyserver.ubuntu.com \
+ hkp://keyserver.ubuntu.com:80 \
+ pgp.mit.edu) ; do \
+ gpg --batch --keyserver "$server" --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 && break || : ; \
+ done && \
+ gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu; \
+ gpgconf --kill all; \
+ rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc; \
+ chmod +x /usr/local/bin/gosu; \
+ gosu nobody true
+
+# Configure Flink version
+ENV FLINK_TGZ_URL=https://www.apache.org/dyn/closer.cgi?action=download&filename=flink/flink-1.17.0/flink-1.17.0-bin-scala_2.12.tgz \
+ FLINK_ASC_URL=https://www.apache.org/dist/flink/flink-1.17.0/flink-1.17.0-bin-scala_2.12.tgz.asc \
+ GPG_KEY=A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B \
+ CHECK_GPG=true
+
+# Prepare environment
+ENV FLINK_HOME=/opt/flink
+ENV PATH=$FLINK_HOME/bin:$PATH
+RUN groupadd --system --gid=9999 flink && \
+ useradd --system --home-dir $FLINK_HOME --uid=9999 --gid=flink flink
+WORKDIR $FLINK_HOME
+
+# Install Flink
+RUN set -ex; \
+ wget -nv -O flink.tgz "$FLINK_TGZ_URL"; \
+ \
+ if [ "$CHECK_GPG" = "true" ]; then \
+ wget -nv -O flink.tgz.asc "$FLINK_ASC_URL"; \
+ export GNUPGHOME="$(mktemp -d)"; \
+ for server in ha.pool.sks-keyservers.net $(shuf -e \
+ hkp://p80.pool.sks-keyservers.net:80 \
+ keyserver.ubuntu.com \
+ hkp://keyserver.ubuntu.com:80 \
+ pgp.mit.edu) ; do \
+ gpg --batch --keyserver "$server" --recv-keys "$GPG_KEY" && break || : ; \
+ done && \
+ gpg --batch --verify flink.tgz.asc flink.tgz; \
+ gpgconf --kill all; \
+ rm -rf "$GNUPGHOME" flink.tgz.asc; \
+ fi; \
+ \
+ tar -xf flink.tgz --strip-components=1; \
+ rm flink.tgz; \
+ \
+ chown -R flink:flink .; \
+ \
+ # Replace default REST/RPC endpoint bind address to use the container's network interface \
+ sed -i 's/rest.address: localhost/rest.address: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i 's/rest.bind-address: localhost/rest.bind-address: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i 's/jobmanager.bind-host: localhost/jobmanager.bind-host: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i 's/taskmanager.bind-host: localhost/taskmanager.bind-host: 0.0.0.0/g' $FLINK_HOME/conf/flink-conf.yaml; \
+ sed -i '/taskmanager.host: localhost/d' $FLINK_HOME/conf/flink-conf.yaml;
+
+# Configure container
+COPY docker-entrypoint.sh /
+ENTRYPOINT ["/docker-entrypoint.sh"]
+EXPOSE 6123 8081
+CMD ["help"]
diff --git a/1.17/scala_2.12-java8-ubuntu/docker-entrypoint.sh b/1.17/scala_2.12-java8-ubuntu/docker-entrypoint.sh
new file mode 100755
index 0000000..8b0350e
--- /dev/null
+++ b/1.17/scala_2.12-java8-ubuntu/docker-entrypoint.sh
@@ -0,0 +1,152 @@
+#!/usr/bin/env bash
+
+###############################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+###############################################################################
+
+COMMAND_STANDALONE="standalone-job"
+COMMAND_HISTORY_SERVER="history-server"
+
+# If unspecified, the hostname of the container is taken as the JobManager address
+JOB_MANAGER_RPC_ADDRESS=${JOB_MANAGER_RPC_ADDRESS:-$(hostname -f)}
+CONF_FILE="${FLINK_HOME}/conf/flink-conf.yaml"
+
+drop_privs_cmd() {
+ if [ $(id -u) != 0 ]; then
+ # Don't need to drop privs if EUID != 0
+ return
+ elif [ -x /sbin/su-exec ]; then
+ # Alpine
+ echo su-exec flink
+ else
+ # Others
+ echo gosu flink
+ fi
+}
+
+copy_plugins_if_required() {
+ if [ -z "$ENABLE_BUILT_IN_PLUGINS" ]; then
+ return 0
+ fi
+
+ echo "Enabling required built-in plugins"
+ for target_plugin in $(echo "$ENABLE_BUILT_IN_PLUGINS" | tr ';' ' '); do
+ echo "Linking ${target_plugin} to plugin directory"
+ plugin_name=${target_plugin%.jar}
+
+ mkdir -p "${FLINK_HOME}/plugins/${plugin_name}"
+ if [ ! -e "${FLINK_HOME}/opt/${target_plugin}" ]; then
+ echo "Plugin ${target_plugin} does not exist. Exiting."
+ exit 1
+ else
+ ln -fs "${FLINK_HOME}/opt/${target_plugin}" "${FLINK_HOME}/plugins/${plugin_name}"
+ echo "Successfully enabled ${target_plugin}"
+ fi
+ done
+}
+
+set_config_option() {
+ local option=$1
+ local value=$2
+
+ # escape periods for usage in regular expressions
+ local escaped_option=$(echo ${option} | sed -e "s/\./\\\./g")
+
+ # either override an existing entry, or append a new one
+ if grep -E "^${escaped_option}:.*" "${CONF_FILE}" > /dev/null; then
+ sed -i -e "s/${escaped_option}:.*/$option: $value/g" "${CONF_FILE}"
+ else
+ echo "${option}: ${value}" >> "${CONF_FILE}"
+ fi
+}
+
+prepare_configuration() {
+ set_config_option jobmanager.rpc.address ${JOB_MANAGER_RPC_ADDRESS}
+ set_config_option blob.server.port 6124
+ set_config_option query.server.port 6125
+
+ if [ -n "${TASK_MANAGER_NUMBER_OF_TASK_SLOTS}" ]; then
+ set_config_option taskmanager.numberOfTaskSlots ${TASK_MANAGER_NUMBER_OF_TASK_SLOTS}
+ fi
+
+ if [ -n "${FLINK_PROPERTIES}" ]; then
+ echo "${FLINK_PROPERTIES}" >> "${CONF_FILE}"
+ fi
+ envsubst < "${CONF_FILE}" > "${CONF_FILE}.tmp" && mv "${CONF_FILE}.tmp" "${CONF_FILE}"
+}
+
+maybe_enable_jemalloc() {
+ if [ "${DISABLE_JEMALLOC:-false}" == "false" ]; then
+ JEMALLOC_PATH="/usr/lib/$(uname -m)-linux-gnu/libjemalloc.so"
+ JEMALLOC_FALLBACK="/usr/lib/x86_64-linux-gnu/libjemalloc.so"
+ if [ -f "$JEMALLOC_PATH" ]; then
+ export LD_PRELOAD=$LD_PRELOAD:$JEMALLOC_PATH
+ elif [ -f "$JEMALLOC_FALLBACK" ]; then
+ export LD_PRELOAD=$LD_PRELOAD:$JEMALLOC_FALLBACK
+ else
+ if [ "$JEMALLOC_PATH" = "$JEMALLOC_FALLBACK" ]; then
+ MSG_PATH=$JEMALLOC_PATH
+ else
+ MSG_PATH="$JEMALLOC_PATH and $JEMALLOC_FALLBACK"
+ fi
+ echo "WARNING: attempted to load jemalloc from $MSG_PATH but the library couldn't be found. glibc will be used instead."
+ fi
+ fi
+}
+
+maybe_enable_jemalloc
+
+copy_plugins_if_required
+
+prepare_configuration
+
+args=("$@")
+if [ "$1" = "help" ]; then
+ printf "Usage: $(basename "$0") (jobmanager|${COMMAND_STANDALONE}|taskmanager|${COMMAND_HISTORY_SERVER})\n"
+ printf " Or $(basename "$0") help\n\n"
+ printf "By default, Flink image adopts jemalloc as default memory allocator. This behavior can be disabled by setting the 'DISABLE_JEMALLOC' environment variable to 'true'.\n"
+ exit 0
+elif [ "$1" = "jobmanager" ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting Job Manager"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/jobmanager.sh" start-foreground "${args[@]}"
+elif [ "$1" = ${COMMAND_STANDALONE} ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting Job Manager"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/standalone-job.sh" start-foreground "${args[@]}"
+elif [ "$1" = ${COMMAND_HISTORY_SERVER} ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting History Server"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/historyserver.sh" start-foreground "${args[@]}"
+elif [ "$1" = "taskmanager" ]; then
+ args=("${args[@]:1}")
+
+ echo "Starting Task Manager"
+
+ exec $(drop_privs_cmd) "$FLINK_HOME/bin/taskmanager.sh" start-foreground "${args[@]}"
+fi
+
+args=("${args[@]}")
+
+# Running command in pass-through mode
+exec $(drop_privs_cmd) "${args[@]}"
diff --git a/1.17/scala_2.12-java8-ubuntu/release.metadata b/1.17/scala_2.12-java8-ubuntu/release.metadata
new file mode 100644
index 0000000..6935d75
--- /dev/null
+++ b/1.17/scala_2.12-java8-ubuntu/release.metadata
@@ -0,0 +1,2 @@
+Tags: 1.17.0-scala_2.12-java8, 1.17-scala_2.12-java8, scala_2.12-java8, 1.17.0-java8, 1.17-java8, java8
+Architectures: amd64,arm64v8
[flink-docker] 02/03: [FLINK-31566] Updates .github/workflows/snapshot.yml to point to make all supported versions point to the right dev-* branches
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
mapohl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-docker.git
commit 07dab5cdef25aa96e81610e675d83f250dcdb892
Author: Matthias Pohl <ma...@aiven.io>
AuthorDate: Thu Mar 23 13:40:31 2023 +0100
[FLINK-31566] Updates .github/workflows/snapshot.yml to point to make all supported versions point to the right dev-* branches
1.15 is left in there for now because we might want to release 1.15.5
---
.github/workflows/snapshot.yml | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/snapshot.yml b/.github/workflows/snapshot.yml
index cdda9a0..b4c9511 100644
--- a/.github/workflows/snapshot.yml
+++ b/.github/workflows/snapshot.yml
@@ -38,8 +38,10 @@ jobs:
matrix:
java_version: [8, 11]
build:
- - flink_version: 1.17-SNAPSHOT
+ - flink_version: 1.18-SNAPSHOT
branch: dev-master
+ - flink_version: 1.17-SNAPSHOT
+ branch: dev-1.17
- flink_version: 1.16-SNAPSHOT
branch: dev-1.16
- flink_version: 1.15-SNAPSHOT