You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by mm...@apache.org on 2022/04/21 03:28:28 UTC
[pulsar] branch master updated: Migrate Docker images and CI to Java 17 (#14355)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new ccfea169269 Migrate Docker images and CI to Java 17 (#14355)
ccfea169269 is described below
commit ccfea16926982240ab36ff1339b361194744f712
Author: Nicolò Boschi <bo...@gmail.com>
AuthorDate: Thu Apr 21 05:28:23 2022 +0200
Migrate Docker images and CI to Java 17 (#14355)
* Migrate Docker images and CI to Java 17
* fix doc
### Motivation
The goal is to run Pulsar using JDK17 at runtime, benefitting from the latest JVM performance improvements and features.
In order to do that we need to run all the tests using JDK17.
The goal is **NOT** to deprecate JDK8 or JDK11 runtime compatibility. The java compiler release flag doesn't change.
### Modifications
* Build docker images using JDK17 instead of JDK11
* All the jobs (except for jdk 1.8 compatibility) are now using JDK17
### Verifying this change
All the CI jobs are passing
Manual tests are encouraged, mainly around `pulsar-io` connectors since we don't have integration tests for all the connectors.
### Documentation
- [x] `doc`
Added the support for JDK17 in the installation page.
---
.github/workflows/ci-build-macos.yaml | 4 ++--
.github/workflows/ci-cpp.yaml | 4 ++--
.github/workflows/ci-maven-cache-update.yaml | 4 ++--
.github/workflows/ci-owasp-dep-check.yaml | 4 ++--
.github/workflows/ci-owasp-dependency-check.yaml | 4 ++--
.github/workflows/ci-pulsar-website-build.yaml | 4 ++--
.github/workflows/pulsar-ci.yaml | 24 ++++++++++++------------
docker/pulsar/Dockerfile | 6 +++---
site2/docs/deploy-bare-metal.md | 2 +-
tests/docker-images/java-test-image/Dockerfile | 6 +++---
10 files changed, 31 insertions(+), 31 deletions(-)
diff --git a/.github/workflows/ci-build-macos.yaml b/.github/workflows/ci-build-macos.yaml
index 7fc95bcad05..0c73fddd005 100644
--- a/.github/workflows/ci-build-macos.yaml
+++ b/.github/workflows/ci-build-macos.yaml
@@ -69,12 +69,12 @@ jobs:
restore-keys: |
${{ runner.os }}-m2-dependencies-all-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: build package
diff --git a/.github/workflows/ci-cpp.yaml b/.github/workflows/ci-cpp.yaml
index 5e355872900..8647cf8ecc9 100644
--- a/.github/workflows/ci-cpp.yaml
+++ b/.github/workflows/ci-cpp.yaml
@@ -69,12 +69,12 @@ jobs:
restore-keys: |
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
diff --git a/.github/workflows/ci-maven-cache-update.yaml b/.github/workflows/ci-maven-cache-update.yaml
index 755f2afdf1b..d78f77707fd 100644
--- a/.github/workflows/ci-maven-cache-update.yaml
+++ b/.github/workflows/ci-maven-cache-update.yaml
@@ -98,12 +98,12 @@ jobs:
# on growing from old entries which wouldn't never expire if the old
# cache would be used as the starting point for a new cache entry
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ (github.event_name == 'schedule' || steps.changes.outputs.poms == 'true') && steps.cache.outputs.cache-hit != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: Download dependencies
if: ${{ (github.event_name == 'schedule' || steps.changes.outputs.poms == 'true') && steps.cache.outputs.cache-hit != 'true' }}
diff --git a/.github/workflows/ci-owasp-dep-check.yaml b/.github/workflows/ci-owasp-dep-check.yaml
index 3fce68fbdec..0f48bfbacf1 100644
--- a/.github/workflows/ci-owasp-dep-check.yaml
+++ b/.github/workflows/ci-owasp-dep-check.yaml
@@ -71,12 +71,12 @@ jobs:
${{ runner.os }}-m2-dependencies-core-modules-${{ hashFiles('**/pom.xml') }}
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ steps.changes.outputs.poms == 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: clean disk
if: ${{ steps.changes.outputs.poms == 'true' }}
diff --git a/.github/workflows/ci-owasp-dependency-check.yaml b/.github/workflows/ci-owasp-dependency-check.yaml
index 0a9232790ea..3b787bb0c20 100644
--- a/.github/workflows/ci-owasp-dependency-check.yaml
+++ b/.github/workflows/ci-owasp-dependency-check.yaml
@@ -63,11 +63,11 @@ jobs:
${{ runner.os }}-m2-dependencies-core-modules-${{ hashFiles('**/pom.xml') }}
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: run install by skip tests
run: mvn -q -B -ntp clean install -DskipTests
diff --git a/.github/workflows/ci-pulsar-website-build.yaml b/.github/workflows/ci-pulsar-website-build.yaml
index 9d47f90e834..be412d3aa73 100644
--- a/.github/workflows/ci-pulsar-website-build.yaml
+++ b/.github/workflows/ci-pulsar-website-build.yaml
@@ -50,11 +50,11 @@ jobs:
${{ runner.os }}-m2-dependencies-core-modules-${{ hashFiles('**/pom.xml') }}
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: clean disk
run: |
diff --git a/.github/workflows/pulsar-ci.yaml b/.github/workflows/pulsar-ci.yaml
index a3ec2a6c1c8..2231c8c3404 100644
--- a/.github/workflows/pulsar-ci.yaml
+++ b/.github/workflows/pulsar-ci.yaml
@@ -92,12 +92,12 @@ jobs:
restore-keys: |
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: Check source code license headers
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
@@ -190,12 +190,12 @@ jobs:
restore-keys: |
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK ${{ matrix.jdk || '11' }}
+ - name: Set up JDK ${{ matrix.jdk || '17' }}
uses: actions/setup-java@v2
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: ${{ matrix.jdk || '11' }}
+ java-version: ${{ matrix.jdk || '17' }}
- name: Install gh-actions-artifact-client.js
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
@@ -292,12 +292,12 @@ jobs:
restore-keys: |
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: Install gh-actions-artifact-client.js
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
@@ -386,12 +386,12 @@ jobs:
restore-keys: |
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: Install gh-actions-artifact-client.js
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
@@ -530,12 +530,12 @@ jobs:
${{ runner.os }}-m2-dependencies-core-modules-${{ hashFiles('**/pom.xml') }}
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: Install gh-actions-artifact-client.js
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
@@ -636,12 +636,12 @@ jobs:
${{ runner.os }}-m2-dependencies-core-modules-${{ hashFiles('**/pom.xml') }}
${{ runner.os }}-m2-dependencies-core-modules-
- - name: Set up JDK 11
+ - name: Set up JDK 17
uses: actions/setup-java@v2
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
with:
distribution: 'temurin'
- java-version: 11
+ java-version: 17
- name: Install gh-actions-artifact-client.js
if: ${{ needs.changed_files_job.outputs.docs_only != 'true' }}
diff --git a/docker/pulsar/Dockerfile b/docker/pulsar/Dockerfile
index 711b9509d3b..1c053a751aa 100644
--- a/docker/pulsar/Dockerfile
+++ b/docker/pulsar/Dockerfile
@@ -54,7 +54,7 @@ RUN sed -i "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-mirror://mirr
&& echo 'Acquire::http::Timeout "30";\nAcquire::ftp::Timeout "30";\nAcquire::Retries "3";' > /etc/apt/apt.conf.d/99timeout_and_retries \
&& apt-get update \
&& apt-get -y dist-upgrade \
- && apt-get -y install --no-install-recommends openjdk-11-jdk-headless netcat dnsutils less procps iputils-ping \
+ && apt-get -y install --no-install-recommends openjdk-17-jdk-headless netcat dnsutils less procps iputils-ping \
python3 python3-yaml python3-kazoo python3-pip \
curl ca-certificates \
&& apt-get -y --purge autoremove \
@@ -72,8 +72,8 @@ RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10
# 4. /pulsar - hadoop writes to this directory
RUN mkdir /pulsar && chmod g+w /pulsar
-ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
-RUN echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/java-11-openjdk-amd64/conf/security/java.security
+ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-amd64
+RUN echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/java-17-openjdk-amd64/conf/security/java.security
ADD target/python-client/ /pulsar/pulsar-client
ENV PULSAR_ROOT_LOGGER=INFO,CONSOLE
diff --git a/site2/docs/deploy-bare-metal.md b/site2/docs/deploy-bare-metal.md
index ff30a0d5237..6d12f2dfae5 100644
--- a/site2/docs/deploy-bare-metal.md
+++ b/site2/docs/deploy-bare-metal.md
@@ -40,7 +40,7 @@ To run Pulsar on bare metal, the following configuration is recommended:
> * Broker is only supported on 64-bit JVM.
> * If you do not have enough machines, or you want to test Pulsar in cluster mode (and expand the cluster later), You can fully deploy Pulsar on a node on which ZooKeeper, bookie and broker run.
> * If you do not have a DNS server, you can use the multi-host format in the service URL instead.
-Each machine in your cluster needs to have [Java 8](https://adoptium.net/?variant=openjdk8) or [Java 11](https://adoptium.net/?variant=openjdk11) installed.
+Each machine in your cluster needs to have [Java 8](https://adoptium.net/?variant=openjdk8), [Java 11](https://adoptium.net/?variant=openjdk11) or [Java 17](https://adoptium.net/?variant=openjdk17) installed.
The following is a diagram showing the basic setup:
diff --git a/tests/docker-images/java-test-image/Dockerfile b/tests/docker-images/java-test-image/Dockerfile
index 564182534ac..b7f12b75968 100644
--- a/tests/docker-images/java-test-image/Dockerfile
+++ b/tests/docker-images/java-test-image/Dockerfile
@@ -38,10 +38,10 @@ ARG UBUNTU_MIRROR=mirror://mirrors.ubuntu.com/mirrors.txt
RUN sed -i "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-mirror://mirrors.ubuntu.com/mirrors.txt}|g" /etc/apt/sources.list \
&& apt-get update \
&& apt-get -y dist-upgrade \
- && apt-get -y install openjdk-11-jdk-headless
+ && apt-get -y install openjdk-17-jdk-headless
-ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
-RUN echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/java-11-openjdk-amd64/conf/security/java.security
+ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-amd64
+RUN echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/java-17-openjdk-amd64/conf/security/java.security
# /pulsar/bin/watch-znode.py requires python3-kazoo
# /pulsar/bin/pulsar-managed-ledger-admin requires python3-protobuf