You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by eo...@apache.org on 2024/02/08 09:55:41 UTC
(bookkeeper) branch master updated: Add support for Junit5 tests and fix CI that has been broken (#4197)
This is an automated email from the ASF dual-hosted git repository.
eolivelli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git
The following commit(s) were added to refs/heads/master by this push:
new 7bd91a7edc Add support for Junit5 tests and fix CI that has been broken (#4197)
7bd91a7edc is described below
commit 7bd91a7edc2b2aa1ebf8527db732a79872077b69
Author: Lari Hotari <lh...@users.noreply.github.com>
AuthorDate: Thu Feb 8 01:55:35 2024 -0800
Add support for Junit5 tests and fix CI that has been broken (#4197)
* Add support for Junit5 tests
- upgrade surefire plugin
- replace <forkMode>always</forkMode> with supported way in newer maven-surefire-plugin
- maven-failsafe-plugin version must match maven-surefire-plugin version
- remove invalid test case. Log4j is in the classpath for other reasons
checked by running
"mvn -pl tests/integration-tests-utils/pom.xml test -X"
also verified by debugging that it's not in the system classpath, it is loaded by the
MavenClassLoader
- add assertj-core for fluent assertions in tests
* Upgrade arquillian-junit-standalone
* Fix error in pom.xml
* Fix java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException
* Pick docker-java-api version that is compatible with arquillian-cube-docker
* Fix snakeyaml issue
* Fix running of bookkeeper in Docker in tests
* Improve GitHub Actions CI workflow logging
* Increase client tests timeout
* Add script for picking fast ubuntu mirror for docker build
* upgrade maven-dependency-plugin version (used for copying .tar.gz for docker image building)
* Fix building of the docker image
* Increase timeouts for integration tests
* Use eclipse-temurin base image
* Replace "FROM ubuntu:22.10" with "FROM ubuntu:22.04" to use LTS version of ubuntu
22.10 (kinetic) is EOL since July 20 2023
* Build docker image also for backward-compatibility-tests
* Install gpg that is required by the scripts
* Add UBUNTU_MIRROR
* Remove redundancy from integration tests
* Fix missing dependency
* Fix classpath issues for arquillian-cube
* Upgrade actions versions
* Fix invalid references in native-io/pom.xml
* Fix circe-checksum native library compilation
* Migrate Gradle specific settings to Maven
* Another attempt to disable backward-compat tests in integration test build job
* Add working solution for skipping backward compat tests in the int tests
* Attempt to fix issue in locating the file
* Fix tar file extraction
* Improve solution
* Backward compat tests build commands without int tests
* Copy tune-runner-vm and clean-disk from Pulsar CI and clean disk
* Revisit skipping of backward compat tests once again
* Use similar MAVEN_OPTS as there is in Pulsar
* Fix directory
* Add way to find git root directory
* Revisit one more time
* Add current-version-image back
* Enable tests for backwardCompatTests
* Don't create /ledgers in ZK if it already exists
* Add uncaughtexceptionhandler
* Add exclusions for reload4j
* Use maven-surefire 2.8.1 for certain tests
* Add failing test back in, also fails with surefire 2.8.1
* Exclude logging implementations
* Revert "Use maven-surefire 2.8.1 for certain tests"
This reverts commit 94e2b8eb959c8d7559edd13d7ebe2c0637bf8ab9.
* Exclude log4j
* Add instructions how to run on Mac Apple Silicon
* Find slf4j version
* Block loading some classes
* Change context classloader while creating new bookkeeper
* Route commons logging to slf4j
* Update shrinkwrap.version
* Wait for ZK client to shutdown before returning from close
* Upgrade Groovy version
* Remove arquillian-junit-container which causes tests to be run twice
* Skip backward compat tests unless -DbackwardCompatTests is passed
* Delay closing of classloaders by 5 seconds
* Fix checkstyle
* Improve instructions for running tests
* Upgrade Testcontainers and import the bom in dependencyManagement
* Fix synchronization bug pointed out by spotbugs
* upgrade markdown-link-check
* Ignore checking Google calendar for dead links
* Ignore anchors
* Add 2 more patterns to dead link checker exclusions
* Collect logs
* Log integration tests to console to observe progress, disable retries
* Use maven version provided by GitHub Action runner
* Improve instructions for docker socket proxy
* Add test timeout of 5 minutes for BookieShellTestBase
* Fix test to match changes made in e41d6728
* Upgrade mockito to 4.11.0
* Align hamcrest with junit
* Revert "Upgrade mockito to 4.11.0"
This reverts commit 55b7fcd4ca3d83534f2ce328bc35d3a666a4b622.
* Make JAVA_HOME optional
* Tune Java's DNS cache timeouts for docker images
* Use different directory for JDK 8
* Add tooling to containers that help debugging
* Upgrade JMH version and configuration
* Specify maven.compiler.release to fix issues with building with Java 11
* Remove gmavenplus plugin since there's already groovy-eclipse compiler in use
* Upgrade apache-rat-plugin
* Fix validation step
* Don't import shaded class
* Move freebuilder out of distributed libraries (there was invalid config for the annotation processor)
* Remove use of shaded class in test
* Upgrade junit5, assertj-core & awaitility to latest stable
* Fix simpletest logic
- LAC is updated asynchronously. Entries might be unconfirmed when read before ledger is closed.
- The last entry id passed to the read method was invalid
* Fix SimpleTestCommandTest
---------
Co-authored-by: Matteo Merli <mm...@apache.org>
---
.dlc.json | 12 ++
.github/actions/clean-disk/action.yml | 57 +++++
.github/actions/tune-runner-vm/action.yml | 73 ++++++-
.github/workflows/bk-ci.yml | 192 ++++++++++-------
.github/workflows/bk-streamstorage-python.yml | 18 +-
.github/workflows/bot.yml | 4 +-
.github/workflows/dead-link-checker.yaml | 6 +-
.github/workflows/website-deploy.yaml | 10 +-
.github/workflows/website-pr-validation.yml | 10 +-
bin/bookkeeper-daemon.sh | 12 +-
bin/common.sh | 13 +-
bookkeeper-common/pom.xml | 3 +-
.../src/main/resources/LICENSE-all.bin.txt | 2 -
.../src/main/resources/LICENSE-bkctl.bin.txt | 2 -
.../src/main/resources/LICENSE-server.bin.txt | 2 -
bookkeeper-server/pom.xml | 4 +-
.../bookkeeper/meta/zk/ZKMetadataDriverBase.java | 3 +-
.../cli/commands/client/SimpleTestCommand.java | 71 +++++--
.../meta/zk/ZKMetadataDriverBaseTest.java | 2 +-
circe-checksum/pom.xml | 93 +++-----
cpu-affinity/pom.xml | 21 --
dev/ci-tool | 39 ++++
docker/Dockerfile | 16 +-
docker/README.md | 1 -
.../metadata/etcd/helpers/KeySetReaderTest.java | 2 +-
microbenchmarks/pom.xml | 32 ++-
native-io/pom.xml | 25 +--
pom.xml | 236 ++++++++++++++++-----
shaded/bookkeeper-server-shaded/pom.xml | 8 +
shaded/bookkeeper-server-tests-shaded/pom.xml | 10 +-
shaded/distributedlog-core-shaded/pom.xml | 8 +
stats/bookkeeper-stats-api/pom.xml | 3 +-
stream/api/pom.xml | 4 -
stream/clients/java/base/pom.xml | 18 ++
.../TestStorageServerClientManagerImpl.java | 2 +-
stream/clients/python/docker/Dockerfile | 16 +-
stream/clients/python/docker/build-local-image.sh | 3 +-
stream/distributedlog/common/pom.xml | 18 +-
stream/distributedlog/core/pom.xml | 6 +-
stream/distributedlog/io/dlfs/pom.xml | 8 +
stream/distributedlog/pom.xml | 5 +-
stream/pom.xml | 4 +-
stream/proto/pom.xml | 9 +-
stream/tests-common/pom.xml | 9 +-
tests/README.md | 81 +++++++
tests/backward-compat/pom.xml | 55 +++++
.../backwardcompat/TestCompatUpgradeDirect.groovy | 6 +
.../all-released-versions-image/Dockerfile | 17 +-
.../all-released-versions-image/image_builder.sh | 3 +-
.../all-released-versions-image/pom.xml | 4 +
.../all-versions-image/image_builder.sh | 3 +-
tests/docker-images/all-versions-image/pom.xml | 2 +
.../Dockerfile | 54 +++--
.../pom.xml | 64 ++++--
.../scripts/install-python-client.sh | 17 +-
tests/docker-images/pom.xml | 1 +
tests/docker-images/statestore-image/Dockerfile | 18 +-
.../statestore-image/image_builder.sh | 3 +-
tests/integration-tests-base-groovy/pom.xml | 42 ++--
tests/integration-tests-utils/pom.xml | 12 +-
.../integration/utils/BookKeeperClusterUtils.java | 13 +-
.../tests/integration/utils/DockerUtils.java | 4 +-
.../tests/integration/utils/MavenClassLoader.java | 162 ++++++++++----
tests/integration/cluster/pom.xml | 1 -
tests/integration/smoke/pom.xml | 28 ++-
.../tests/integration/BookieShellTestBase.java | 4 +
tests/integration/standalone/pom.xml | 1 -
tests/pom.xml | 5 +-
tests/shaded/bookkeeper-server-shaded-test/pom.xml | 2 +-
.../cli/commands/client/SimpleTestCommandTest.java | 30 +--
70 files changed, 1209 insertions(+), 515 deletions(-)
diff --git a/.dlc.json b/.dlc.json
index 1f86124c5a..49a1467aba 100644
--- a/.dlc.json
+++ b/.dlc.json
@@ -62,6 +62,18 @@
},
{
"pattern": "^http://daxue.qq.com/content/content/id/2492"
+ },
+ {
+ "pattern": "^https://calendar.google.com/"
+ },
+ {
+ "pattern": "^#"
+ },
+ {
+ "pattern": ".*\\{\\{.*"
+ },
+ {
+ "pattern": "^//"
}
],
"timeout": "10s",
diff --git a/.github/actions/clean-disk/action.yml b/.github/actions/clean-disk/action.yml
new file mode 100644
index 0000000000..d74c3f25fc
--- /dev/null
+++ b/.github/actions/clean-disk/action.yml
@@ -0,0 +1,57 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: clean disk
+description: makes some more space available on the disk by removing files
+inputs:
+ mode:
+ description: "Use 'full' to clean as much as possible"
+ required: false
+runs:
+ using: composite
+ steps:
+ - run: |
+ if [[ "$OSTYPE" == "linux-gnu"* ]]; then
+ directories=(/usr/local/lib/android /opt/ghc)
+ if [[ "${{ inputs.mode }}" == "full" ]]; then
+ # remove these directories only when mode is 'full'
+ directories+=(/usr/share/dotnet /opt/hostedtoolcache/CodeQL)
+ fi
+ emptydir=/tmp/empty$$/
+ mkdir $emptydir
+ echo "::group::Available diskspace"
+ time df -BM / /mnt
+ echo "::endgroup::"
+ for directory in "${directories[@]}"; do
+ echo "::group::Removing $directory"
+ # fast way to delete a lot of files on linux
+ time sudo eatmydata rsync -a --delete $emptydir ${directory}/
+ time sudo eatmydata rm -rf ${directory}
+ time df -BM / /mnt
+ echo "::endgroup::"
+ done
+ echo "::group::Cleaning apt state"
+ time sudo bash -c "apt-get clean; apt-get autoclean; apt-get -y --purge autoremove"
+ time df -BM / /mnt
+ echo "::endgroup::"
+ fi
+ echo "::group::Available diskspace"
+ time df -BM / /mnt
+ echo "::endgroup::"
+ shell: bash
diff --git a/.github/actions/tune-runner-vm/action.yml b/.github/actions/tune-runner-vm/action.yml
index 59f977c315..402b9201dc 100644
--- a/.github/actions/tune-runner-vm/action.yml
+++ b/.github/actions/tune-runner-vm/action.yml
@@ -24,8 +24,79 @@ runs:
steps:
- run: |
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
+ echo "::group::Configure and tune OS"
# Ensure that reverse lookups for current hostname are handled properly
# Add the current IP address, long hostname and short hostname record to /etc/hosts file
echo -e "$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)\t$(hostname -f) $(hostname -s)" | sudo tee -a /etc/hosts
+
+ # The default vm.swappiness setting is 60 which has a tendency to start swapping when memory
+ # consumption is high.
+ # Set vm.swappiness=1 to avoid swapping and allow high RAM usage
+ echo 1 | sudo tee /proc/sys/vm/swappiness
+ (
+ shopt -s nullglob
+ # Set swappiness to 1 for all cgroups and sub-groups
+ for swappiness_file in /sys/fs/cgroup/memory/*/memory.swappiness /sys/fs/cgroup/memory/*/*/memory.swappiness; do
+ echo 1 | sudo tee $swappiness_file > /dev/null
+ done
+ ) || true
+
+ # use "madvise" Linux Transparent HugePages (THP) setting
+ # https://www.kernel.org/doc/html/latest/admin-guide/mm/transhuge.html
+ # "madvise" is generally a better option than the default "always" setting
+ # Based on Azul instructions from https://docs.azul.com/prime/Enable-Huge-Pages#transparent-huge-pages-thp
+ echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
+ echo advise | sudo tee /sys/kernel/mm/transparent_hugepage/shmem_enabled
+ echo defer+madvise | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
+ echo 1 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/defrag
+
+ # tune filesystem mount options, https://www.kernel.org/doc/Documentation/filesystems/ext4.txt
+ # commit=999999, effectively disables automatic syncing to disk (default is every 5 seconds)
+ # nobarrier/barrier=0, loosen data consistency on system crash (no negative impact to empheral CI nodes)
+ sudo mount -o remount,nodiscard,commit=999999,barrier=0 /
+ sudo mount -o remount,nodiscard,commit=999999,barrier=0 /mnt
+ # disable discard/trim at device level since remount with nodiscard doesn't seem to be effective
+ # https://www.spinics.net/lists/linux-ide/msg52562.html
+ for i in /sys/block/sd*/queue/discard_max_bytes; do
+ echo 0 | sudo tee $i
+ done
+ # disable any background jobs that run SSD discard/trim
+ sudo systemctl disable fstrim.timer || true
+ sudo systemctl stop fstrim.timer || true
+ sudo systemctl disable fstrim.service || true
+ sudo systemctl stop fstrim.service || true
+
+ # stop php-fpm
+ sudo systemctl stop php8.0-fpm.service || true
+ sudo systemctl stop php7.4-fpm.service || true
+ # stop mono-xsp4
+ sudo systemctl disable mono-xsp4.service || true
+ sudo systemctl stop mono-xsp4.service || true
+ sudo killall mono || true
+
+ # stop Azure Linux agent to save RAM
+ sudo systemctl stop walinuxagent.service || true
+
+ # enable docker experimental mode which is
+ # required for using "docker build --squash" / "-Ddocker.squash=true"
+ daemon_json="$(sudo cat /etc/docker/daemon.json | jq '.experimental = true')"
+ echo "$daemon_json" | sudo tee /etc/docker/daemon.json
+ # restart docker daemon
+ sudo systemctl restart docker
+ echo '::endgroup::'
+
+ # show memory
+ echo "::group::Available Memory"
+ free -m
+ echo '::endgroup::'
+ # show disk
+ echo "::group::Available diskspace"
+ df -BM
+ echo "::endgroup::"
+ # show cggroup
+ echo "::group::Cgroup settings for current cgroup $CURRENT_CGGROUP"
+ CURRENT_CGGROUP=$(cat /proc/self/cgroup | grep '0::' | awk -F: '{ print $3 }')
+ sudo cgget -a $CURRENT_CGGROUP || true
+ echo '::endgroup::'
fi
- shell: bash
\ No newline at end of file
+ shell: bash
diff --git a/.github/workflows/bk-ci.yml b/.github/workflows/bk-ci.yml
index c9f9226dd8..035c2c5e27 100644
--- a/.github/workflows/bk-ci.yml
+++ b/.github/workflows/bk-ci.yml
@@ -31,7 +31,7 @@ on:
workflow_dispatch:
env:
- MAVEN_OPTS: -Dmaven.wagon.httpconnectionManager.ttlSeconds=25 -Dmaven.wagon.http.retryHandler.count=3
+ MAVEN_OPTS: -Xss1500k -Xmx1500m -Daether.connector.http.reuseConnections=false -Daether.connector.requestTimeout=60000 -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.http.retryHandler.class=standard -Dmaven.wagon.http.retryHandler.count=3 -Dmaven.wagon.http.retryHandler.requestSentEnabled=true -Dmaven.wagon.http.serviceUnavailableRetryStrategy.class=standard -Dmaven.wagon.rto=60000
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -44,7 +44,7 @@ jobs:
timeout-minutes: 60
steps:
- name: Checkout
- uses: actions/checkout@v2
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
@@ -64,7 +64,7 @@ jobs:
- name: Cache local Maven repository
if: steps.check_changes.outputs.docs_only != 'true'
id: cache
- uses: actions/cache@v3
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -74,16 +74,11 @@ jobs:
- name: Set up JDK 11
if: steps.check_changes.outputs.docs_only != 'true'
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
-
- name: Validate pull request
if: steps.check_changes.outputs.docs_only != 'true'
run: |
@@ -116,6 +111,7 @@ jobs:
module: bookkeeper-server
flag: client
test_args: "-Dtest='org.apache.bookkeeper.client.**'"
+ timeout: 75
- step_name: Replication Tests
module: bookkeeper-server
flag: replication
@@ -137,14 +133,14 @@ jobs:
steps:
- name: checkout
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v3
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -152,18 +148,12 @@ jobs:
!~/.m2/repository/org/apache/distributedlog
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
-
- name: Set up JDK 11
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
-
- name: Build
run: |
projects_list=
@@ -192,11 +182,11 @@ jobs:
annotate_only: 'true'
- name: Upload Surefire reports
- uses: actions/upload-artifact@v3
+ uses: actions/upload-artifact@v4
if: failure()
continue-on-error: true
with:
- name: bookie-tests-reports
+ name: unit-${{ matrix.step_name }}-reports
path: surefire-reports
retention-days: 7
@@ -207,19 +197,24 @@ jobs:
integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
- timeout-minutes: 60
+ timeout-minutes: 75
needs: [ 'build-and-license-check' ]
if: ${{ needs.build-and-license-check.outputs.docs_only != 'true' }}
steps:
- name: Checkout
- uses: actions/checkout@v2
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
+ - name: Clean Disk
+ uses: ./.github/actions/clean-disk
+ with:
+ mode: full
+
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v2
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -228,15 +223,15 @@ jobs:
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
- name: Set up JDK 11
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
+ - name: Pick ubuntu mirror for the docker image build
+ run: |
+ # pick the closest ubuntu mirror and set it to UBUNTU_MIRROR environment variable
+ $GITHUB_WORKSPACE/dev/ci-tool pick_ubuntu_mirror
- name: Build with Maven
run: mvn -B -nsu clean install -Pdocker -DskipTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
@@ -244,29 +239,67 @@ jobs:
- name: Run metadata driver tests
run: mvn -B -nsu -f metadata-drivers/pom.xml test -DintegrationTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
- - name: Run all integration tests
- run: mvn -B -nsu -f tests/pom.xml test -DintegrationTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
+ - name: Run all integration tests (except backward compatibility tests)
+ run: |
+ mvn -B -nsu -f tests/pom.xml test -DintegrationTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO -DredirectTestOutputToFile=false -DtestRetryCount=0
- name: print JVM thread dumps when cancelled
if: cancelled()
run: ./dev/ci-tool print_thread_dumps
+ - name: Upload container logs on failure
+ uses: actions/upload-artifact@v4
+ if: ${{ !success() }}
+ continue-on-error: true
+ with:
+ retention-days: 7
+ name: integration-tests-container-logs
+ if-no-files-found: ignore
+ path: |
+ **/docker.log
+
+ - name: Aggregates all test reports to ./test-reports and ./surefire-reports directories
+ if: ${{ always() }}
+ uses: ./.github/actions/copy-test-reports
+
+ - name: Publish Test Report
+ uses: apache/pulsar-test-infra/action-junit-report@master
+ if: ${{ always() }}
+ with:
+ report_paths: 'surefire-reports/TEST-*.xml'
+ annotate_only: 'true'
+
+ - name: Upload Surefire reports
+ uses: actions/upload-artifact@v4
+ if: failure()
+ continue-on-error: true
+ with:
+ name: integration-tests-reports
+ path: surefire-reports
+ if-no-files-found: ignore
+ retention-days: 7
+
backward-compatibility-tests:
name: Backward compatibility tests
runs-on: ubuntu-latest
- timeout-minutes: 60
+ timeout-minutes: 75
needs: [ 'build-and-license-check' ]
if: ${{ needs.build-and-license-check.outputs.docs_only != 'true' }}
steps:
- name: Checkout
- uses: actions/checkout@v2
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
+ - name: Clean Disk
+ uses: ./.github/actions/clean-disk
+ with:
+ mode: full
+
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v2
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -275,28 +308,60 @@ jobs:
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
- name: Set up JDK 8
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 8
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
+ - name: Pick ubuntu mirror for the docker image build
+ run: |
+ # pick the closest ubuntu mirror and set it to UBUNTU_MIRROR environment variable
+ $GITHUB_WORKSPACE/dev/ci-tool pick_ubuntu_mirror
- name: Build with Maven
- run: mvn -B -nsu clean install -DskipTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
+ run: mvn -B -nsu clean install -Pdocker -DskipTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
- name: Test current server with old clients
- run: mvn -B -nsu -DintegrationTests -pl :backward-compat-current-server-old-clients test
+ run: mvn -B -nsu -DbackwardCompatTests -pl :backward-compat-current-server-old-clients test
- name: Test progressive upgrade
- run: mvn -B -nsu -DintegrationTests -pl :upgrade test
+ run: mvn -B -nsu -DbackwardCompatTests -pl :upgrade test
- name: Other tests
run: |
- mvn -B -nsu -DintegrationTests -pl :bc-non-fips,:hierarchical-ledger-manager,:hostname-bookieid,:old-cookie-new-cluster,:recovery-no-password,:upgrade-direct,:yahoo-custom-version test
+ mvn -B -nsu -DbackwardCompatTests -pl :bc-non-fips,:hierarchical-ledger-manager,:hostname-bookieid,:old-cookie-new-cluster,:recovery-no-password,:upgrade-direct,:yahoo-custom-version test
+
+ - name: Upload container logs on failure
+ uses: actions/upload-artifact@v4
+ if: ${{ !success() }}
+ continue-on-error: true
+ with:
+ retention-days: 7
+ name: backward-compatibility-tests-container-logs
+ if-no-files-found: ignore
+ path: |
+ **/docker.log
+
+ - name: Aggregates all test reports to ./test-reports and ./surefire-reports directories
+ if: ${{ always() }}
+ uses: ./.github/actions/copy-test-reports
+
+ - name: Publish Test Report
+ uses: apache/pulsar-test-infra/action-junit-report@master
+ if: ${{ always() }}
+ with:
+ report_paths: 'surefire-reports/TEST-*.xml'
+ annotate_only: 'true'
+
+ - name: Upload Surefire reports
+ uses: actions/upload-artifact@v4
+ if: failure()
+ continue-on-error: true
+ with:
+ name: backward-compatibility-tests-reports
+ path: surefire-reports
+ if-no-files-found: ignore
+ retention-days: 7
windows-build:
name: Build with windows on JDK 11
@@ -306,7 +371,7 @@ jobs:
if: ${{ needs.build-and-license-check.outputs.docs_only != 'true' }}
steps:
- name: Checkout
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
@@ -316,7 +381,7 @@ jobs:
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v2
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -324,18 +389,12 @@ jobs:
!~/.m2/repository/org/apache/distributedlog
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
-
- name: Set up JDK 11
- uses: actions/setup-java@v3
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
-
- name: mvn package
run: mvn -B -nsu clean package -DskipTests
@@ -347,14 +406,14 @@ jobs:
if: ${{ needs.build-and-license-check.outputs.docs_only != 'true' }}
steps:
- name: Checkout
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v2
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -363,16 +422,11 @@ jobs:
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
- name: Set up JDK 11
- uses: actions/setup-java@v3
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
-
- name: mvn package
run: mvn -B -nsu clean package -DskipTests
@@ -395,14 +449,14 @@ jobs:
steps:
- name: checkout
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v3
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -410,18 +464,12 @@ jobs:
!~/.m2/repository/org/apache/distributedlog
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
-
- name: Set up JDK ${{ matrix.jdk_version }}
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: ${{ matrix.jdk_version }}
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
-
- name: Build with Maven
run: mvn clean package -B -nsu -DskipBookKeeperServerTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO
@@ -437,14 +485,14 @@ jobs:
if: ${{ needs.build-and-license-check.outputs.need_owasp == 'true' }}
steps:
- name: Checkout
- uses: actions/checkout@v2
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v2
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -453,7 +501,7 @@ jobs:
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
- name: Set up JDK 11
- uses: actions/setup-java@v3
+ uses: actions/setup-java@v4
with:
java-version: 11
@@ -468,7 +516,7 @@ jobs:
run: mvn -q -B -ntp clean install verify -Powasp-dependency-check -DskipTests -pl '!stream/distributedlog/io/dlfs'
- name: Upload report
- uses: actions/upload-artifact@v3
+ uses: actions/upload-artifact@v4
if: ${{ cancelled() || failure() }}
continue-on-error: true
with:
diff --git a/.github/workflows/bk-streamstorage-python.yml b/.github/workflows/bk-streamstorage-python.yml
index 3cf7d93583..71dbe61045 100644
--- a/.github/workflows/bk-streamstorage-python.yml
+++ b/.github/workflows/bk-streamstorage-python.yml
@@ -41,12 +41,12 @@ jobs:
timeout-minutes: 60
steps:
- name: checkout
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Test
run: ./stream/clients/python/scripts/test.sh
-
+
Stream-storage-python-client-integration-tests:
name: StreamStorage Python Client Integration Tests
@@ -54,12 +54,12 @@ jobs:
timeout-minutes: 60
steps:
- name: checkout
- uses: actions/checkout@v3
+ uses: actions/checkout@v4
- name: Tune Runner VM
uses: ./.github/actions/tune-runner-vm
- name: Cache local Maven repository
id: cache
- uses: actions/cache@v3
+ uses: actions/cache@v4
with:
path: |
~/.m2/repository/*/*/*
@@ -67,16 +67,16 @@ jobs:
!~/.m2/repository/org/apache/distributedlog
key: ${{ runner.os }}-bookkeeper-all-${{ hashFiles('**/pom.xml') }}
- name: Set up JDK 11
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
- name: Build
run: mvn -q -T 1C -B -nsu clean install -DskipTests -Dcheckstyle.skip -Dspotbugs.skip -Drat.skip -Dmaven.javadoc.skip
+ - name: Pick ubuntu mirror for the docker image build
+ run: |
+ # pick the closest ubuntu mirror and set it to UBUNTU_MIRROR environment variable
+ $GITHUB_WORKSPACE/dev/ci-tool pick_ubuntu_mirror
- name: Build Test image
run: ./stream/clients/python/docker/build-local-image.sh
- name: Test
diff --git a/.github/workflows/bot.yml b/.github/workflows/bot.yml
index b29d767c70..43d61c1179 100644
--- a/.github/workflows/bot.yml
+++ b/.github/workflows/bot.yml
@@ -29,10 +29,10 @@ jobs:
steps:
- name: clone repository
- uses: actions/checkout@v2
+ uses: actions/checkout@v4
- name: bot actions
- uses: actions/github-script@v1
+ uses: actions/github-script@v7
env:
PROVIDER: 'apache'
REPOSITORY: 'bookkeeper'
diff --git a/.github/workflows/dead-link-checker.yaml b/.github/workflows/dead-link-checker.yaml
index 8b7c564df4..8dae314d40 100644
--- a/.github/workflows/dead-link-checker.yaml
+++ b/.github/workflows/dead-link-checker.yaml
@@ -21,7 +21,7 @@ on:
pull_request:
branches:
- master
- - branch-*
+ - branch-*
paths:
- '**.md'
@@ -35,8 +35,8 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- - uses: actions/checkout@v2
- - run: sudo npm install -g markdown-link-check@3.8.7
+ - uses: actions/checkout@v4
+ - run: sudo npm install -g markdown-link-check@3.11.2
- run: |
for file in $(find . -name "*.md"); do
markdown-link-check -c .dlc.json -q "$file"
diff --git a/.github/workflows/website-deploy.yaml b/.github/workflows/website-deploy.yaml
index b35eb4b7ef..868b1db0c7 100644
--- a/.github/workflows/website-deploy.yaml
+++ b/.github/workflows/website-deploy.yaml
@@ -38,21 +38,17 @@ jobs:
timeout-minutes: 180
steps:
- name: Checkout
- uses: actions/checkout@v2
+ uses: actions/checkout@v4
- name: Set up JDK 11
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
- name: Setup NodeJS
- uses: actions/setup-node@v2
+ uses: actions/setup-node@v4
with:
node-version: '16'
diff --git a/.github/workflows/website-pr-validation.yml b/.github/workflows/website-pr-validation.yml
index f66dbf9f5d..9d38cb59f2 100644
--- a/.github/workflows/website-pr-validation.yml
+++ b/.github/workflows/website-pr-validation.yml
@@ -35,21 +35,17 @@ jobs:
timeout-minutes: 60
steps:
- name: Checkout
- uses: actions/checkout@v2
+ uses: actions/checkout@v4
- name: Set up JDK 11
- uses: actions/setup-java@v2
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
- - name: Set up Maven
- uses: apache/pulsar-test-infra/setup-maven@master
- with:
- maven-version: 3.8.7
- name: Setup NodeJS
- uses: actions/setup-node@v2
+ uses: actions/setup-node@v4
with:
node-version: '16'
diff --git a/bin/bookkeeper-daemon.sh b/bin/bookkeeper-daemon.sh
index 7e93b86334..4e85a50ed6 100755
--- a/bin/bookkeeper-daemon.sh
+++ b/bin/bookkeeper-daemon.sh
@@ -157,7 +157,17 @@ stop()
if kill -0 $TARGET_PID > /dev/null 2>&1; then
fileName=$location/$command.out
- $JAVA_HOME/bin/jstack $TARGET_PID > $fileName
+ # Check for the java to use
+ if [[ -z ${JAVA_HOME} ]]; then
+ JSTACK=$(which jstack)
+ if [ $? -ne 0 ]; then
+ echo "Error: JAVA_HOME not set, and no jstack executable found in $PATH." 1>&2
+ exit 1
+ fi
+ else
+ JSTACK=${JAVA_HOME}/bin/jstack
+ fi
+ $JSTACK $TARGET_PID > $fileName
echo Thread dumps are taken for analysis at $fileName
if [ "$1" == "-force" ]
then
diff --git a/bin/common.sh b/bin/common.sh
index e872779531..72779396a6 100755
--- a/bin/common.sh
+++ b/bin/common.sh
@@ -51,9 +51,7 @@ fi
# Check for the java to use
if [[ -z ${JAVA_HOME} ]]; then
JAVA=$(which java)
- if [ $? = 0 ]; then
- echo "JAVA_HOME not set, using java from PATH. ($JAVA)"
- else
+ if [ $? != 0 ]; then
echo "Error: JAVA_HOME not set, and no java executable found in $PATH." 1>&2
exit 1
fi
@@ -71,13 +69,12 @@ source ${BK_CONFDIR}/bkenv.sh
source ${BK_CONFDIR}/bk_cli_env.sh
detect_jdk8() {
-
- if [ -f "$JAVA_HOME/lib/modules" ]; then
+ local is_java_8=$($JAVA -version 2>&1 | grep version | grep '"1\.8')
+ if [ -z "$is_java_8" ]; then
echo "0"
else
echo "1"
fi
- return
}
# default netty settings
@@ -349,7 +346,7 @@ find_table_service() {
TABLE_SERVICE_RELEASED="false"
fi
fi
-
+
# check the configuration to see if table service is enabled or not.
if [ -z "${ENABLE_TABLE_SERVICE}" ]; then
# mask exit code if the configuration file doesn't contain `StreamStorageLifecycleComponent`
@@ -362,7 +359,7 @@ find_table_service() {
ENABLE_TABLE_SERVICE="true"
fi
fi
-
+
# standalone only run
if [ \( "x${SERVICE_COMMAND}" == "xstandalone" \) -a \( "x${TABLE_SERVICE_RELEASED}" == "xfalse" \) ]; then
echo "The release binary is built without table service. Use \`localbookie <n>\` instead of \`standalone\` for local development."
diff --git a/bookkeeper-common/pom.xml b/bookkeeper-common/pom.xml
index d586c50fab..2947822782 100644
--- a/bookkeeper-common/pom.xml
+++ b/bookkeeper-common/pom.xml
@@ -130,12 +130,13 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
- <version>${maven-javadoc-plugin.version}</version>
<configuration>
<sourcepath>${src.dir}</sourcepath>
<!-- Avoid for missing javadoc comments to be marked as errors -->
<doclint>none</doclint>
+ <!--
<subpackages>org.apache.bookkeeper.common.annotation</subpackages>
+ -->
<groups>
<group>
<title>Bookkeeper Client</title>
diff --git a/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt b/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt
index 830e9a7275..28d0baefaf 100644
--- a/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt
+++ b/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt
@@ -298,7 +298,6 @@ Apache Software License, Version 2.
- lib/org.apache.curator-curator-client-5.1.0.jar [34]
- lib/org.apache.curator-curator-framework-5.1.0.jar [34]
- lib/org.apache.curator-curator-recipes-5.1.0.jar [34]
-- lib/org.inferred-freebuilder-2.7.0.jar [35]
- lib/com.google.errorprone-error_prone_annotations-2.9.0.jar [36]
- lib/org.apache.yetus-audience-annotations-0.12.0.jar [37]
- lib/org.jctools-jctools-core-2.1.2.jar [38]
@@ -380,7 +379,6 @@ Apache Software License, Version 2.
[30] Source available at https://github.com/census-instrumentation/opencensus-java/tree/v0.28.0
[33] Source available at https://github.com/grpc/grpc-java/tree/v1.56.0
[34] Source available at https://github.com/apache/curator/releases/tag/apache.curator-5.1.0
-[35] Source available at https://github.com/inferred/FreeBuilder/tree/v2.7.0
[36] Source available at https://github.com/google/error-prone/tree/v2.9.0
[37] Source available at https://github.com/apache/yetus/tree/rel/0.12.0
[38] Source available at https://github.com/JCTools/JCTools/tree/v2.1.2
diff --git a/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt b/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt
index 18f830c4f7..df93ad4db4 100644
--- a/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt
+++ b/bookkeeper-dist/src/main/resources/LICENSE-bkctl.bin.txt
@@ -273,7 +273,6 @@ Apache Software License, Version 2.
- lib/org.apache.curator-curator-client-5.1.0.jar [33]
- lib/org.apache.curator-curator-framework-5.1.0.jar [33]
- lib/org.apache.curator-curator-recipes-5.1.0.jar [33]
-- lib/org.inferred-freebuilder-2.7.0.jar [34]
- lib/com.google.errorprone-error_prone_annotations-2.9.0.jar [35]
- lib/org.apache.yetus-audience-annotations-0.12.0.jar [36]
- lib/org.jctools-jctools-core-2.1.2.jar [37]
@@ -315,7 +314,6 @@ Apache Software License, Version 2.
[29] Source available at https://github.com/census-instrumentation/opencensus-java/tree/v0.28.0
[32] Source available at https://github.com/grpc/grpc-java/tree/v1.56.0
[33] Source available at https://github.com/apache/curator/tree/apache-curator-5.1.0
-[34] Source available at https://github.com/inferred/FreeBuilder/tree/v2.7.0
[35] Source available at https://github.com/google/error-prone/tree/v2.9.0
[36] Source available at https://github.com/apache/yetus/tree/rel/0.12.0
[37] Source available at https://github.com/JCTools/JCTools/tree/v2.1.2
diff --git a/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt b/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt
index 0628981359..3735796025 100644
--- a/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt
+++ b/bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt
@@ -298,7 +298,6 @@ Apache Software License, Version 2.
- lib/org.apache.curator-curator-client-5.1.0.jar [34]
- lib/org.apache.curator-curator-framework-5.1.0.jar [34]
- lib/org.apache.curator-curator-recipes-5.1.0.jar [34]
-- lib/org.inferred-freebuilder-2.7.0.jar [35]
- lib/com.google.errorprone-error_prone_annotations-2.9.0.jar [36]
- lib/org.apache.yetus-audience-annotations-0.12.0.jar [37]
- lib/org.jctools-jctools-core-2.1.2.jar [38]
@@ -376,7 +375,6 @@ Apache Software License, Version 2.
[30] Source available at https://github.com/census-instrumentation/opencensus-java/tree/v0.28.0
[33] Source available at https://github.com/grpc/grpc-java/tree/v1.56.0
[34] Source available at https://github.com/apache/curator/releases/tag/apache.curator-5.1.0
-[35] Source available at https://github.com/inferred/FreeBuilder/tree/v2.7.0
[36] Source available at https://github.com/google/error-prone/tree/v2.9.0
[37] Source available at https://github.com/apache/yetus/tree/rel/0.12.0
[38] Source available at https://github.com/JCTools/JCTools/tree/v2.1.2
diff --git a/bookkeeper-server/pom.xml b/bookkeeper-server/pom.xml
index 47b2addc71..0b7cdf4545 100644
--- a/bookkeeper-server/pom.xml
+++ b/bookkeeper-server/pom.xml
@@ -275,7 +275,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<trimStackTrace>false</trimStackTrace>
<properties>
@@ -289,12 +288,13 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
- <version>${maven-javadoc-plugin.version}</version>
<configuration>
<sourcepath>${src.dir}</sourcepath>
<!-- Avoid for missing javadoc comments to be marked as errors -->
<doclint>none</doclint>
+ <!--
<subpackages>org.apache.bookkeeper.client:org.apache.bookkeeper.conf:org.apache.bookkeeper.feature</subpackages>
+ -->
<groups>
<group>
<title>Bookkeeper</title>
diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBase.java b/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBase.java
index 430333bd2f..643ddd3ced 100644
--- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBase.java
+++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBase.java
@@ -64,6 +64,7 @@ import org.apache.zookeeper.data.Stat;
public class ZKMetadataDriverBase implements AutoCloseable {
protected static final String SCHEME = "zk";
+ private static final int ZK_CLIENT_WAIT_FOR_SHUTDOWN_TIMEOUT_MS = 5000;
public static String getZKServersFromServiceUri(URI uri) {
String authority = uri.getAuthority();
@@ -341,7 +342,7 @@ public class ZKMetadataDriverBase implements AutoCloseable {
}
if (ownZKHandle && null != zk) {
try {
- zk.close();
+ zk.close(ZK_CLIENT_WAIT_FOR_SHUTDOWN_TIMEOUT_MS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
log.warn("Interrupted on closing zookeeper client", e);
diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommand.java b/bookkeeper-server/src/main/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommand.java
index ca94cac4a7..7b43006a19 100644
--- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommand.java
+++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommand.java
@@ -88,17 +88,18 @@ public class SimpleTestCommand extends ClientCommand<Flags> {
for (int i = 0; i < data.length; i++) {
data[i] = (byte) (random.nextInt(26) + 65);
}
- WriteHandle wh = null;
- try {
- wh = result(bk.newCreateLedgerOp()
- .withEnsembleSize(flags.ensembleSize)
- .withWriteQuorumSize(flags.writeQuorumSize)
- .withAckQuorumSize(flags.ackQuorumSize)
- .withDigestType(DigestType.CRC32C)
- .withCustomMetadata(ImmutableMap.of("Bookie", NAME.getBytes(StandardCharsets.UTF_8)))
- .withPassword(new byte[0])
- .execute());
- LOG.info("Ledger ID: {}", wh.getId());
+ long ledgerId = -1L;
+ long lastEntryId = -1L;
+ try (WriteHandle wh = result(bk.newCreateLedgerOp()
+ .withEnsembleSize(flags.ensembleSize)
+ .withWriteQuorumSize(flags.writeQuorumSize)
+ .withAckQuorumSize(flags.ackQuorumSize)
+ .withDigestType(DigestType.CRC32C)
+ .withCustomMetadata(ImmutableMap.of("Bookie", NAME.getBytes(StandardCharsets.UTF_8)))
+ .withPassword(new byte[0])
+ .execute())) {
+ ledgerId = wh.getId();
+ LOG.info("Ledger ID: {}", ledgerId);
long lastReport = System.nanoTime();
for (int i = 0; i < flags.numEntries; i++) {
wh.append(data);
@@ -108,22 +109,50 @@ public class SimpleTestCommand extends ClientCommand<Flags> {
lastReport = System.nanoTime();
}
}
- LOG.info("{} entries written to ledger {}", flags.numEntries, wh.getId());
+ lastEntryId = wh.getLastAddPushed();
+ LOG.info("{} entries written to ledger {}. Last entry Id {}", flags.numEntries, ledgerId, lastEntryId);
+ if (lastEntryId != flags.numEntries - 1) {
+ throw new IllegalStateException("Last entry id doesn't match the expected value");
+ }
+ // check that all entries are readable
+ readEntries(bk, ledgerId, lastEntryId, flags.numEntries, true, data);
+ }
+ if (ledgerId != -1) {
+ try {
+ if (lastEntryId != -1) {
+ // check that all entries are readable and confirmed
+ readEntries(bk, ledgerId, lastEntryId, flags.numEntries, false, data);
+ } else {
+ throw new IllegalStateException("Last entry id is not set");
+ }
+ } finally {
+ // delete the ledger
+ result(bk.newDeleteLedgerOp().withLedgerId(ledgerId).execute());
+ }
+ } else {
+ throw new IllegalStateException("Ledger id is not set");
+ }
+ }
- try (ReadHandle rh = result(bk.newOpenLedgerOp().withLedgerId(wh.getId()).withDigestType(DigestType.CRC32C)
- .withPassword(new byte[0]).execute())) {
- LedgerEntries ledgerEntries = rh.read(0, flags.numEntries);
+ private static void readEntries(BookKeeper bk, long ledgerId, long lastEntryId, int expectedNumberOfEntries,
+ boolean readUnconfirmed, byte[] data) throws Exception {
+ int entriesRead = 0;
+ try (ReadHandle rh = result(bk.newOpenLedgerOp().withLedgerId(ledgerId).withDigestType(DigestType.CRC32C)
+ .withPassword(new byte[0]).execute())) {
+ try (LedgerEntries ledgerEntries = readUnconfirmed ? rh.readUnconfirmed(0, lastEntryId) :
+ rh.read(0, lastEntryId)) {
for (LedgerEntry ledgerEntry : ledgerEntries) {
if (!Arrays.equals(ledgerEntry.getEntryBytes(), data)) {
- LOG.error("Read test failed, the reading data is not equals writing data.");
+ throw new IllegalStateException("Read data doesn't match the data written.");
}
+ entriesRead++;
}
}
- } finally {
- if (wh != null) {
- wh.close();
- result(bk.newDeleteLedgerOp().withLedgerId(wh.getId()).execute());
- }
+ }
+ if (entriesRead != expectedNumberOfEntries) {
+ throw new IllegalStateException(
+ String.format("Number of entries read (%d) doesn't match the expected value (%d).",
+ entriesRead, expectedNumberOfEntries));
}
}
}
diff --git a/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBaseTest.java b/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBaseTest.java
index ab9c0030f4..e02fb29bae 100644
--- a/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBaseTest.java
+++ b/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/zk/ZKMetadataDriverBaseTest.java
@@ -90,7 +90,7 @@ public class ZKMetadataDriverBaseTest extends ZKMetadataDriverTestBase {
driver.close();
- verify(mockZkc, times(1)).close();
+ verify(mockZkc, times(1)).close(5000);
assertNull(driver.zk);
}
diff --git a/circe-checksum/pom.xml b/circe-checksum/pom.xml
index d5aaf5fa9f..7100f4da6b 100644
--- a/circe-checksum/pom.xml
+++ b/circe-checksum/pom.xml
@@ -63,25 +63,32 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
- <configuration>
- <source>1.8</source>
- <target>1.8</target>
- <compilerArgs>
- <!-- Object.finalize() is deprecated at java 9 -->
- <!-- <compilerArg>-Werror</compilerArg> -->
- <compilerArg>-Xlint:deprecation</compilerArg>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
- </compilerArgs>
- </configuration>
</plugin>
<plugin>
<groupId>com.github.maven-nar</groupId>
<artifactId>nar-maven-plugin</artifactId>
- <version>${nar-maven-plugin.version}</version>
<extensions>true</extensions>
+ <configuration>
+ <runtime>${nar.runtime}</runtime>
+ <output>circe-checksum</output>
+ <libraries>
+ <library>
+ <type>jni</type>
+ <narSystemPackage>com.scurrilous.circe.checksum</narSystemPackage>
+ </library>
+ </libraries>
+ <cpp>
+ <optionSet>${nar.cpp.optionSet}</optionSet>
+ <exceptions>false</exceptions>
+ <rtti>false</rtti>
+ <optimize>full</optimize>
+ <includes>
+ <include>**/*.cpp</include>
+ </includes>
+ </cpp>
+ <!-- skip by default, enabled in OS profiles -->
+ <skip>true</skip>
+ </configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
@@ -121,7 +128,6 @@
<plugin>
<groupId>com.github.maven-nar</groupId>
<artifactId>nar-maven-plugin</artifactId>
- <version>${nar-maven-plugin.version}</version>
<extensions>true</extensions>
<executions>
<execution>
@@ -135,17 +141,8 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
<configuration>
- <source>1.8</source>
- <target>1.8</target>
<compilerArgs>
- <!-- Object.finalize() is deprecated at java 9 -->
- <!-- <compilerArg>-Werror</compilerArg> -->
- <compilerArg>-Xlint:deprecation</compilerArg>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
<!-- add -h flag to javac -->
<compilerArg>-h</compilerArg>
<compilerArg>${project.build.directory}/nar/javah-include</compilerArg>
@@ -167,23 +164,9 @@
<plugin>
<groupId>com.github.maven-nar</groupId>
<artifactId>nar-maven-plugin</artifactId>
- <version>${nar-maven-plugin.version}</version>
<extensions>true</extensions>
<configuration>
- <runtime>${nar.runtime}</runtime>
- <output>circe-checksum</output>
- <libraries>
- <library>
- <type>jni</type>
- <narSystemPackage>com.scurrilous.circe.checksum</narSystemPackage>
- </library>
- </libraries>
- <cpp>
- <optionSet>${nar.cpp.optionSet}</optionSet>
- <exceptions>false</exceptions>
- <rtti>false</rtti>
- <optimize>full</optimize>
- </cpp>
+ <skip>false</skip>
</configuration>
</plugin>
</plugins>
@@ -201,23 +184,9 @@
<plugin>
<groupId>com.github.maven-nar</groupId>
<artifactId>nar-maven-plugin</artifactId>
- <version>${nar-maven-plugin.version}</version>
<extensions>true</extensions>
<configuration>
- <runtime>${nar.runtime}</runtime>
- <output>circe-checksum</output>
- <libraries>
- <library>
- <type>jni</type>
- <narSystemPackage>com.scurrilous.circe.checksum</narSystemPackage>
- </library>
- </libraries>
- <cpp>
- <optionSet>${nar.cpp.optionSet}</optionSet>
- <exceptions>false</exceptions>
- <rtti>false</rtti>
- <optimize>full</optimize>
- </cpp>
+ <skip>false</skip>
<linker>
<libSet>rt</libSet>
</linker>
@@ -238,23 +207,9 @@
<plugin>
<groupId>com.github.maven-nar</groupId>
<artifactId>nar-maven-plugin</artifactId>
- <version>${nar-maven-plugin.version}</version>
<extensions>true</extensions>
<configuration>
- <runtime>${nar.runtime}</runtime>
- <output>circe-checksum</output>
- <libraries>
- <library>
- <type>jni</type>
- <narSystemPackage>com.scurrilous.circe.checksum</narSystemPackage>
- </library>
- </libraries>
- <cpp>
- <optionSet>${nar.cpp.optionSet}</optionSet>
- <exceptions>false</exceptions>
- <rtti>false</rtti>
- <optimize>full</optimize>
- </cpp>
+ <skip>false</skip>
<linker>
<name>g++</name>
</linker>
diff --git a/cpu-affinity/pom.xml b/cpu-affinity/pom.xml
index c78e5f1ea9..a80b712587 100644
--- a/cpu-affinity/pom.xml
+++ b/cpu-affinity/pom.xml
@@ -48,18 +48,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <configuration>
- <source>1.8</source>
- <target>1.8</target>
- <compilerArgs>
- <!-- Object.finalize() is deprecated at java 9 -->
- <!-- <compilerArg>-Werror</compilerArg> -->
- <compilerArg>-Xlint:deprecation</compilerArg>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
- </compilerArgs>
- </configuration>
</plugin>
<plugin>
<groupId>com.github.maven-nar</groupId>
@@ -129,17 +117,8 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
<configuration>
- <source>1.8</source>
- <target>1.8</target>
<compilerArgs>
- <!-- Object.finalize() is deprecated at java 9 -->
- <!-- <compilerArg>-Werror</compilerArg> -->
- <compilerArg>-Xlint:deprecation</compilerArg>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
<!-- add -h flag to javac -->
<compilerArg>-h</compilerArg>
<compilerArg>${project.build.directory}/nar/javah-include</compilerArg>
diff --git a/dev/ci-tool b/dev/ci-tool
index 08bc2e1386..0f26955f64 100755
--- a/dev/ci-tool
+++ b/dev/ci-tool
@@ -80,6 +80,45 @@ function ci_move_test_reports() {
)
}
+# Finds fastest up-to-date ubuntu mirror based on download speed
+function ci_find_fast_ubuntu_mirror() {
+ local ubuntu_release=${1:-"$(lsb_release -c 2>/dev/null | cut -f2 || echo "jammy")"}
+ local ubuntu_arch=${2:-"$(dpkg --print-architecture 2>/dev/null || echo "amd64")"}
+ {
+ # choose mirrors that are up-to-date by checking the Last-Modified header for
+ {
+ # randomly choose up to 10 mirrors using http:// protocol
+ # (https isn't supported in docker containers that don't have ca-certificates installed)
+ curl -s http://mirrors.ubuntu.com/mirrors.txt | grep '^http://' | shuf -n 10
+ # also consider Azure's Ubuntu mirror
+ echo http://azure.archive.ubuntu.com/ubuntu/
+ } | xargs -I {} sh -c "ubuntu_release=$ubuntu_release ubuntu_arch=$ubuntu_arch;"'echo "$(curl -m 5 -sI {}dists/${ubuntu_release}/Contents-${ubuntu_arch}.gz|sed s/\\r\$//|grep Last-Modified|awk -F": " "{ print \$2 }" | LANG=C date -f- -u +%s)" "{}"' | sort -rg | awk '{ if (NR==1) TS=$1; if ($1 == TS) print $2 }'
+ } | xargs -I {} sh -c 'echo `curl -r 0-102400 -m 5 -s -w %{speed_download} -o /dev/null {}ls-lR.gz` {}' \
+ |sort -g -r |head -1| awk '{ print $2 }'
+}
+
+function ci_pick_ubuntu_mirror() {
+ echo "Choosing fastest up-to-date ubuntu mirror based on download speed..."
+ UBUNTU_MIRROR=$(ci_find_fast_ubuntu_mirror)
+ if [ -z "$UBUNTU_MIRROR" ]; then
+ # fallback to no mirror
+ UBUNTU_MIRROR="http://archive.ubuntu.com/ubuntu/"
+ UBUNTU_SECURITY_MIRROR="http://security.ubuntu.com/ubuntu/"
+ else
+ UBUNTU_SECURITY_MIRROR="${UBUNTU_MIRROR}"
+ fi
+ echo "Picked '$UBUNTU_MIRROR'."
+ # set the chosen mirror also in the UBUNTU_MIRROR and UBUNTU_SECURITY_MIRROR environment variables
+ # that can be used by docker builds
+ export UBUNTU_MIRROR
+ export UBUNTU_SECURITY_MIRROR
+ # make environment variables available for later GitHub Actions steps
+ if [ -n "$GITHUB_ENV" ]; then
+ echo "UBUNTU_MIRROR=$UBUNTU_MIRROR" >> $GITHUB_ENV
+ echo "UBUNTU_SECURITY_MIRROR=$UBUNTU_SECURITY_MIRROR" >> $GITHUB_ENV
+ fi
+}
+
if [ -z "$1" ]; then
echo "usage: $0 [ci_tool_function_name]"
echo "Available ci tool functions:"
diff --git a/docker/Dockerfile b/docker/Dockerfile
index fafb7dea0d..87b8899b39 100644
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -17,11 +17,9 @@
# under the License.
#
-FROM ubuntu:22.10
+FROM ubuntu:22.04
MAINTAINER Apache BookKeeper <de...@bookkeeper.apache.org>
-ARG TARGETARCH
-
ARG BK_VERSION=4.15.1
ARG DISTRO_NAME=bookkeeper-server-${BK_VERSION}-bin
ARG DISTRO_URL=https://archive.apache.org/dist/bookkeeper/bookkeeper-${BK_VERSION}/${DISTRO_NAME}.tar.gz
@@ -31,13 +29,18 @@ ENV BOOKIE_HTTP_PORT=8080
EXPOSE $BOOKIE_PORT $BOOKIE_HTTP_PORT
ENV BK_USER=bookkeeper
ENV BK_HOME=/opt/bookkeeper
-ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-$TARGETARCH
ENV DEBIAN_FRONTEND=noninteractive
+ARG UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/
+ARG UBUNTU_SECURITY_MIRROR=http://security.ubuntu.com/ubuntu/
# Download Apache Bookkeeper, untar and clean up
RUN set -x \
+ && sed -i -e "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}|g" \
+ -e "s|http://security\.ubuntu\.com/ubuntu/|${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}|g" /etc/apt/sources.list \
+ && echo 'Acquire::http::Timeout "30";\nAcquire::http::ConnectionAttemptDelayMsec "2000";\nAcquire::https::Timeout "30";\nAcquire::https::ConnectionAttemptDelayMsec "2000";\nAcquire::ftp::Timeout "30";\nAcquire::ftp::ConnectionAttemptDelayMsec "2000";\nAcquire::Retries "15";' > /etc/apt/apt.conf.d/99timeout_and_retries \
&& adduser "${BK_USER}" \
&& apt-get update \
+ && apt-get install -y ca-certificates apt-transport-https \
&& apt-get install -y --no-install-recommends openjdk-17-jdk \
&& apt-get install -y --no-install-recommends python3 pip \
&& ln -s /usr/bin/python3 /usr/bin/python \
@@ -58,7 +61,10 @@ RUN set -x \
&& tar -xzf "$DISTRO_NAME.tar.gz" \
&& mv bookkeeper-server-${BK_VERSION}/ /opt/bookkeeper/ \
&& rm -rf "$DISTRO_NAME.tar.gz" "$DISTRO_NAME.tar.gz.asc" "$DISTRO_NAME.tar.gz.sha512" \
- && pip install zk-shell
+ && pip install zk-shell \
+ && JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))) \
+ && echo networkaddress.cache.ttl=1 >> $JAVA_HOME/conf/security/java.security \
+ && echo networkaddress.cache.negative.ttl=1 >> $JAVA_HOME/conf/security/java.security
WORKDIR /opt/bookkeeper
diff --git a/docker/README.md b/docker/README.md
index 462f1ae2a1..c2bbe420d8 100644
--- a/docker/README.md
+++ b/docker/README.md
@@ -13,7 +13,6 @@ Bookkeeper needs [Zookeeper](https://zookeeper.apache.org/) in order to preserve
Just like running a BookKeeper cluster in one machine(https://bookkeeper.apache.org/docs/getting-started/run-locally/), you can run a standalone BookKeeper in one docker container, the command is:
```
docker run -it \
- --env JAVA_HOME=/usr/lib/jvm/java-11 \
--entrypoint "/bin/bash" \
apache/bookkeeper \
-c "/opt/bookkeeper/bin/bookkeeper localbookie 3"
diff --git a/metadata-drivers/etcd/src/test/java/org/apache/bookkeeper/metadata/etcd/helpers/KeySetReaderTest.java b/metadata-drivers/etcd/src/test/java/org/apache/bookkeeper/metadata/etcd/helpers/KeySetReaderTest.java
index b4bdf6ea79..d7a07fc2ff 100644
--- a/metadata-drivers/etcd/src/test/java/org/apache/bookkeeper/metadata/etcd/helpers/KeySetReaderTest.java
+++ b/metadata-drivers/etcd/src/test/java/org/apache/bookkeeper/metadata/etcd/helpers/KeySetReaderTest.java
@@ -40,8 +40,8 @@ import org.apache.bookkeeper.versioning.LongVersion;
import org.apache.bookkeeper.versioning.Version.Occurred;
import org.apache.bookkeeper.versioning.Versioned;
import org.apache.commons.compress.utils.Sets;
+import org.apache.commons.lang3.RandomStringUtils;
import org.junit.Test;
-import org.testcontainers.shaded.org.apache.commons.lang3.RandomStringUtils;
/**
* Integration test {@link KeySetReader}.
diff --git a/microbenchmarks/pom.xml b/microbenchmarks/pom.xml
index 57afff4e4e..881b7cd751 100644
--- a/microbenchmarks/pom.xml
+++ b/microbenchmarks/pom.xml
@@ -16,7 +16,7 @@
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
- <modelVersion>4.0.0</modelVersion>
+ <modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper</artifactId>
@@ -58,14 +58,24 @@
</dependencies>
<build>
<plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-deploy-plugin</artifactId>
+ <configuration>
+ <skip>true</skip>
+ </configuration>
+ </plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
<configuration>
- <compilerVersion>${javac.target}</compilerVersion>
- <source>${javac.target}</source>
- <target>${javac.target}</target>
+ <annotationProcessorPaths combine.children="append">
+ <path>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-generator-annprocess</artifactId>
+ <version>${jmh.version}</version>
+ </path>
+ </annotationProcessorPaths>
</configuration>
</plugin>
<plugin>
@@ -82,7 +92,10 @@
<finalName>${uberjar.name}</finalName>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
- <mainClass>org.openjdk.jmh.Main</mainClass>
+ <manifestEntries>
+ <Main-Class>org.openjdk.jmh.Main</Main-Class>
+ <Multi-Release>true</Multi-Release>
+ </manifestEntries>
</transformer>
</transformers>
<filters>
@@ -108,6 +121,13 @@
<skip>true</skip>
</configuration>
</plugin>
+ <plugin>
+ <groupId>com.github.spotbugs</groupId>
+ <artifactId>spotbugs-maven-plugin</artifactId>
+ <configuration>
+ <skip>true</skip>
+ </configuration>
+ </plugin>
</plugins>
</build>
</project>
diff --git a/native-io/pom.xml b/native-io/pom.xml
index 5bba152a50..fd64b4f0ca 100644
--- a/native-io/pom.xml
+++ b/native-io/pom.xml
@@ -45,18 +45,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <configuration>
- <source>1.8</source>
- <target>1.8</target>
- <compilerArgs>
- <!-- Object.finalize() is deprecated at java 9 -->
- <!-- <compilerArg>-Werror</compilerArg> -->
- <compilerArg>-Xlint:deprecation</compilerArg>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
- </compilerArgs>
- </configuration>
</plugin>
<plugin>
<groupId>com.github.maven-nar</groupId>
@@ -120,17 +108,8 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
<configuration>
- <source>1.8</source>
- <target>1.8</target>
<compilerArgs>
- <!-- Object.finalize() is deprecated at java 9 -->
- <!-- <compilerArg>-Werror</compilerArg> -->
- <compilerArg>-Xlint:deprecation</compilerArg>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
<!-- add -h flag to javac -->
<compilerArg>-h</compilerArg>
<compilerArg>${project.build.directory}/nar/javah-include</compilerArg>
@@ -228,11 +207,11 @@
<extensions>true</extensions>
<configuration>
<runtime>${nar.runtime}</runtime>
- <output>circe-checksum</output>
+ <output>native-io</output>
<libraries>
<library>
<type>jni</type>
- <narSystemPackage>com.scurrilous.circe.checksum</narSystemPackage>
+ <narSystemPackage>org.apache.bookkeeper.util.nativeio</narSystemPackage>
</library>
</libraries>
<cpp>
diff --git a/pom.xml b/pom.xml
index a7394b4907..cb2cb2ed54 100644
--- a/pom.xml
+++ b/pom.xml
@@ -105,9 +105,10 @@
</developer>
</developers>
<properties>
+ <maven.compiler.source>8</maven.compiler.source>
+ <maven.compiler.target>8</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
- <javac.target>1.8</javac.target>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<testRetryCount>2</testRetryCount>
<src.dir>src/main/java</src.dir>
@@ -115,7 +116,7 @@
<!-- dependencies -->
<arquillian-cube.version>1.18.2</arquillian-cube.version>
- <arquillian-junit.version>1.6.0.Final</arquillian-junit.version>
+ <arquillian-junit.version>1.8.0.Final</arquillian-junit.version>
<codahale.metrics.version>3.0.1</codahale.metrics.version>
<commons-cli.version>1.2</commons-cli.version>
<commons-collections4.version>4.1</commons-collections4.version>
@@ -129,26 +130,27 @@
<curator.version>5.1.0</curator.version>
<dropwizard.version>4.1.12.1</dropwizard.version>
<etcd.version>0.5.11</etcd.version>
- <freebuilder.version>2.7.0</freebuilder.version>
+ <freebuilder.version>2.8.0</freebuilder.version>
<google.code.version>3.0.2</google.code.version>
<google.errorprone.version>2.9.0</google.errorprone.version>
<grpc.version>1.56.0</grpc.version>
<guava.version>32.0.1-jre</guava.version>
<kerby.version>1.1.1</kerby.version>
<hadoop.version>3.3.5</hadoop.version>
- <hamcrest.version>1.3</hamcrest.version>
<hdrhistogram.version>2.1.10</hdrhistogram.version>
<jackson.version>2.13.4.20221013</jackson.version>
<jcommander.version>1.82</jcommander.version>
<jetty.version>9.4.53.v20231009</jetty.version>
- <jmh.version>1.19</jmh.version>
+ <jmh.version>1.37</jmh.version>
<jmock.version>2.8.2</jmock.version>
<jsoup.version>1.15.3</jsoup.version>
- <junit.version>4.12</junit.version>
- <!-- required by zookeeper test utilities imported from ZooKeeper -->
- <junit5.version>5.8.2</junit5.version>
+ <junit.version>4.13.2</junit.version>
+ <hamcrest.version>1.3</hamcrest.version>
+ <junit5.version>5.10.2</junit5.version>
+ <assertj-core.version>3.25.3</assertj-core.version>
+ <awaitility.version>4.2.0</awaitility.version>
<libthrift.version>0.14.2</libthrift.version>
- <lombok.version>1.18.26</lombok.version>
+ <lombok.version>1.18.30</lombok.version>
<log4j.version>2.18.0</log4j.version>
<lz4.version>1.3.0</lz4.version>
<mockito.version>3.12.4</mockito.version>
@@ -166,40 +168,39 @@
<protoc-gen-grpc-java.version>${grpc.version}</protoc-gen-grpc-java.version>
<reflections.version>0.9.11</reflections.version>
<rocksdb.version>7.9.2</rocksdb.version>
- <shrinkwrap.version>3.0.1</shrinkwrap.version>
+ <shrinkwrap.version>3.3.0</shrinkwrap.version>
<slf4j.version>1.7.32</slf4j.version>
<snakeyaml.version>2.0</snakeyaml.version>
<spotbugs-annotations.version>4.6.0</spotbugs-annotations.version>
<javax-annotations-api.version>1.3.2</javax-annotations-api.version>
- <testcontainers.version>1.17.6</testcontainers.version>
+ <testcontainers.version>1.19.4</testcontainers.version>
<vertx.version>4.3.8</vertx.version>
<zookeeper.version>3.8.3</zookeeper.version>
<snappy.version>1.1.10.5</snappy.version>
<jctools.version>2.1.2</jctools.version>
<hppc.version>0.9.1</hppc.version>
<!-- plugin dependencies -->
- <apache-rat-plugin.version>0.12</apache-rat-plugin.version>
+ <apache-rat-plugin.version>0.16.1</apache-rat-plugin.version>
<cobertura-maven-plugin.version>2.7</cobertura-maven-plugin.version>
<coveralls-maven-plugin.version>4.3.0</coveralls-maven-plugin.version>
<dockerfile-maven-plugin.version>1.4.13</dockerfile-maven-plugin.version>
<download-maven-plugin.version>1.6.8</download-maven-plugin.version>
<exec-maven-plugin.version>3.0.0</exec-maven-plugin.version>
- <gmavenplus-plugin.version>1.13.1</gmavenplus-plugin.version>
<license-maven-plugin.version>1.6</license-maven-plugin.version>
<lombok-maven-plugin.version>1.18.20.0</lombok-maven-plugin.version>
<maven-antrun-plugin.version>1.8</maven-antrun-plugin.version>
- <maven-assembly-plugin.version>3.1.0</maven-assembly-plugin.version>
- <maven-checkstyle-plugin.version>3.2.1</maven-checkstyle-plugin.version>
- <maven-clean-plugin.version>3.2.0</maven-clean-plugin.version>
- <maven-compiler-plugin.version>3.10.1</maven-compiler-plugin.version>
- <maven-dependency-plugin.version>3.0.2</maven-dependency-plugin.version>
- <maven-deploy-plugin.version>2.7</maven-deploy-plugin.version>
- <maven-install-plugin.version>2.5.1</maven-install-plugin.version>
- <maven-jar-plugin.version>3.2.2</maven-jar-plugin.version>
- <maven-javadoc-plugin.version>3.2.0</maven-javadoc-plugin.version>
- <maven-shade-plugin.version>3.2.0</maven-shade-plugin.version>
- <maven-source-plugin.version>3.2.1</maven-source-plugin.version>
- <maven-surefire-plugin.version>3.0.0-M5</maven-surefire-plugin.version>
+ <maven-assembly-plugin.version>3.6.0</maven-assembly-plugin.version>
+ <maven-checkstyle-plugin.version>3.3.1</maven-checkstyle-plugin.version>
+ <maven-clean-plugin.version>3.3.2</maven-clean-plugin.version>
+ <maven-compiler-plugin.version>3.12.1</maven-compiler-plugin.version>
+ <maven-dependency-plugin.version>3.6.1</maven-dependency-plugin.version>
+ <maven-deploy-plugin.version>3.1.1</maven-deploy-plugin.version>
+ <maven-install-plugin.version>3.1.1</maven-install-plugin.version>
+ <maven-jar-plugin.version>3.3.0</maven-jar-plugin.version>
+ <maven-javadoc-plugin.version>3.6.3</maven-javadoc-plugin.version>
+ <maven-shade-plugin.version>3.5.1</maven-shade-plugin.version>
+ <maven-source-plugin.version>3.3.0</maven-source-plugin.version>
+ <maven-surefire-plugin.version>3.2.5</maven-surefire-plugin.version>
<dependency-check-maven.version>8.0.2</dependency-check-maven.version>
<nar-maven-plugin.version>3.10.1</nar-maven-plugin.version>
<os-maven-plugin.version>1.4.1.Final</os-maven-plugin.version>
@@ -209,9 +210,9 @@
<forkCount.variable>1</forkCount.variable>
<servlet-api.version>4.0.0</servlet-api.version>
<rxjava.version>3.0.1</rxjava.version>
- <maven-failsafe-plugin.version>3.0.0-M6</maven-failsafe-plugin.version>
- <awaitility.version>4.0.3</awaitility.version>
<otel.version>1.26.0</otel.version>
+ <UBUNTU_MIRROR>http://archive.ubuntu.com/ubuntu/</UBUNTU_MIRROR>
+ <UBUNTU_SECURITY_MIRROR>http://security.ubuntu.com/ubuntu/</UBUNTU_SECURITY_MIRROR>
</properties>
<!-- dependency definitions -->
@@ -247,7 +248,6 @@
<groupId>org.inferred</groupId>
<artifactId>freebuilder</artifactId>
<version>${freebuilder.version}</version>
- <optional>true</optional>
</dependency>
<!-- logging dependencies -->
@@ -536,13 +536,6 @@
</exclusions>
</dependency>
- <dependency>
- <!-- needed by ZooKeeper server tests utilities -->
- <groupId>org.junit.jupiter</groupId>
- <artifactId>junit-jupiter-api</artifactId>
- <version>${junit5.version}</version>
- </dependency>
-
<dependency>
<!-- needed by ZooKeeper server -->
<groupId>org.xerial.snappy</groupId>
@@ -697,11 +690,33 @@
<artifactId>junit</artifactId>
<version>${junit.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.junit</groupId>
+ <artifactId>junit-bom</artifactId>
+ <version>${junit5.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.assertj</groupId>
+ <artifactId>assertj-core</artifactId>
+ <version>${assertj-core.version}</version>
+ </dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>${hamcrest.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.hamcrest</groupId>
+ <artifactId>hamcrest-core</artifactId>
+ <version>${hamcrest.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.hamcrest</groupId>
+ <artifactId>hamcrest-library</artifactId>
+ <version>${hamcrest.version}</version>
+ </dependency>
<dependency>
<groupId>org.jmock</groupId>
<artifactId>jmock</artifactId>
@@ -749,6 +764,11 @@
</exclusion>
</exclusions>
</dependency>
+ <dependency>
+ <groupId>javax.ws.rs</groupId>
+ <artifactId>javax.ws.rs-api</artifactId>
+ <version>2.1.1</version>
+ </dependency>
<dependency>
<groupId>org.jboss.arquillian.junit</groupId>
<artifactId>arquillian-junit-standalone</artifactId>
@@ -760,11 +780,6 @@
</exclusion>
</exclusions>
</dependency>
- <dependency>
- <groupId>org.jboss.arquillian.junit</groupId>
- <artifactId>arquillian-junit-container</artifactId>
- <version>${arquillian-junit.version}</version>
- </dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
@@ -783,8 +798,10 @@
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
- <artifactId>testcontainers</artifactId>
+ <artifactId>testcontainers-bom</artifactId>
<version>${testcontainers.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
@@ -848,6 +865,31 @@
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter-api</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter-engine</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.vintage</groupId>
+ <artifactId>junit-vintage-engine</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter-params</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.assertj</groupId>
+ <artifactId>assertj-core</artifactId>
+ <scope>test</scope>
+ </dependency>
<dependency><!-- Needed by junit -->
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
@@ -895,7 +937,7 @@
<configuration>
<configLocation>buildtools/src/main/resources/bookkeeper/checkstyle.xml</configLocation>
<suppressionsLocation>buildtools/src/main/resources/bookkeeper/suppressions.xml</suppressionsLocation>
- <encoding>UTF-8</encoding>
+ <inputEncoding>UTF-8</inputEncoding>
<consoleOutput>true</consoleOutput>
<failOnViolation>true</failOnViolation>
<includeResources>false</includeResources>
@@ -911,6 +953,31 @@
</execution>
</executions>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <version>${maven-surefire-plugin.version}</version>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-failsafe-plugin</artifactId>
+ <version>${maven-surefire-plugin.version}</version>
+ </plugin>
+ <plugin>
+ <groupId>com.github.maven-nar</groupId>
+ <artifactId>nar-maven-plugin</artifactId>
+ <version>${nar-maven-plugin.version}</version>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <version>${maven-javadoc-plugin.version}</version>
+ <configuration>
+ <!-- skip javadoc generation by default, use -Pdelombok to activate -->
+ <skip>true</skip>
+ <doclint>none</doclint>
+ </configuration>
+ </plugin>
</plugins>
</pluginManagement>
<plugins>
@@ -923,23 +990,28 @@
</configuration>
</plugin>
<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
- <source>${javac.target}</source>
- <target>${javac.target}</target>
+ <encoding>UTF-8</encoding>
+ <showDeprecation>true</showDeprecation>
+ <showWarnings>true</showWarnings>
+ <annotationProcessorPaths>
+ <path>
+ <groupId>org.projectlombok</groupId>
+ <artifactId>lombok</artifactId>
+ <version>${lombok.version}</version>
+ </path>
+ </annotationProcessorPaths>
<compilerArgs>
- <compilerArg>-Xlint:deprecation</compilerArg>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
- </compilerArgs>
+ <arg>-parameters</arg>
+ </compilerArgs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<argLine>-Xmx2G -Djava.net.preferIPv4Stack=true -Dio.netty.leakDetection.level=paranoid ${test.additional.args}</argLine>
<redirectTestOutputToFile>${redirectTestOutputToFile}</redirectTestOutputToFile>
@@ -953,17 +1025,16 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
- <version>${maven-failsafe-plugin.version}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
- <version>${maven-javadoc-plugin.version}</version>
<configuration>
<notimestamp>true</notimestamp>
- <!-- Prevent missing javadoc comments from being marked as errors -->
- <doclint>none</doclint>
+ <!--
+ Cannot specify subpackages with maven.compiler.release because of bug https://bugs.openjdk.org/browse/JDK-8175277
<subpackages>org.apache.bookkeeper.client:org.apache.bookkeeper.client.api:org.apache.bookkeeper.common.annotation:org.apache.bookkeeper.conf:org.apache.bookkeeper.feature:org.apache.bookkeeper.stats</subpackages>
+ -->
<groups>
<group>
<title>Bookkeeper Client</title>
@@ -1023,6 +1094,9 @@
<version>${apache-rat-plugin.version}</version>
<configuration>
<excludes>
+ <!-- This is generated during maven build -->
+ <exclude>dependency-reduced-pom.xml</exclude>
+
<!-- IntelliJ -->
<exclude>**/.idea/**</exclude>
@@ -1052,6 +1126,8 @@
<exclude>**/*.iml</exclude>
<exclude>**/*.iws</exclude>
<exclude>**/*.ipr</exclude>
+ <!-- sdkman -->
+ <exclude>**/.sdkmanrc</exclude>
<!-- Maven (CI builds) -->
<exclude>.repository/**</exclude>
@@ -1198,6 +1274,14 @@
</execution>
</executions>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-javadoc-plugin</artifactId>
+ <configuration>
+ <!-- activate javadoc generation -->
+ <skip>false</skip>
+ </configuration>
+ </plugin>
</plugins>
</build>
</profile>
@@ -1209,7 +1293,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<argLine>-Xmx2G -Djava.net.preferIPv4Stack=true</argLine>
<redirectTestOutputToFile>false</redirectTestOutputToFile>
@@ -1230,7 +1313,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<argLine>-Xmx2G -Djava.net.preferIPv4Stack=true -Dio.netty.leakDetection.level=paranoid -Dbookkeeper.log.root.level=INFO -Dbookkeeper.log.root.appender=CONSOLE</argLine>
<redirectTestOutputToFile>false</redirectTestOutputToFile>
@@ -1273,6 +1355,10 @@
<jdk>[11,)</jdk>
</activation>
<properties>
+ <!-- prevents silent NoSuchMethodErrors that happen at runtime on Java 8 -->
+ <!-- see https://github.com/apache/bookkeeper/issues/4202 -->
+ <maven.compiler.release>${maven.compiler.target}</maven.compiler.release>
+ <!-- required for running tests on Java 11+ -->
<test.additional.args>
--add-opens java.base/java.io=ALL-UNNAMED
--add-opens java.base/java.lang=ALL-UNNAMED
@@ -1295,6 +1381,20 @@
--add-opens java.xml/jdk.xml.internal=ALL-UNNAMED
</test.additional.args>
</properties>
+ <build>
+ <pluginManagement>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <configuration>
+ <!-- for some reason, setting maven.compiler.release property alone doesn't work -->
+ <release>${maven.compiler.release}</release>
+ </configuration>
+ </plugin>
+ </plugins>
+ </pluginManagement>
+ </build>
</profile>
<profile>
<id>apache-release</id>
@@ -1315,5 +1415,29 @@
</plugins>
</build>
</profile>
+ <profile>
+ <id>ubuntu-mirror-set</id>
+ <activation>
+ <property>
+ <name>env.UBUNTU_MIRROR</name>
+ </property>
+ </activation>
+ <properties>
+ <!-- Override the default value with the environment variable -->
+ <UBUNTU_MIRROR>${env.UBUNTU_MIRROR}</UBUNTU_MIRROR>
+ </properties>
+ </profile>
+ <profile>
+ <id>ubuntu-security-mirror-set</id>
+ <activation>
+ <property>
+ <name>env.UBUNTU_SECURITY_MIRROR</name>
+ </property>
+ </activation>
+ <properties>
+ <!-- Override the default value with the environment variable -->
+ <UBUNTU_SECURITY_MIRROR>${env.UBUNTU_SECURITY_MIRROR}</UBUNTU_SECURITY_MIRROR>
+ </properties>
+ </profile>
</profiles>
</project>
diff --git a/shaded/bookkeeper-server-shaded/pom.xml b/shaded/bookkeeper-server-shaded/pom.xml
index 77c077af46..3e266c6cda 100644
--- a/shaded/bookkeeper-server-shaded/pom.xml
+++ b/shaded/bookkeeper-server-shaded/pom.xml
@@ -38,10 +38,18 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-reload4j</artifactId>
+ </exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>ch.qos.reload4j</groupId>
+ <artifactId>reload4j</artifactId>
+ </exclusion>
</exclusions>
</dependency>
</dependencies>
diff --git a/shaded/bookkeeper-server-tests-shaded/pom.xml b/shaded/bookkeeper-server-tests-shaded/pom.xml
index e2a5f0dd6a..ab854fcb7f 100644
--- a/shaded/bookkeeper-server-tests-shaded/pom.xml
+++ b/shaded/bookkeeper-server-tests-shaded/pom.xml
@@ -39,10 +39,18 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-reload4j</artifactId>
+ </exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>ch.qos.reload4j</groupId>
+ <artifactId>reload4j</artifactId>
+ </exclusion>
<exclusion>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-common</artifactId>
@@ -103,7 +111,7 @@
</configuration>
</execution>
</executions>
- </plugin>
+ </plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>license-maven-plugin</artifactId>
diff --git a/shaded/distributedlog-core-shaded/pom.xml b/shaded/distributedlog-core-shaded/pom.xml
index 2f1562932b..2fc7a82750 100644
--- a/shaded/distributedlog-core-shaded/pom.xml
+++ b/shaded/distributedlog-core-shaded/pom.xml
@@ -39,10 +39,18 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-reload4j</artifactId>
+ </exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>ch.qos.reload4j</groupId>
+ <artifactId>reload4j</artifactId>
+ </exclusion>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-buffer</artifactId>
diff --git a/stats/bookkeeper-stats-api/pom.xml b/stats/bookkeeper-stats-api/pom.xml
index 87aacb3d8a..0246386735 100644
--- a/stats/bookkeeper-stats-api/pom.xml
+++ b/stats/bookkeeper-stats-api/pom.xml
@@ -33,12 +33,13 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
- <version>${maven-javadoc-plugin.version}</version>
<configuration>
<sourcepath>${src.dir}</sourcepath>
<!-- Avoid for missing javadoc comments to be marked as errors -->
<doclint>none</doclint>
+ <!--
<subpackages>org.apache.bookkeeper.stats</subpackages>
+ -->
<groups>
<group>
<title>Bookkeeper Stats API</title>
diff --git a/stream/api/pom.xml b/stream/api/pom.xml
index 7db758e62d..34ea6f78ca 100644
--- a/stream/api/pom.xml
+++ b/stream/api/pom.xml
@@ -41,9 +41,5 @@
<groupId>io.netty</groupId>
<artifactId>netty-buffer</artifactId>
</dependency>
- <dependency>
- <groupId>org.inferred</groupId>
- <artifactId>freebuilder</artifactId>
- </dependency>
</dependencies>
</project>
diff --git a/stream/clients/java/base/pom.xml b/stream/clients/java/base/pom.xml
index 1a9bf4ee4e..5df4c47c21 100644
--- a/stream/clients/java/base/pom.xml
+++ b/stream/clients/java/base/pom.xml
@@ -36,6 +36,11 @@
<artifactId>stream-storage-proto</artifactId>
<version>${project.parent.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.inferred</groupId>
+ <artifactId>freebuilder</artifactId>
+ <optional>true</optional>
+ </dependency>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>testtools</artifactId>
@@ -45,6 +50,19 @@
</dependencies>
<build>
<plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <configuration>
+ <annotationProcessorPaths combine.children="append">
+ <path>
+ <groupId>org.inferred</groupId>
+ <artifactId>freebuilder</artifactId>
+ <version>${freebuilder.version}</version>
+ </path>
+ </annotationProcessorPaths>
+ </configuration>
+ </plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
diff --git a/stream/clients/java/base/src/test/java/org/apache/bookkeeper/clients/impl/internal/TestStorageServerClientManagerImpl.java b/stream/clients/java/base/src/test/java/org/apache/bookkeeper/clients/impl/internal/TestStorageServerClientManagerImpl.java
index 5f68d11c5c..d72a69e431 100644
--- a/stream/clients/java/base/src/test/java/org/apache/bookkeeper/clients/impl/internal/TestStorageServerClientManagerImpl.java
+++ b/stream/clients/java/base/src/test/java/org/apache/bookkeeper/clients/impl/internal/TestStorageServerClientManagerImpl.java
@@ -22,6 +22,7 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
+import com.google.common.collect.Lists;
import io.grpc.stub.StreamObserver;
import java.util.List;
import org.apache.bookkeeper.clients.grpc.GrpcClientTestBase;
@@ -36,7 +37,6 @@ import org.apache.bookkeeper.stream.proto.storage.GetStreamResponse;
import org.apache.bookkeeper.stream.proto.storage.OneStorageContainerEndpointResponse;
import org.apache.bookkeeper.stream.proto.storage.RootRangeServiceGrpc.RootRangeServiceImplBase;
import org.apache.bookkeeper.stream.proto.storage.StatusCode;
-import org.inferred.freebuilder.shaded.com.google.common.collect.Lists;
import org.junit.Test;
/**
diff --git a/stream/clients/python/docker/Dockerfile b/stream/clients/python/docker/Dockerfile
index 7b3d7073a8..2bdb91fda6 100644
--- a/stream/clients/python/docker/Dockerfile
+++ b/stream/clients/python/docker/Dockerfile
@@ -17,22 +17,27 @@
# under the License.
#
-FROM ubuntu:22.10
+FROM ubuntu:22.04
MAINTAINER Apache BookKeeper <de...@bookkeeper.apache.org>
ARG BK_TAR_BALL
-ARG TARGETARCH=amd64
ENV BOOKIE_PORT=3181
ENV BOOKIE_HTTP_PORT=8080
EXPOSE $BOOKIE_PORT $BOOKIE_HTTP_PORT
ENV BK_USER=bookkeeper
ENV BK_HOME=/opt/bookkeeper
-ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-$TARGETARCH
+ENV DEBIAN_FRONTEND=noninteractive
+ARG UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/
+ARG UBUNTU_SECURITY_MIRROR=http://security.ubuntu.com/ubuntu/
RUN set -x \
+ && sed -i -e "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}|g" \
+ -e "s|http://security\.ubuntu\.com/ubuntu/|${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}|g" /etc/apt/sources.list \
+ && echo 'Acquire::http::Timeout "30";\nAcquire::http::ConnectionAttemptDelayMsec "2000";\nAcquire::https::Timeout "30";\nAcquire::https::ConnectionAttemptDelayMsec "2000";\nAcquire::ftp::Timeout "30";\nAcquire::ftp::ConnectionAttemptDelayMsec "2000";\nAcquire::Retries "15";' > /etc/apt/apt.conf.d/99timeout_and_retries \
&& adduser "${BK_USER}" \
&& apt-get update \
+ && apt-get install -y ca-certificates apt-transport-https \
&& apt-get install -y --no-install-recommends openjdk-17-jdk \
&& apt-get install -y --no-install-recommends python3 pip \
&& ln -s /usr/bin/python3 /usr/bin/python \
@@ -41,7 +46,10 @@ RUN set -x \
&& apt-get autoclean \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
- && pip install zk-shell
+ && pip install zk-shell \
+ && JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))) \
+ && echo networkaddress.cache.ttl=1 >> $JAVA_HOME/conf/security/java.security \
+ && echo networkaddress.cache.negative.ttl=1 >> $JAVA_HOME/conf/security/java.security
RUN mkdir -pv /opt \
&& cd /opt
diff --git a/stream/clients/python/docker/build-local-image.sh b/stream/clients/python/docker/build-local-image.sh
index 3958bb253c..b6fee88d8b 100755
--- a/stream/clients/python/docker/build-local-image.sh
+++ b/stream/clients/python/docker/build-local-image.sh
@@ -28,5 +28,6 @@ fi
cp ${ROOT_DIR}/bookkeeper-dist/server/target/bookkeeper-server*-bin.tar.gz "${CURRENT_DIR}/${BK_TAR_BALL}"
pushd "${CURRENT_DIR}"
-docker build --build-arg BK_TAR_BALL=${BK_TAR_BALL} -t apache/bookkeeper:local .
+docker build --build-arg UBUNTU_MIRROR="${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}" --build-arg UBUNTU_SECURITY_MIRROR="${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}" \
+ --build-arg BK_TAR_BALL=${BK_TAR_BALL} -t apache/bookkeeper:local .
popd
diff --git a/stream/distributedlog/common/pom.xml b/stream/distributedlog/common/pom.xml
index 44cc6de0f0..36d964a2c6 100644
--- a/stream/distributedlog/common/pom.xml
+++ b/stream/distributedlog/common/pom.xml
@@ -34,6 +34,18 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-reload4j</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>log4j</groupId>
+ <artifactId>log4j</artifactId>
+ </exclusion>
+ <exclusion>
+ <groupId>ch.qos.reload4j</groupId>
+ <artifactId>reload4j</artifactId>
+ </exclusion>
</exclusions>
</dependency>
<dependency>
@@ -78,8 +90,8 @@
<artifactId>spotbugs-maven-plugin</artifactId>
</plugin>
<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
@@ -96,11 +108,11 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<redirectTestOutputToFile>${redirectTestOutputToFile}</redirectTestOutputToFile>
<argLine>-Xmx3G -Djava.net.preferIPv4Stack=true -XX:MaxDirectMemorySize=2G ${test.additional.args}</argLine>
- <forkMode>always</forkMode>
+ <forkCount>1</forkCount>
+ <reuseForks>false</reuseForks>
<forkedProcessTimeoutInSeconds>1800</forkedProcessTimeoutInSeconds>
</configuration>
</plugin>
diff --git a/stream/distributedlog/core/pom.xml b/stream/distributedlog/core/pom.xml
index e6fcf27d3c..a60acc09f7 100644
--- a/stream/distributedlog/core/pom.xml
+++ b/stream/distributedlog/core/pom.xml
@@ -90,8 +90,8 @@
<build>
<plugins>
<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
@@ -108,12 +108,12 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<trimStackTrace>false</trimStackTrace>
<redirectTestOutputToFile>${redirectTestOutputToFile}</redirectTestOutputToFile>
<argLine>-Xmx3G -Djava.net.preferIPv4Stack=true -XX:MaxDirectMemorySize=2G ${test.additional.args}</argLine>
- <forkMode>always</forkMode>
+ <forkCount>1</forkCount>
+ <reuseForks>false</reuseForks>
<forkedProcessTimeoutInSeconds>1800</forkedProcessTimeoutInSeconds>
<properties>
<property>
diff --git a/stream/distributedlog/io/dlfs/pom.xml b/stream/distributedlog/io/dlfs/pom.xml
index b9a27004f3..00aadf247b 100644
--- a/stream/distributedlog/io/dlfs/pom.xml
+++ b/stream/distributedlog/io/dlfs/pom.xml
@@ -54,10 +54,18 @@
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-reload4j</artifactId>
+ </exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
+ <exclusion>
+ <groupId>ch.qos.reload4j</groupId>
+ <artifactId>reload4j</artifactId>
+ </exclusion>
<exclusion>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
diff --git a/stream/distributedlog/pom.xml b/stream/distributedlog/pom.xml
index cda07db55c..944e421574 100644
--- a/stream/distributedlog/pom.xml
+++ b/stream/distributedlog/pom.xml
@@ -45,7 +45,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
- <version>${maven-javadoc-plugin.version}</version>
<configuration>
<sourcepath>${src.dir}</sourcepath>
<notimestamp>true</notimestamp>
@@ -74,11 +73,11 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<redirectTestOutputToFile>${redirectTestOutputToFile}</redirectTestOutputToFile>
<argLine>-Xmx3G -Djava.net.preferIPv4Stack=true -XX:MaxDirectMemorySize=2G -Dio.netty.leakDetection.level=PARANOID ${test.additional.args}</argLine>
- <forkMode>always</forkMode>
+ <forkCount>1</forkCount>
+ <reuseForks>false</reuseForks>
<forkedProcessTimeoutInSeconds>1800</forkedProcessTimeoutInSeconds>
</configuration>
</plugin>
diff --git a/stream/pom.xml b/stream/pom.xml
index 13547299c0..f7336072ab 100644
--- a/stream/pom.xml
+++ b/stream/pom.xml
@@ -53,13 +53,13 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- only run tests when -DstreamTests is specified //-->
<skipTests>true</skipTests>
<redirectTestOutputToFile>${redirectTestOutputToFile}</redirectTestOutputToFile>
<argLine>-Xmx3G -Djava.net.preferIPv4Stack=true -XX:MaxDirectMemorySize=2G -Dio.netty.leakDetection.level=PARANOID ${test.additional.args}</argLine>
- <forkMode>always</forkMode>
+ <forkCount>1</forkCount>
+ <reuseForks>false</reuseForks>
<forkedProcessTimeoutInSeconds>1800</forkedProcessTimeoutInSeconds>
</configuration>
</plugin>
diff --git a/stream/proto/pom.xml b/stream/proto/pom.xml
index bec7f7c6f7..3f7b621db5 100644
--- a/stream/proto/pom.xml
+++ b/stream/proto/pom.xml
@@ -71,16 +71,9 @@
</extensions>
<plugins>
<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
<configuration>
- <source>${javac.target}</source>
- <target>${javac.target}</target>
- <compilerArgs>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
- </compilerArgs>
<showDeprecation>false</showDeprecation>
<showWarnings>false</showWarnings>
</configuration>
diff --git a/stream/tests-common/pom.xml b/stream/tests-common/pom.xml
index 7ea60b63e7..5d6831e7ba 100644
--- a/stream/tests-common/pom.xml
+++ b/stream/tests-common/pom.xml
@@ -76,16 +76,9 @@
</extensions>
<plugins>
<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
<configuration>
- <source>${javac.target}</source>
- <target>${javac.target}</target>
- <compilerArgs>
- <compilerArg>-Xlint:unchecked</compilerArg>
- <!-- https://issues.apache.org/jira/browse/MCOMPILER-205 -->
- <compilerArg>-Xpkginfo:always</compilerArg>
- </compilerArgs>
<showDeprecation>false</showDeprecation>
<showWarnings>false</showWarnings>
</configuration>
diff --git a/tests/README.md b/tests/README.md
new file mode 100644
index 0000000000..4d79d4b6d3
--- /dev/null
+++ b/tests/README.md
@@ -0,0 +1,81 @@
+# Integration tests for Bookkeeper
+
+## Prerequisites
+
+### Workaround for running the tests with Mac Apple Silicon
+
+The current version of the outdated maven plugin requires a workaround.
+
+Install socat
+```bash
+brew install socat
+```
+
+Run a TCP -> Unix socket proxy for docker in a separate terminal
+```bash
+socat TCP-LISTEN:2375,bind=127.0.0.1,reuseaddr,fork UNIX-CLIENT:/var/run/docker.sock &
+```
+
+Set the `DOCKER_HOST` environment variable in the terminal where you run the tests
+```bash
+export DOCKER_HOST=tcp://localhost:2375
+```
+
+Here's a shell script function to automate starting the proxy and setting `DOCKER_HOST`:
+```bash
+function docker_socket_proxy() {
+ local port=2375
+ socat /dev/null TCP4:127.0.0.1:$port,connect-timeout=2 &> /dev/null
+ if [ $? -ne 0 ]; then
+ echo "Starting socat tcp proxy on port $port for docker socket /var/run/docker.sock"
+ socat TCP-LISTEN:$port,bind=127.0.0.1,reuseaddr,fork UNIX-CLIENT:/var/run/docker.sock &> /dev/null &
+ echo "Stop the proxy with 'kill $!'"
+ fi
+ export DOCKER_HOST=tcp://127.0.0.1:$port
+ echo "Added DOCKER_HOST=$DOCKER_HOST to environment"
+}
+```
+You can add the function to your shell profile (`~/.bashrc`, `~/.zshrc`, etc.).
+
+### Support for connecting to docker network from Mac host
+
+You will also need to install [docker-mac-net-connect](https://github.com/chipmk/docker-mac-net-connect). It allows the tests to connect to the docker network from the Mac host.
+
+```bash
+brew install chipmk/tap/docker-mac-net-connect
+sudo brew services start chipmk/tap/docker-mac-net-connect
+```
+
+## Running the tests
+
+Remember to start the unix socket proxy for docker as described in the previous section and set the `DOCKER_HOST` environment variable in the terminal where you run the tests.
+
+### Building the docker images together with the project source code
+
+```bash
+mvn -B -nsu clean install -Pdocker -DskipTests
+docker images | grep apachebookkeeper
+```
+
+### Running integration tests
+
+```bash
+# Run metadata driver tests
+mvn -f metadata-drivers/pom.xml test -DintegrationTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO -DredirectTestOutputToFile=false -DtestRetryCount=0
+
+# Run all integration tests
+mvn -f tests/pom.xml test -DintegrationTests -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO -DredirectTestOutputToFile=false -DtestRetryCount=0
+```
+
+### Running backward compatibility tests
+
+```bash
+# Test current server with old clients
+mvn -DbackwardCompatTests -pl :backward-compat-current-server-old-clients test -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO -DredirectTestOutputToFile=false -DtestRetryCount=0
+
+# Test progressive upgrade
+mvn -DbackwardCompatTests -pl :upgrade test -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO -DredirectTestOutputToFile=false -DtestRetryCount=0
+
+# Other backward compat tests
+mvn -DbackwardCompatTests -pl :bc-non-fips,:hierarchical-ledger-manager,:hostname-bookieid,:old-cookie-new-cluster,:recovery-no-password,:upgrade-direct,:yahoo-custom-version test -Dorg.slf4j.simpleLogger.defaultLogLevel=INFO -DredirectTestOutputToFile=false -DtestRetryCount=0
+```
\ No newline at end of file
diff --git a/tests/backward-compat/pom.xml b/tests/backward-compat/pom.xml
index d189dfd09b..f10d40b072 100644
--- a/tests/backward-compat/pom.xml
+++ b/tests/backward-compat/pom.xml
@@ -38,6 +38,33 @@
<module>yahoo-custom-version</module>
<module>bc-non-fips</module>
</modules>
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>com.github.docker-java</groupId>
+ <artifactId>docker-java-bom</artifactId>
+ <version>${arquillian-cube.docker-java.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.arquillian.cube</groupId>
+ <artifactId>arquillian-cube-docker</artifactId>
+ <version>${arquillian-cube.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.yaml</groupId>
+ <artifactId>snakeyaml</artifactId>
+ <version>${arquillian-cube.snakeyaml.version}</version>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>org.arquillian.cube</groupId>
+ <artifactId>arquillian-cube-docker</artifactId>
+ </dependency>
+ </dependencies>
<build>
<plugins>
<plugin>
@@ -47,6 +74,34 @@
<skip>true</skip>
</configuration>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <skipTests>true</skipTests>
+ </configuration>
+ </plugin>
</plugins>
</build>
+ <profiles>
+ <profile>
+ <id>backwardCompatTests</id>
+ <activation>
+ <property>
+ <name>backwardCompatTests</name>
+ </property>
+ </activation>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <skipTests>false</skipTests>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ </profiles>
</project>
diff --git a/tests/backward-compat/upgrade-direct/src/test/groovy/org/apache/bookkeeper/tests/backwardcompat/TestCompatUpgradeDirect.groovy b/tests/backward-compat/upgrade-direct/src/test/groovy/org/apache/bookkeeper/tests/backwardcompat/TestCompatUpgradeDirect.groovy
index 2ee1c7d8aa..448c0f83b8 100644
--- a/tests/backward-compat/upgrade-direct/src/test/groovy/org/apache/bookkeeper/tests/backwardcompat/TestCompatUpgradeDirect.groovy
+++ b/tests/backward-compat/upgrade-direct/src/test/groovy/org/apache/bookkeeper/tests/backwardcompat/TestCompatUpgradeDirect.groovy
@@ -36,6 +36,12 @@ class TestCompatUpgradeDirect {
private static final Logger LOG = LoggerFactory.getLogger(TestCompatUpgradeDirect.class)
private static byte[] PASSWD = "foobar".getBytes()
+ static {
+ Thread.setDefaultUncaughtExceptionHandler { Thread t, Throwable e ->
+ LOG.error("Uncaught exception in thread {}", t, e)
+ }
+ }
+
@ArquillianResource
DockerClient docker
diff --git a/tests/docker-images/all-released-versions-image/Dockerfile b/tests/docker-images/all-released-versions-image/Dockerfile
index aee3227ecd..a8c761c7e8 100644
--- a/tests/docker-images/all-released-versions-image/Dockerfile
+++ b/tests/docker-images/all-released-versions-image/Dockerfile
@@ -17,16 +17,27 @@
# under the License.
#
-FROM openjdk:8-jdk
+FROM eclipse-temurin:8-jdk
MAINTAINER Apache BookKeeper <de...@bookkeeper.apache.org>
ENV BK_JOURNALDIR=/opt/bookkeeper/data/journal
ENV BK_LEDGERDIR=/opt/bookkeeper/data/ledgers
ENV BK_ZKCONNECTSTRING=zookeeper1,zookeeper2,zookeeper3
+ENV DEBIAN_FRONTEND=noninteractive
+ARG UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/
+ARG UBUNTU_SECURITY_MIRROR=http://security.ubuntu.com/ubuntu/
-RUN apt-get update && apt-get install -qy wget supervisor bash \
+RUN sed -i -e "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}|g" \
+ -e "s|http://security\.ubuntu\.com/ubuntu/|${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}|g" /etc/apt/sources.list \
+ && echo 'Acquire::http::Timeout "30";\nAcquire::http::ConnectionAttemptDelayMsec "2000";\nAcquire::https::Timeout "30";\nAcquire::https::ConnectionAttemptDelayMsec "2000";\nAcquire::ftp::Timeout "30";\nAcquire::ftp::ConnectionAttemptDelayMsec "2000";\nAcquire::Retries "15";' > /etc/apt/apt.conf.d/99timeout_and_retries \
+ && apt-get update && apt-get install -qy wget curl supervisor bash ca-certificates apt-transport-https \
+ && apt-get -y install netcat dnsutils less procps iputils-ping \
+ && apt-get install -y --no-install-recommends gpg gpg-agent sudo \
&& echo "dash dash/sh boolean false" | debconf-set-selections \
- && DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+ && DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash \
+ && JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))) \
+ && echo networkaddress.cache.ttl=1 >> $JAVA_HOME/jre/lib/security/java.security \
+ && echo networkaddress.cache.negative.ttl=1 >> $JAVA_HOME/jre/lib/security/java.security
RUN mkdir /tarballs
WORKDIR /tarballs
diff --git a/tests/docker-images/all-released-versions-image/image_builder.sh b/tests/docker-images/all-released-versions-image/image_builder.sh
index a06c29e49b..b89afad3e6 100755
--- a/tests/docker-images/all-released-versions-image/image_builder.sh
+++ b/tests/docker-images/all-released-versions-image/image_builder.sh
@@ -29,4 +29,5 @@ SCRIPT_DIR=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P )
## BASE_DIR will be ./bookkeeper/
BASE_DIR=${SCRIPT_DIR}/../../../
-docker build -t ${IMAGE_NAME} "${BASE_DIR}"/tests/docker-images/all-released-versions-image
\ No newline at end of file
+docker build --build-arg UBUNTU_MIRROR="${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}" --build-arg UBUNTU_SECURITY_MIRROR="${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}" \
+ -t ${IMAGE_NAME} "${BASE_DIR}"/tests/docker-images/all-released-versions-image
\ No newline at end of file
diff --git a/tests/docker-images/all-released-versions-image/pom.xml b/tests/docker-images/all-released-versions-image/pom.xml
index 7ba654658e..1ffb55b02d 100644
--- a/tests/docker-images/all-released-versions-image/pom.xml
+++ b/tests/docker-images/all-released-versions-image/pom.xml
@@ -62,6 +62,10 @@
<repository>apachebookkeeper/bookkeeper-all-released-versions</repository>
<tag>${project.version}</tag>
<pullNewerImage>false</pullNewerImage>
+ <buildArgs>
+ <UBUNTU_MIRROR>${UBUNTU_MIRROR}</UBUNTU_MIRROR>
+ <UBUNTU_SECURITY_MIRROR>${UBUNTU_SECURITY_MIRROR}</UBUNTU_SECURITY_MIRROR>
+ </buildArgs>
</configuration>
</plugin>
</plugins>
diff --git a/tests/docker-images/all-versions-image/image_builder.sh b/tests/docker-images/all-versions-image/image_builder.sh
index 7b55336230..f021c5f034 100755
--- a/tests/docker-images/all-versions-image/image_builder.sh
+++ b/tests/docker-images/all-versions-image/image_builder.sh
@@ -34,4 +34,5 @@ mkdir -p "${BASE_DIR}"/tests/docker-images/all-versions-image/build
ls -la ${BASE_DIR}bookkeeper-dist/server/build/distributions
cp ${BASE_DIR}bookkeeper-dist/server/build/distributions/bookkeeper-server-${VERSION}-bin.tar.gz "${BASE_DIR}"/tests/docker-images/all-versions-image/build/
TARBALL=build/bookkeeper-server-${VERSION}-bin.tar.gz
-docker build -t ${IMAGE_NAME} "${BASE_DIR}"tests/docker-images/all-versions-image --build-arg BK_TARBALL="${TARBALL}"
\ No newline at end of file
+docker build --build-arg UBUNTU_MIRROR="${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}" --build-arg UBUNTU_SECURITY_MIRROR="${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}" \
+ -t ${IMAGE_NAME} "${BASE_DIR}"tests/docker-images/all-versions-image --build-arg BK_TARBALL="${TARBALL}"
\ No newline at end of file
diff --git a/tests/docker-images/all-versions-image/pom.xml b/tests/docker-images/all-versions-image/pom.xml
index d62c40f452..4df9f761ea 100644
--- a/tests/docker-images/all-versions-image/pom.xml
+++ b/tests/docker-images/all-versions-image/pom.xml
@@ -82,6 +82,8 @@
<noCache>true</noCache>
<buildArgs>
<BK_TARBALL>target/bookkeeper-dist-server-${project.version}-bin.tar.gz</BK_TARBALL>
+ <UBUNTU_MIRROR>${UBUNTU_MIRROR}</UBUNTU_MIRROR>
+ <UBUNTU_SECURITY_MIRROR>${UBUNTU_SECURITY_MIRROR}</UBUNTU_SECURITY_MIRROR>
</buildArgs>
</configuration>
</plugin>
diff --git a/tests/docker-images/statestore-image/Dockerfile b/tests/docker-images/current-version-image/Dockerfile
similarity index 50%
copy from tests/docker-images/statestore-image/Dockerfile
copy to tests/docker-images/current-version-image/Dockerfile
index e89175be81..0e489eb5ea 100644
--- a/tests/docker-images/statestore-image/Dockerfile
+++ b/tests/docker-images/current-version-image/Dockerfile
@@ -17,63 +17,59 @@
# under the License.
#
-FROM ubuntu:22.10
+FROM ubuntu:22.04
MAINTAINER Apache BookKeeper <de...@bookkeeper.apache.org>
-ARG TARGETARCH
+ARG BK_VERSION=DOESNOTEXISTS
+ARG DISTRO_NAME=bookkeeper-dist-server-${BK_VERSION}-bin
+ARG PKG_NAME=bookkeeper-server-${BK_VERSION}
-#ENV OPTS="$OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
ENV BOOKIE_PORT=3181
ENV BOOKIE_HTTP_PORT=8080
ENV BOOKIE_GRPC_PORT=4181
EXPOSE ${BOOKIE_PORT} ${BOOKIE_HTTP_PORT} ${BOOKIE_GRPC_PORT}
ENV BK_USER=bookkeeper
ENV BK_HOME=/opt/bookkeeper
-ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-$TARGETARCH
ENV DEBIAN_FRONTEND=noninteractive
-
+ARG UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/
+ARG UBUNTU_SECURITY_MIRROR=http://security.ubuntu.com/ubuntu/
RUN set -x \
+ && sed -i -e "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}|g" \
+ -e "s|http://security\.ubuntu\.com/ubuntu/|${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}|g" /etc/apt/sources.list \
+ && echo 'Acquire::http::Timeout "30";\nAcquire::http::ConnectionAttemptDelayMsec "2000";\nAcquire::https::Timeout "30";\nAcquire::https::ConnectionAttemptDelayMsec "2000";\nAcquire::ftp::Timeout "30";\nAcquire::ftp::ConnectionAttemptDelayMsec "2000";\nAcquire::Retries "15";' > /etc/apt/apt.conf.d/99timeout_and_retries \
&& adduser "${BK_USER}" \
&& apt-get update \
+ && apt-get install -y ca-certificates apt-transport-https \
&& apt-get install -y --no-install-recommends openjdk-17-jdk \
&& apt-get install -y --no-install-recommends python3 pip \
&& ln -s /usr/bin/python3 /usr/bin/python \
&& apt-get install -y --no-install-recommends gpg gpg-agent wget sudo \
+ && apt-get -y install netcat dnsutils less procps iputils-ping \
&& apt-get -y --purge autoremove \
&& apt-get autoclean \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& pip install zk-shell \
- && mkdir -pv /opt \
- && cd /opt
-
-WORKDIR /opt/bookkeeper
-RUN mkdir /opt/bookkeeper/lib
-RUN mkdir /opt/bookkeeper/bin
-RUN mkdir /opt/bookkeeper/conf
-RUN mkdir /opt/bookkeeper/scripts
+ && JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))) \
+ && echo networkaddress.cache.ttl=1 >> $JAVA_HOME/conf/security/java.security \
+ && echo networkaddress.cache.negative.ttl=1 >> $JAVA_HOME/conf/security/java.security
-### -----Copy Jars------###
-ADD ./dist/server-bin.tar.gz /opt/
-RUN mv /opt/server-bin/lib/*.jar /opt/bookkeeper/lib/
-### --------------------###
+# untar tarballs
+ADD target/${DISTRO_NAME}.tar.gz /opt
+RUN mv /opt/${PKG_NAME} /opt/bookkeeper
-### ----Copy scripts----------###
-COPY ./scripts/* /opt/bookkeeper/scripts/
-COPY ./temp_bin/* /opt/bookkeeper/bin/
-COPY ./bin/* /opt/bookkeeper/bin/
-### ----Copy scripts----------###
+WORKDIR /opt/bookkeeper
-### ----Copy conf-------------###
-COPY ./temp_conf/* /opt/bookkeeper/conf/
-COPY ./conf/* /opt/bookkeeper/conf/
-### ----Copy conf-------------###
+COPY target/scripts /opt/bookkeeper/scripts
+COPY scripts/install-python-client.sh /opt/bookkeeper/scripts
+RUN chmod +x -R /opt/bookkeeper/scripts/
-RUN chmod +x -R /opt/bookkeeper/scripts
-RUN chmod +x -R /opt/bookkeeper/bin
+# copy the python client
+ADD target/bookkeeper-client/ /opt/bookkeeper/bookkeeper-client
+RUN /opt/bookkeeper/scripts/install-python-client.sh
ENTRYPOINT [ "/bin/bash", "/opt/bookkeeper/scripts/entrypoint.sh" ]
CMD ["bookie"]
-#HEALTHCHECK --interval=10s --timeout=50s CMD /bin/bash /opt/bookkeeper/scripts/healthcheck.sh
+#HEALTHCHECK --interval=10s --timeout=60s CMD /bin/bash /opt/bookkeeper/scripts/healthcheck.sh
diff --git a/tests/docker-images/all-versions-image/pom.xml b/tests/docker-images/current-version-image/pom.xml
similarity index 58%
copy from tests/docker-images/all-versions-image/pom.xml
copy to tests/docker-images/current-version-image/pom.xml
index d62c40f452..905ec47ebe 100644
--- a/tests/docker-images/all-versions-image/pom.xml
+++ b/tests/docker-images/current-version-image/pom.xml
@@ -23,17 +23,10 @@
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.apache.bookkeeper.tests</groupId>
- <artifactId>all-versions-image</artifactId>
- <name>Apache BookKeeper :: Tests :: Docker Images :: All Versions</name>
+ <artifactId>current-version-image</artifactId>
+ <name>Apache BookKeeper :: Tests :: Docker Images :: Current Version</name>
<packaging>pom</packaging>
<dependencies>
- <dependency>
- <groupId>org.apache.bookkeeper.tests</groupId>
- <artifactId>all-released-versions-image</artifactId>
- <version>${project.parent.version}</version>
- <type>pom</type>
- </dependency>
-
<dependency>
<groupId>org.apache.bookkeeper</groupId>
<artifactId>bookkeeper-dist-server</artifactId>
@@ -53,6 +46,49 @@
</activation>
<build>
<plugins>
+ <!-- build cpp client, copy the wheel file and then build docker image -->
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>exec-maven-plugin</artifactId>
+ <version>${exec-maven-plugin.version}</version>
+ <executions>
+ <execution>
+ <id>build-python-client</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>exec</goal>
+ </goals>
+ <configuration>
+ <workingDirectory>${project.basedir}/target</workingDirectory>
+ <executable>${project.basedir}/../../../stream/clients/python/scripts/docker_build.sh</executable>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <!-- this task is used for copy docker scripts & python wheel file to build docker image -->
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <version>${maven-antrun-plugin.version}</version>
+ <executions>
+ <execution>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ <configuration>
+ <tasks>
+ <echo>copy python wheel file</echo>
+ <mkdir dir="${project.basedir}/target/bookkeeper-client" />
+ <copydir src="${project.basedir}/../../../stream/clients/python/dist" dest="${project.basedir}/target/bookkeeper-client" />
+ <echo>copying docker scripts</echo>
+ <mkdir dir="${project.basedir}/target/scripts" />
+ <copydir src="${project.basedir}/../../../docker/scripts" dest="${project.basedir}/target/scripts" />
+ </tasks>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
<plugin>
<groupId>com.spotify</groupId>
<artifactId>dockerfile-maven-plugin</artifactId>
@@ -70,18 +106,20 @@
<goal>tag</goal>
</goals>
<configuration>
- <repository>apachebookkeeper/bookkeeper-all-versions</repository>
+ <repository>apachebookkeeper/bookkeeper-current</repository>
<tag>latest</tag>
</configuration>
</execution>
</executions>
<configuration>
- <repository>apachebookkeeper/bookkeeper-all-versions</repository>
+ <repository>apachebookkeeper/bookkeeper-current</repository>
<tag>${project.version}</tag>
<pullNewerImage>false</pullNewerImage>
<noCache>true</noCache>
<buildArgs>
- <BK_TARBALL>target/bookkeeper-dist-server-${project.version}-bin.tar.gz</BK_TARBALL>
+ <BK_VERSION>${project.version}</BK_VERSION>
+ <UBUNTU_MIRROR>${UBUNTU_MIRROR}</UBUNTU_MIRROR>
+ <UBUNTU_SECURITY_MIRROR>${UBUNTU_SECURITY_MIRROR}</UBUNTU_SECURITY_MIRROR>
</buildArgs>
</configuration>
</plugin>
@@ -91,7 +129,7 @@
<version>${maven-dependency-plugin.version}</version>
<executions>
<execution>
- <id>copy-tarball</id>
+ <id>copy-docker-dependencies</id>
<goals>
<goal>copy-dependencies</goal>
</goals>
diff --git a/.github/actions/tune-runner-vm/action.yml b/tests/docker-images/current-version-image/scripts/install-python-client.sh
similarity index 59%
copy from .github/actions/tune-runner-vm/action.yml
copy to tests/docker-images/current-version-image/scripts/install-python-client.sh
index 59f977c315..2719b335ac 100644
--- a/.github/actions/tune-runner-vm/action.yml
+++ b/tests/docker-images/current-version-image/scripts/install-python-client.sh
@@ -1,3 +1,4 @@
+#!/usr/bin/env bash
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
@@ -17,15 +18,7 @@
# under the License.
#
-name: Tune Runner VM performance
-description: tunes the GitHub Runner VM operation system
-runs:
- using: composite
- steps:
- - run: |
- if [[ "$OSTYPE" == "linux-gnu"* ]]; then
- # Ensure that reverse lookups for current hostname are handled properly
- # Add the current IP address, long hostname and short hostname record to /etc/hosts file
- echo -e "$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)\t$(hostname -f) $(hostname -s)" | sudo tee -a /etc/hosts
- fi
- shell: bash
\ No newline at end of file
+set -x
+
+WHEEL_FILE=`ls /opt/bookkeeper/bookkeeper-client/*.whl`
+pip install ${WHEEL_FILE}
diff --git a/tests/docker-images/pom.xml b/tests/docker-images/pom.xml
index 6482394774..1891b8e84f 100644
--- a/tests/docker-images/pom.xml
+++ b/tests/docker-images/pom.xml
@@ -29,5 +29,6 @@
<modules>
<module>all-released-versions-image</module>
<module>all-versions-image</module>
+ <module>current-version-image</module>
</modules>
</project>
diff --git a/tests/docker-images/statestore-image/Dockerfile b/tests/docker-images/statestore-image/Dockerfile
index e89175be81..5a7a168885 100644
--- a/tests/docker-images/statestore-image/Dockerfile
+++ b/tests/docker-images/statestore-image/Dockerfile
@@ -17,28 +17,30 @@
# under the License.
#
-FROM ubuntu:22.10
+FROM ubuntu:22.04
MAINTAINER Apache BookKeeper <de...@bookkeeper.apache.org>
-ARG TARGETARCH
-
-#ENV OPTS="$OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
ENV BOOKIE_PORT=3181
ENV BOOKIE_HTTP_PORT=8080
ENV BOOKIE_GRPC_PORT=4181
EXPOSE ${BOOKIE_PORT} ${BOOKIE_HTTP_PORT} ${BOOKIE_GRPC_PORT}
ENV BK_USER=bookkeeper
ENV BK_HOME=/opt/bookkeeper
-ENV JAVA_HOME /usr/lib/jvm/java-17-openjdk-$TARGETARCH
ENV DEBIAN_FRONTEND=noninteractive
-
+ARG UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/
+ARG UBUNTU_SECURITY_MIRROR=http://security.ubuntu.com/ubuntu/
RUN set -x \
+ && sed -i -e "s|http://archive\.ubuntu\.com/ubuntu/|${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}|g" \
+ -e "s|http://security\.ubuntu\.com/ubuntu/|${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}|g" /etc/apt/sources.list \
+ && echo 'Acquire::http::Timeout "30";\nAcquire::http::ConnectionAttemptDelayMsec "2000";\nAcquire::https::Timeout "30";\nAcquire::https::ConnectionAttemptDelayMsec "2000";\nAcquire::ftp::Timeout "30";\nAcquire::ftp::ConnectionAttemptDelayMsec "2000";\nAcquire::Retries "15";' > /etc/apt/apt.conf.d/99timeout_and_retries \
&& adduser "${BK_USER}" \
&& apt-get update \
+ && apt-get install -y ca-certificates apt-transport-https \
&& apt-get install -y --no-install-recommends openjdk-17-jdk \
&& apt-get install -y --no-install-recommends python3 pip \
&& ln -s /usr/bin/python3 /usr/bin/python \
+ && apt-get -y install netcat dnsutils less procps iputils-ping \
&& apt-get install -y --no-install-recommends gpg gpg-agent wget sudo \
&& apt-get -y --purge autoremove \
&& apt-get autoclean \
@@ -46,7 +48,9 @@ RUN set -x \
&& rm -rf /var/lib/apt/lists/* \
&& pip install zk-shell \
&& mkdir -pv /opt \
- && cd /opt
+ && JAVA_HOME=$(dirname $(dirname $(readlink -f $(which java)))) \
+ && echo networkaddress.cache.ttl=1 >> $JAVA_HOME/conf/security/java.security \
+ && echo networkaddress.cache.negative.ttl=1 >> $JAVA_HOME/conf/security/java.security
WORKDIR /opt/bookkeeper
RUN mkdir /opt/bookkeeper/lib
diff --git a/tests/docker-images/statestore-image/image_builder.sh b/tests/docker-images/statestore-image/image_builder.sh
index e17ee608b7..8f305d0ec1 100755
--- a/tests/docker-images/statestore-image/image_builder.sh
+++ b/tests/docker-images/statestore-image/image_builder.sh
@@ -42,7 +42,8 @@ cp "${BASE_DIR}"/stream/server/build/distributions/server-bin.tar.gz "${BASE_DIR
cp "${BASE_DIR}"/docker/scripts/* "${BASE_DIR}"/tests/docker-images/statestore-image/scripts
cp "${BASE_DIR}"/conf/* "${BASE_DIR}"/tests/docker-images/statestore-image/temp_conf
cp "${BASE_DIR}"/bin/* "${BASE_DIR}"/tests/docker-images/statestore-image/temp_bin
-docker build -t ${IMAGE_NAME} "${BASE_DIR}"/tests/docker-images/statestore-image
+docker build --build-arg UBUNTU_MIRROR="${UBUNTU_MIRROR:-http://archive.ubuntu.com/ubuntu/}" --build-arg UBUNTU_SECURITY_MIRROR="${UBUNTU_SECURITY_MIRROR:-http://security.ubuntu.com/ubuntu/}" \
+ -t ${IMAGE_NAME} "${BASE_DIR}"/tests/docker-images/statestore-image
rm -rf "${BASE_DIR}"/tests/docker-images/statestore-image/dist
rm -rf "${BASE_DIR}"/tests/docker-images/statestore-image/scripts
diff --git a/tests/integration-tests-base-groovy/pom.xml b/tests/integration-tests-base-groovy/pom.xml
index 6d28fbe506..10b410c363 100644
--- a/tests/integration-tests-base-groovy/pom.xml
+++ b/tests/integration-tests-base-groovy/pom.xml
@@ -38,12 +38,12 @@
<build>
<plugins>
<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration combine.self="override">
<!-- combine.self="override" stops compilerArgs from parent pom being merged //-->
- <source>${javac.target}</source>
- <target>${javac.target}</target>
<compilerId>groovy-eclipse-compiler</compilerId>
+ <fork>true</fork>
</configuration>
<dependencies>
<dependency>
@@ -64,18 +64,6 @@
<version>${groovy-eclipse-compiler.version}</version>
<extensions>true</extensions>
</plugin>
- <plugin>
- <groupId>org.codehaus.gmavenplus</groupId>
- <artifactId>gmavenplus-plugin</artifactId>
- <version>${gmavenplus-plugin.version}</version>
- <executions>
- <execution>
- <goals>
- <goal>compileTests</goal>
- </goals>
- </execution>
- </executions>
- </plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
@@ -93,6 +81,27 @@
</plugins>
</build>
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>com.github.docker-java</groupId>
+ <artifactId>docker-java-bom</artifactId>
+ <version>${arquillian-cube.docker-java.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.arquillian.cube</groupId>
+ <artifactId>arquillian-cube-docker</artifactId>
+ <version>${arquillian-cube.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.yaml</groupId>
+ <artifactId>snakeyaml</artifactId>
+ <version>${arquillian-cube.snakeyaml.version}</version>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
<dependencies>
<dependency>
<groupId>org.codehaus.groovy</groupId>
@@ -104,9 +113,8 @@
<artifactId>arquillian-cube-docker</artifactId>
</dependency>
<dependency>
- <groupId>org.jboss.arquillian.junit</groupId>
- <artifactId>arquillian-junit-container</artifactId>
- <scope>test</scope>
+ <groupId>javax.ws.rs</groupId>
+ <artifactId>javax.ws.rs-api</artifactId>
</dependency>
</dependencies>
</project>
diff --git a/tests/integration-tests-utils/pom.xml b/tests/integration-tests-utils/pom.xml
index 7ce96055e4..90034983d8 100644
--- a/tests/integration-tests-utils/pom.xml
+++ b/tests/integration-tests-utils/pom.xml
@@ -47,7 +47,7 @@
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
- </dependency>
+ </dependency>
<dependency>
<groupId>org.arquillian.cube</groupId>
@@ -71,6 +71,11 @@
<type>pom</type>
</dependency>
+ <dependency>
+ <groupId>javax.ws.rs</groupId>
+ <artifactId>javax.ws.rs-api</artifactId>
+ </dependency>
+
</dependencies>
<build>
@@ -78,12 +83,9 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <!-- DO NOT CHANGE VERSION
- Versions newer than 2.8.1 do not respect useSystemClassLoader=false
- https://issues.apache.org/jira/browse/SUREFIRE-1476 //-->
- <version>2.8.1</version>
<configuration>
<forkCount>1</forkCount>
+ <reuseForks>false</reuseForks>
<useSystemClassLoader>false</useSystemClassLoader>
</configuration>
</plugin>
diff --git a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/BookKeeperClusterUtils.java b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/BookKeeperClusterUtils.java
index d6f02ca5c2..1e5044999a 100644
--- a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/BookKeeperClusterUtils.java
+++ b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/BookKeeperClusterUtils.java
@@ -91,8 +91,12 @@ public class BookKeeperClusterUtils {
public static void legacyMetadataFormat(DockerClient docker) throws Exception {
@Cleanup
ZooKeeper zk = BookKeeperClusterUtils.zookeeperClient(docker);
- zk.create("/ledgers", new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
- zk.create("/ledgers/available", new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
+ if (zk.exists("/ledgers", false) == null) {
+ zk.create("/ledgers", new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
+ }
+ if (zk.exists("/ledgers/available", false) == null) {
+ zk.create("/ledgers/available", new byte[0], Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
+ }
}
public static boolean metadataFormatIfNeeded(DockerClient docker, String version) throws Exception {
@@ -259,9 +263,6 @@ public class BookKeeperClusterUtils {
}
private static String computeBinFilenameByVersion(String version) {
- if (OLD_CLIENT_VERSIONS_WITH_OLD_BK_BIN_NAME.contains(version)) {
- return "bookkeeper";
- }
- return "bookkeeper_gradle";
+ return "bookkeeper";
}
}
diff --git a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/DockerUtils.java b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/DockerUtils.java
index 4c8d5bfaeb..bae9057fa1 100644
--- a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/DockerUtils.java
+++ b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/DockerUtils.java
@@ -52,9 +52,9 @@ public class DockerUtils {
private static final Logger LOG = LoggerFactory.getLogger(DockerUtils.class);
private static File getTargetDirectory(String containerId) {
- String base = System.getProperty("gradle.buildDirectory");
+ String base = System.getProperty("maven.buildDirectory");
if (base == null) {
- base = "build";
+ base = "target";
}
File directory = new File(base + "/container-logs/" + containerId);
if (!directory.exists() && !directory.mkdirs()) {
diff --git a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java
index c393f609c1..2b1fabf6be 100644
--- a/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java
+++ b/tests/integration-tests-utils/src/main/java/org/apache/bookkeeper/tests/integration/utils/MavenClassLoader.java
@@ -17,6 +17,8 @@
*/
package org.apache.bookkeeper.tests.integration.utils;
+import static org.jboss.shrinkwrap.resolver.api.maven.coordinate.MavenDependencies.createExclusion;
+
import com.google.common.collect.Lists;
import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
import groovy.lang.Closure;
@@ -40,21 +42,43 @@ import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Optional;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ThreadPoolExecutor;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
import java.util.zip.GZIPInputStream;
+import lombok.extern.slf4j.Slf4j;
import org.apache.commons.compress.archivers.ArchiveStreamFactory;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
+import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.jboss.shrinkwrap.resolver.api.maven.ConfigurableMavenResolverSystem;
import org.jboss.shrinkwrap.resolver.api.maven.Maven;
+import org.jboss.shrinkwrap.resolver.api.maven.MavenResolvedArtifact;
import org.jboss.shrinkwrap.resolver.api.maven.ScopeType;
+import org.jboss.shrinkwrap.resolver.api.maven.coordinate.MavenCoordinate;
import org.jboss.shrinkwrap.resolver.api.maven.coordinate.MavenDependencies;
import org.jboss.shrinkwrap.resolver.api.maven.coordinate.MavenDependency;
/**
* A maven class loader for resolving and loading maven artifacts.
*/
+@Slf4j
public class MavenClassLoader implements AutoCloseable {
+ private static ScheduledExecutorService delayedCloseExecutor = createExecutorThatShutsDownIdleThreads();
+
+ private static ScheduledExecutorService createExecutorThatShutsDownIdleThreads() {
+ ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
+ // Cast to ThreadPoolExecutor to access additional configuration methods
+ ThreadPoolExecutor poolExecutor = (ThreadPoolExecutor) executor;
+ // Set the timeout for idle threads
+ poolExecutor.setKeepAliveTime(10, TimeUnit.SECONDS);
+ // Allow core threads to time out
+ poolExecutor.allowCoreThreadTimeOut(true);
+ return executor;
+ }
private static List<File> currentVersionLibs;
@@ -77,22 +101,45 @@ public class MavenClassLoader implements AutoCloseable {
String mainArtifact) throws Exception {
Optional<String> slf4jVersion = Arrays.stream(resolver.resolve(mainArtifact)
.withTransitivity().asResolvedArtifact())
- .filter((a) -> a.getCoordinate().getGroupId().equals("org.slf4j")
- && a.getCoordinate().getArtifactId().equals("slf4j-1.2-api"))
+ .filter((a) -> a.getCoordinate().getGroupId().equals("org.slf4j"))
.map((a) -> a.getCoordinate().getVersion())
.findFirst();
+ MavenDependency dependency = MavenDependencies.createDependency(mainArtifact, ScopeType.COMPILE, false,
+ createExclusion("log4j", "log4j"),
+ createExclusion("org.slf4j", "slf4j-log4j12"),
+ createExclusion("ch.qos.reload4j", "log4j"),
+ createExclusion("org.slf4j", "slf4j-reload4j"),
+ createExclusion("org.apache.logging.log4j", "*")
+ );
List<MavenDependency> deps = Lists.newArrayList(
- MavenDependencies.createDependency(
- mainArtifact, ScopeType.COMPILE, false));
+ dependency);
if (slf4jVersion.isPresent()) {
deps.add(MavenDependencies.createDependency("org.slf4j:slf4j-simple:" + slf4jVersion.get(),
ScopeType.COMPILE, false));
+ deps.add(MavenDependencies.createDependency("org.slf4j:jcl-over-slf4j:" + slf4jVersion.get(),
+ ScopeType.COMPILE, false));
}
- File[] files = resolver.addDependencies(deps.toArray(new MavenDependency[0]))
- .resolve().withTransitivity().asFile();
- return createClassLoader(files);
+ MavenResolvedArtifact[] resolvedArtifact = resolver.addDependencies(deps.toArray(new MavenDependency[0]))
+ .resolve().withTransitivity().asResolvedArtifact();
+ File[] files = Arrays.stream(resolvedArtifact)
+ .filter((a) -> {
+ MavenCoordinate c = a.getCoordinate();
+ // exclude log4j
+ if (c.getGroupId().equals("org.apache.logging.log4j") || c.getGroupId().equals("log4j")
+ || c.getGroupId().equals("ch.qos.reload4j")
+ || c.getGroupId().equals("commons-logging")) {
+ return false;
+ }
+ if (c.getArtifactId().contains("log4j") || c.getArtifactId().contains("commons-logging")) {
+ return false;
+ }
+ return true;
+ }).map(MavenResolvedArtifact::asFile)
+ .collect(Collectors.toList())
+ .toArray(new File[0]);
+ return createClassLoader(files);
}
private static MavenClassLoader createClassLoader(File[] jars) {
@@ -115,6 +162,12 @@ public class MavenClassLoader implements AutoCloseable {
try {
loadedClass = findClass(name);
} catch (ClassNotFoundException ignored) {
+ // never load these classes from the parent classloader
+ if (name.startsWith("org.apache")
+ || name.startsWith("org.slf4j")
+ || name.startsWith("log4j")) {
+ throw ignored;
+ }
}
if (loadedClass == null) {
try {
@@ -139,13 +192,22 @@ public class MavenClassLoader implements AutoCloseable {
return forArtifact("org.apache.bookkeeper:bookkeeper-server:" + version);
}
- private static MavenClassLoader forBookkeeperCurrentVersion() throws Exception {
+ private static synchronized MavenClassLoader forBookkeeperCurrentVersion() throws Exception {
if (currentVersionLibs == null) {
final String version = BookKeeperClusterUtils.CURRENT_VERSION;
+ String rootDirectory = System.getenv("GITHUB_WORKSPACE");
+ if (rootDirectory == null) {
+ File gitDirectory = findGitRoot(new File("."));
+ if (gitDirectory != null) {
+ rootDirectory = gitDirectory.getAbsolutePath();
+ } else {
+ rootDirectory = System.getProperty("maven.buildDirectory", ".") + "/../../../..";
+ }
+ }
final String artifactName = "bookkeeper-server-" + version + "-bin";
- final Path tarFile = Paths.get("..", "..", "..",
- "bookkeeper-dist", "server", "build", "distributions", artifactName + ".tar.gz");
- final File tempDir = new File("build");
+ final Path tarFile = Paths.get(rootDirectory,
+ "bookkeeper-dist", "server", "target", artifactName + ".tar.gz").toAbsolutePath();
+ final File tempDir = new File(System.getProperty("maven.buildDirectory", "target"));
extractTarGz(tarFile.toFile(), tempDir);
List<File> jars = new ArrayList<>();
Files.list(Paths.get(tempDir.getAbsolutePath(), "bookkeeper-server-" + version, "lib"))
@@ -157,6 +219,16 @@ public class MavenClassLoader implements AutoCloseable {
return createClassLoader(currentVersionLibs.toArray(new File[]{}));
}
+ private static File findGitRoot(File currentDir) {
+ while (currentDir != null) {
+ if (new File(currentDir, ".git").exists()) {
+ return currentDir;
+ }
+ currentDir = currentDir.getParentFile();
+ }
+ return null;
+ }
+
public Object callStaticMethod(String className, String methodName, ArrayList<?> args) throws Exception {
Class<?> klass = Class.forName(className, true, classloader);
@@ -216,26 +288,31 @@ public class MavenClassLoader implements AutoCloseable {
}
public Object newBookKeeper(String zookeeper) throws Exception {
- Class<?> clientConfigurationClass = Class
- .forName("org.apache.bookkeeper.conf.ClientConfiguration", true, classloader);
- Object clientConfiguration = newInstance("org.apache.bookkeeper.conf.ClientConfiguration");
- clientConfigurationClass
- .getMethod("setZkServers", String.class)
- .invoke(clientConfiguration, zookeeper);
-
- // relax timeouts in order to get tests passing in limited environments
- clientConfigurationClass
- .getMethod("setReadTimeout", int.class)
- .invoke(clientConfiguration, 15);
-
- clientConfigurationClass
- .getMethod("setZkTimeout", int.class)
- .invoke(clientConfiguration, 30_000);
- Class<?> klass = Class.forName("org.apache.bookkeeper.client.BookKeeper", true, classloader);
- return klass
- .getConstructor(clientConfigurationClass)
- .newInstance(clientConfiguration);
+ ClassLoader contextClassLoader = Thread.currentThread().getContextClassLoader();
+ try {
+ Thread.currentThread().setContextClassLoader(classloader);
+ Class<?> clientConfigurationClass = Class
+ .forName("org.apache.bookkeeper.conf.ClientConfiguration", true, classloader);
+ Object clientConfiguration = newInstance("org.apache.bookkeeper.conf.ClientConfiguration");
+ clientConfigurationClass
+ .getMethod("setZkServers", String.class)
+ .invoke(clientConfiguration, zookeeper);
+
+ // relax timeouts in order to get tests passing in limited environments
+ clientConfigurationClass
+ .getMethod("setReadTimeout", int.class)
+ .invoke(clientConfiguration, 15);
+ clientConfigurationClass
+ .getMethod("setZkTimeout", int.class)
+ .invoke(clientConfiguration, 30_000);
+ Class<?> klass = Class.forName("org.apache.bookkeeper.client.BookKeeper", true, classloader);
+ return klass
+ .getConstructor(clientConfigurationClass)
+ .newInstance(clientConfiguration);
+ } finally {
+ Thread.currentThread().setContextClassLoader(contextClassLoader);
+ }
}
public Object digestType(String type) throws Exception {
@@ -251,7 +328,14 @@ public class MavenClassLoader implements AutoCloseable {
@Override
public void close() throws Exception {
if (classloader instanceof Closeable) {
- ((Closeable) classloader).close();
+ // delay closing the classloader so that currently executing asynchronous tasks can complete
+ delayedCloseExecutor.schedule(() -> {
+ try {
+ ((Closeable) classloader).close();
+ } catch (Exception e) {
+ log.error("Failed to close classloader", e);
+ }
+ }, 5, TimeUnit.SECONDS);
}
}
@@ -283,14 +367,16 @@ public class MavenClassLoader implements AutoCloseable {
TarArchiveEntry entry;
while ((entry = (TarArchiveEntry) debInputStream.getNextEntry()) != null) {
final File outputFile = new File(outputDir, entry.getName());
+ if (!outputFile.getParentFile().exists()) {
+ outputFile.getParentFile().mkdirs();
+ }
if (entry.isDirectory()) {
- if (!outputFile.exists()) {
- if (!outputFile.mkdirs()) {
- throw new IllegalStateException(
- String.format("Couldn't create directory %s.", outputFile.getAbsolutePath()));
- }
- } else {
- outputFile.delete();
+ if (outputFile.exists()) {
+ FileUtils.deleteDirectory(outputFile);
+ }
+ if (!outputFile.mkdirs()) {
+ throw new IllegalStateException(
+ String.format("Couldn't create directory %s.", outputFile.getAbsolutePath()));
}
} else {
try (final OutputStream outputFileStream = new FileOutputStream(outputFile)) {
diff --git a/tests/integration/cluster/pom.xml b/tests/integration/cluster/pom.xml
index b2573f6ea7..1f92628715 100644
--- a/tests/integration/cluster/pom.xml
+++ b/tests/integration/cluster/pom.xml
@@ -74,7 +74,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- smoke test should never flake //-->
<rerunFailingTestsCount>0</rerunFailingTestsCount>
diff --git a/tests/integration/smoke/pom.xml b/tests/integration/smoke/pom.xml
index a187044c2b..999c4ad89d 100644
--- a/tests/integration/smoke/pom.xml
+++ b/tests/integration/smoke/pom.xml
@@ -29,6 +29,28 @@
<packaging>jar</packaging>
<name>Apache BookKeeper :: Tests :: Integration :: Smoke test</name>
+ <dependencyManagement>
+ <dependencies>
+ <dependency>
+ <groupId>com.github.docker-java</groupId>
+ <artifactId>docker-java-bom</artifactId>
+ <version>${arquillian-cube.docker-java.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.arquillian.cube</groupId>
+ <artifactId>arquillian-cube-docker</artifactId>
+ <version>${arquillian-cube.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.yaml</groupId>
+ <artifactId>snakeyaml</artifactId>
+ <version>${arquillian-cube.snakeyaml.version}</version>
+ </dependency>
+ </dependencies>
+ </dependencyManagement>
+
<dependencies>
<dependency>
<groupId>org.apache.bookkeeper</groupId>
@@ -56,6 +78,11 @@
<artifactId>arquillian-junit-standalone</artifactId>
<scope>test</scope>
</dependency>
+
+ <dependency>
+ <groupId>org.arquillian.cube</groupId>
+ <artifactId>arquillian-cube-docker</artifactId>
+ </dependency>
</dependencies>
<build>
@@ -63,7 +90,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- smoke test should never flake //-->
<rerunFailingTestsCount>0</rerunFailingTestsCount>
diff --git a/tests/integration/smoke/src/test/java/org/apache/bookkeeper/tests/integration/BookieShellTestBase.java b/tests/integration/smoke/src/test/java/org/apache/bookkeeper/tests/integration/BookieShellTestBase.java
index 05ff030470..59626881da 100644
--- a/tests/integration/smoke/src/test/java/org/apache/bookkeeper/tests/integration/BookieShellTestBase.java
+++ b/tests/integration/smoke/src/test/java/org/apache/bookkeeper/tests/integration/BookieShellTestBase.java
@@ -22,13 +22,17 @@ import static org.junit.Assert.assertTrue;
import lombok.extern.slf4j.Slf4j;
import org.junit.Before;
+import org.junit.Rule;
import org.junit.Test;
+import org.junit.rules.Timeout;
/**
* Test Base for testing bookie shell scripts.
*/
@Slf4j
public abstract class BookieShellTestBase {
+ @Rule
+ public Timeout testTimeout = Timeout.seconds(300);
String currentVersion = System.getProperty("currentVersion");
String bkScript;
diff --git a/tests/integration/standalone/pom.xml b/tests/integration/standalone/pom.xml
index da506d8fa1..ff4c0010b4 100644
--- a/tests/integration/standalone/pom.xml
+++ b/tests/integration/standalone/pom.xml
@@ -59,7 +59,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
- <version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- smoke test should never flake //-->
<rerunFailingTestsCount>0</rerunFailingTestsCount>
diff --git a/tests/pom.xml b/tests/pom.xml
index 1a6127cc04..60862c5bb6 100644
--- a/tests/pom.xml
+++ b/tests/pom.xml
@@ -28,7 +28,10 @@
<name>Apache BookKeeper :: Tests</name>
<properties>
- <groovy.version>3.0.11</groovy.version>
+ <groovy.version>3.0.20</groovy.version>
+ <!-- must be compatible with arquillian-cube-docker -->
+ <arquillian-cube.docker-java.version>3.2.14</arquillian-cube.docker-java.version>
+ <arquillian-cube.snakeyaml.version>1.19</arquillian-cube.snakeyaml.version>
</properties>
<modules>
diff --git a/tests/shaded/bookkeeper-server-shaded-test/pom.xml b/tests/shaded/bookkeeper-server-shaded-test/pom.xml
index e9f07a0b2e..279e563f46 100644
--- a/tests/shaded/bookkeeper-server-shaded-test/pom.xml
+++ b/tests/shaded/bookkeeper-server-shaded-test/pom.xml
@@ -54,8 +54,8 @@
<artifactId>spotbugs-maven-plugin</artifactId>
</plugin>
<plugin>
+ <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
- <version>${maven-compiler-plugin.version}</version>
</plugin>
</plugins>
</build>
diff --git a/tools/ledger/src/test/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommandTest.java b/tools/ledger/src/test/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommandTest.java
index b06f0a62c3..70ffa46f00 100644
--- a/tools/ledger/src/test/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommandTest.java
+++ b/tools/ledger/src/test/java/org/apache/bookkeeper/tools/cli/commands/client/SimpleTestCommandTest.java
@@ -38,6 +38,7 @@ import java.util.Map;
import java.util.Random;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicLong;
+import java.util.stream.Collectors;
import org.apache.bookkeeper.client.api.CreateBuilder;
import org.apache.bookkeeper.client.api.DeleteBuilder;
import org.apache.bookkeeper.client.api.DigestType;
@@ -79,7 +80,6 @@ public class SimpleTestCommandTest extends ClientCommandTestBase {
public void testCommand(String... args) throws Exception {
WriteHandle wh = mock(WriteHandle.class);
AtomicLong counter = new AtomicLong(0L);
- when(wh.append(any(byte[].class))).thenReturn(counter.get());
CreateBuilder createBuilder = mock(CreateBuilder.class);
when(createBuilder.execute()).thenReturn(FutureUtils.value(wh));
when(createBuilder.withEnsembleSize(anyInt())).thenReturn(createBuilder);
@@ -90,23 +90,29 @@ public class SimpleTestCommandTest extends ClientCommandTestBase {
when(createBuilder.withPassword(any(byte[].class))).thenReturn(createBuilder);
when(createBuilder.execute()).thenReturn(CompletableFuture.completedFuture(wh));
when(mockBk.newCreateLedgerOp()).thenReturn(createBuilder);
-
+ long ledgerId = 1234L;
+ when(wh.getId()).thenReturn(ledgerId);
+ when(wh.getLastAddPushed()).then(__ -> counter.get() - 1L);
List<LedgerEntry> entries = new ArrayList<>();
byte[] data = new byte[100]; // test data
Random random = new Random(0);
for (int i = 0; i < data.length; i++) {
data[i] = (byte) (random.nextInt(26) + 65);
}
- for (int i = 0; i < 10; i++) {
+ when(wh.append(any(byte[].class))).then(invocation -> {
+ long entryId = counter.getAndIncrement();
ByteBuf buffer = UnpooledByteBufAllocator.DEFAULT.heapBuffer(100);
buffer.writeBytes(data);
- entries.add(LedgerEntryImpl.create(counter.get(), i, data.length, buffer));
- }
-
- LedgerEntriesImpl ledgerEntries = LedgerEntriesImpl.create(entries);
-
+ entries.add(LedgerEntryImpl.create(ledgerId, entryId, data.length, buffer));
+ return entryId;
+ });
ReadHandle rh = mock(ReadHandle.class);
- when(rh.read(anyLong(), anyLong())).thenReturn(ledgerEntries);
+ when(rh.read(anyLong(), anyLong())).then(
+ __ -> LedgerEntriesImpl.create(entries.stream()
+ .map(LedgerEntry::duplicate).collect(Collectors.toList())));
+ when(rh.readUnconfirmed(anyLong(), anyLong())).then(
+ __ -> LedgerEntriesImpl.create(entries.stream()
+ .map(LedgerEntry::duplicate).collect(Collectors.toList())));
OpenBuilder openBuilder = mock(OpenBuilder.class);
when(openBuilder.withLedgerId(anyLong())).thenReturn(openBuilder);
when(openBuilder.withDigestType(any())).thenReturn(openBuilder);
@@ -134,10 +140,10 @@ public class SimpleTestCommandTest extends ClientCommandTestBase {
verify(createBuilder, times(1)).withPassword(eq(new byte[0]));
verify(createBuilder, times(1)).execute();
- verify(openBuilder, times(1)).withLedgerId(eq(0L));
- verify(openBuilder, times(1)).execute();
+ verify(openBuilder, times(2)).withLedgerId(eq(1234L));
+ verify(openBuilder, times(2)).execute();
- verify(deleteBuilder, times(1)).withLedgerId(eq(0L));
+ verify(deleteBuilder, times(1)).withLedgerId(eq(1234L));
verify(deleteBuilder, times(1)).execute();
// verify appends