You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by pe...@apache.org on 2021/09/09 07:22:31 UTC
[pulsar] branch branch-2.8 updated (0873c09 -> 7384b7a)
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a change to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from 0873c09 Fix race condition in concurrent schema deletion (#11606)
new 3978657 [Broker] Support disabling non-TLS service ports (#11681)
new f96f967 Failed update partition of topic (#11683)
new 5344b82 [Enhancement] avoid duplicate deletion of schema (#11640)
new e4cdfe5 [Tests] Use TestRetrySupport for BaseMetadataStoreTests to cleanup state between retries (#11771)
new ff23ae9 Fix build from submodules (broker, transaction coordinator) (#11795)
new 4a1c426 [Build] Remove replace_maven-wagon-http-version.sh script which is obsolete with Maven 3.8.1+ (#11718)
new 7384b7a [Functions] ConcurrentHashMap should be used for caching producers (#11820)
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/ci-build-macos.yaml | 4 --
.github/workflows/ci-cpp.yaml | 4 --
.../ci-integration-backwards-compatibility.yaml | 4 --
.github/workflows/ci-integration-cli.yaml | 4 --
.github/workflows/ci-integration-function.yaml | 4 --
.github/workflows/ci-integration-messaging.yaml | 4 --
.github/workflows/ci-integration-process.yaml | 4 --
.github/workflows/ci-integration-schema.yaml | 4 --
.github/workflows/ci-integration-sql.yaml | 4 --
.github/workflows/ci-integration-standalone.yaml | 4 --
.github/workflows/ci-integration-thread.yaml | 4 --
.../ci-integration-tiered-filesystem.yaml | 4 --
.../workflows/ci-integration-tiered-jcloud.yaml | 4 --
.github/workflows/ci-integration-transaction.yaml | 4 --
.github/workflows/ci-license.yaml | 4 --
.github/workflows/ci-maven-cache-update.yaml | 4 --
.github/workflows/ci-pulsar-website-build.yaml | 3 --
.github/workflows/ci-shade-test.yaml | 4 --
.github/workflows/ci-unit-broker-broker-gp1.yaml | 4 --
.github/workflows/ci-unit-broker-broker-gp2.yaml | 4 --
.github/workflows/ci-unit-broker-client-api.yaml | 4 --
.github/workflows/ci-unit-broker-client-impl.yaml | 4 --
.github/workflows/ci-unit-broker-jdk8.yaml | 4 --
.github/workflows/ci-unit-broker-other.yaml | 4 --
.github/workflows/ci-unit-proxy.yaml | 4 --
.github/workflows/ci-unit.yaml | 4 --
build/replace_maven-wagon-http-version.sh | 28 -----------
.../org/apache/pulsar/tests/TestRetrySupport.java | 17 +++++++
pom.xml | 6 +++
pulsar-broker/pom.xml | 25 +++++++++-
.../org/apache/pulsar/broker/PulsarService.java | 7 ++-
.../pulsar/broker/admin/impl/NamespacesBase.java | 34 +++++++++++--
.../broker/admin/impl/PersistentTopicsBase.java | 15 ++++++
.../pulsar/broker/loadbalance/NoopLoadManager.java | 10 +++-
.../loadbalance/impl/SimpleLoadManagerImpl.java | 2 +-
.../apache/pulsar/broker/admin/AdminApiTest2.java | 56 ++++++++++++++++++++++
.../broker/auth/MockedPulsarServiceBaseTest.java | 8 ++--
.../pulsar/broker/service/BrokerServiceTest.java | 32 +++++++++++++
.../common/naming/ServiceConfigurationTest.java | 9 ++++
.../pulsar/functions/instance/ContextImpl.java | 2 +-
.../functions/worker/PulsarWorkerService.java | 17 ++++++-
.../pulsar/metadata/BaseMetadataStoreTest.java | 13 +++--
pulsar-transaction/coordinator/pom.xml | 29 +++++++++--
.../pulsar/websocket/service/ProxyServer.java | 6 ++-
site2/docs/security-tls-keystore.md | 13 +++++
45 files changed, 277 insertions(+), 155 deletions(-)
delete mode 100755 build/replace_maven-wagon-http-version.sh
[pulsar] 06/07: [Build] Remove replace_maven-wagon-http-version.sh
script which is obsolete with Maven 3.8.1+ (#11718)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 4a1c426248278e8c2e0fad66912cdd1799b8c1f5
Author: Lari Hotari <lh...@users.noreply.github.com>
AuthorDate: Thu Aug 19 22:07:09 2021 +0300
[Build] Remove replace_maven-wagon-http-version.sh script which is obsolete with Maven 3.8.1+ (#11718)
- reverts #9883 changes
- the solution is no more needed since GitHub Actions uses maven version 3.8.1+
and contains the maven-wagon-http version which includes fixes for
https://issues.apache.org/jira/browse/HTTPCORE-634 / https://issues.apache.org/jira/browse/WAGON-607
(cherry picked from commit 061fb4b313bd592f4218ebaec5a030734be5142e)
---
.github/workflows/ci-build-macos.yaml | 4 ----
.github/workflows/ci-cpp.yaml | 4 ----
.../ci-integration-backwards-compatibility.yaml | 4 ----
.github/workflows/ci-integration-cli.yaml | 4 ----
.github/workflows/ci-integration-function.yaml | 4 ----
.github/workflows/ci-integration-messaging.yaml | 4 ----
.github/workflows/ci-integration-process.yaml | 4 ----
.github/workflows/ci-integration-schema.yaml | 4 ----
.github/workflows/ci-integration-sql.yaml | 4 ----
.github/workflows/ci-integration-standalone.yaml | 4 ----
.github/workflows/ci-integration-thread.yaml | 4 ----
.../ci-integration-tiered-filesystem.yaml | 4 ----
.../workflows/ci-integration-tiered-jcloud.yaml | 4 ----
.github/workflows/ci-integration-transaction.yaml | 4 ----
.github/workflows/ci-license.yaml | 4 ----
.github/workflows/ci-maven-cache-update.yaml | 4 ----
.github/workflows/ci-pulsar-website-build.yaml | 3 ---
.github/workflows/ci-shade-test.yaml | 4 ----
.github/workflows/ci-unit-broker-broker-gp1.yaml | 4 ----
.github/workflows/ci-unit-broker-broker-gp2.yaml | 4 ----
.github/workflows/ci-unit-broker-client-api.yaml | 4 ----
.github/workflows/ci-unit-broker-client-impl.yaml | 4 ----
.github/workflows/ci-unit-broker-jdk8.yaml | 4 ----
.github/workflows/ci-unit-broker-other.yaml | 4 ----
.github/workflows/ci-unit-proxy.yaml | 4 ----
.github/workflows/ci-unit.yaml | 4 ----
build/replace_maven-wagon-http-version.sh | 28 ----------------------
27 files changed, 131 deletions(-)
diff --git a/.github/workflows/ci-build-macos.yaml b/.github/workflows/ci-build-macos.yaml
index 49167ab..5adc9df 100644
--- a/.github/workflows/ci-build-macos.yaml
+++ b/.github/workflows/ci-build-macos.yaml
@@ -73,10 +73,6 @@ jobs:
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build package
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B clean install -DskipTests
diff --git a/.github/workflows/ci-cpp.yaml b/.github/workflows/ci-cpp.yaml
index 4780a93..c36be95 100644
--- a/.github/workflows/ci-cpp.yaml
+++ b/.github/workflows/ci-cpp.yaml
@@ -79,10 +79,6 @@ jobs:
docker rmi $(docker images -q) -f
df -h
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build package
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-integration-backwards-compatibility.yaml b/.github/workflows/ci-integration-backwards-compatibility.yaml
index 9058060..2656714 100644
--- a/.github/workflows/ci-integration-backwards-compatibility.yaml
+++ b/.github/workflows/ci-integration-backwards-compatibility.yaml
@@ -73,10 +73,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-cli.yaml b/.github/workflows/ci-integration-cli.yaml
index aaf4e12..7dea8f6 100644
--- a/.github/workflows/ci-integration-cli.yaml
+++ b/.github/workflows/ci-integration-cli.yaml
@@ -73,10 +73,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-function.yaml b/.github/workflows/ci-integration-function.yaml
index 862e2a4..c0baf5b 100644
--- a/.github/workflows/ci-integration-function.yaml
+++ b/.github/workflows/ci-integration-function.yaml
@@ -73,10 +73,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-messaging.yaml b/.github/workflows/ci-integration-messaging.yaml
index c6e05f5..ac2167b 100644
--- a/.github/workflows/ci-integration-messaging.yaml
+++ b/.github/workflows/ci-integration-messaging.yaml
@@ -73,10 +73,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-process.yaml b/.github/workflows/ci-integration-process.yaml
index 56fa445..1142f1d 100644
--- a/.github/workflows/ci-integration-process.yaml
+++ b/.github/workflows/ci-integration-process.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-schema.yaml b/.github/workflows/ci-integration-schema.yaml
index bb3b158..f0f4df8 100644
--- a/.github/workflows/ci-integration-schema.yaml
+++ b/.github/workflows/ci-integration-schema.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-sql.yaml b/.github/workflows/ci-integration-sql.yaml
index cfcc7b2..3872b41 100644
--- a/.github/workflows/ci-integration-sql.yaml
+++ b/.github/workflows/ci-integration-sql.yaml
@@ -73,10 +73,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-standalone.yaml b/.github/workflows/ci-integration-standalone.yaml
index d8c55fb..07eae2f 100644
--- a/.github/workflows/ci-integration-standalone.yaml
+++ b/.github/workflows/ci-integration-standalone.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-thread.yaml b/.github/workflows/ci-integration-thread.yaml
index 18fcfd6..d58b0ae 100644
--- a/.github/workflows/ci-integration-thread.yaml
+++ b/.github/workflows/ci-integration-thread.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-tiered-filesystem.yaml b/.github/workflows/ci-integration-tiered-filesystem.yaml
index 8be3ac0..03126ec 100644
--- a/.github/workflows/ci-integration-tiered-filesystem.yaml
+++ b/.github/workflows/ci-integration-tiered-filesystem.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-tiered-jcloud.yaml b/.github/workflows/ci-integration-tiered-jcloud.yaml
index cf345e3..c935c4d 100644
--- a/.github/workflows/ci-integration-tiered-jcloud.yaml
+++ b/.github/workflows/ci-integration-tiered-jcloud.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-integration-transaction.yaml b/.github/workflows/ci-integration-transaction.yaml
index b2a7111..2793292 100644
--- a/.github/workflows/ci-integration-transaction.yaml
+++ b/.github/workflows/ci-integration-transaction.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-license.yaml b/.github/workflows/ci-license.yaml
index 52bc815..28af0c6 100644
--- a/.github/workflows/ci-license.yaml
+++ b/.github/workflows/ci-license.yaml
@@ -74,10 +74,6 @@ jobs:
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
# license check fails with 3.6.2 so we have to downgrade
- name: Set up Maven
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
diff --git a/.github/workflows/ci-maven-cache-update.yaml b/.github/workflows/ci-maven-cache-update.yaml
index 2de49a5..b04d286 100644
--- a/.github/workflows/ci-maven-cache-update.yaml
+++ b/.github/workflows/ci-maven-cache-update.yaml
@@ -105,10 +105,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ (github.event_name == 'schedule' || steps.changes.outputs.poms == 'true') && steps.cache.outputs.cache-hit != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: Download dependencies
if: ${{ (github.event_name == 'schedule' || steps.changes.outputs.poms == 'true') && steps.cache.outputs.cache-hit != 'true' }}
run: |
diff --git a/.github/workflows/ci-pulsar-website-build.yaml b/.github/workflows/ci-pulsar-website-build.yaml
index a6b196e..58d67ad 100644
--- a/.github/workflows/ci-pulsar-website-build.yaml
+++ b/.github/workflows/ci-pulsar-website-build.yaml
@@ -56,9 +56,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
run: |
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc
diff --git a/.github/workflows/ci-shade-test.yaml b/.github/workflows/ci-shade-test.yaml
index c3e2b42..d281e1d 100644
--- a/.github/workflows/ci-shade-test.yaml
+++ b/.github/workflows/ci-shade-test.yaml
@@ -73,10 +73,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: clean disk
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
diff --git a/.github/workflows/ci-unit-broker-broker-gp1.yaml b/.github/workflows/ci-unit-broker-broker-gp1.yaml
index 5cd6e1a..e7bbae8 100644
--- a/.github/workflows/ci-unit-broker-broker-gp1.yaml
+++ b/.github/workflows/ci-unit-broker-broker-gp1.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build modules
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q clean install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-unit-broker-broker-gp2.yaml b/.github/workflows/ci-unit-broker-broker-gp2.yaml
index 11b7928..947ce69 100644
--- a/.github/workflows/ci-unit-broker-broker-gp2.yaml
+++ b/.github/workflows/ci-unit-broker-broker-gp2.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build modules
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q clean install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-unit-broker-client-api.yaml b/.github/workflows/ci-unit-broker-client-api.yaml
index fc8da08..5001e43 100644
--- a/.github/workflows/ci-unit-broker-client-api.yaml
+++ b/.github/workflows/ci-unit-broker-client-api.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build modules
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-unit-broker-client-impl.yaml b/.github/workflows/ci-unit-broker-client-impl.yaml
index a739062..00e0d04 100644
--- a/.github/workflows/ci-unit-broker-client-impl.yaml
+++ b/.github/workflows/ci-unit-broker-client-impl.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build modules
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-unit-broker-jdk8.yaml b/.github/workflows/ci-unit-broker-jdk8.yaml
index a61def5..11ff3db 100644
--- a/.github/workflows/ci-unit-broker-jdk8.yaml
+++ b/.github/workflows/ci-unit-broker-jdk8.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 8
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build modules
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q clean install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-unit-broker-other.yaml b/.github/workflows/ci-unit-broker-other.yaml
index 6301ae9..fd21a87 100644
--- a/.github/workflows/ci-unit-broker-other.yaml
+++ b/.github/workflows/ci-unit-broker-other.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build modules
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q clean install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-unit-proxy.yaml b/.github/workflows/ci-unit-proxy.yaml
index 105cfc3..9fa9475 100644
--- a/.github/workflows/ci-unit-proxy.yaml
+++ b/.github/workflows/ci-unit-proxy.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: build modules pulsar-proxy
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: mvn -B -ntp -q install -Pcore-modules,-main -DskipTests
diff --git a/.github/workflows/ci-unit.yaml b/.github/workflows/ci-unit.yaml
index 373df8f..a01e4ac 100644
--- a/.github/workflows/ci-unit.yaml
+++ b/.github/workflows/ci-unit.yaml
@@ -72,10 +72,6 @@ jobs:
distribution: 'adopt'
java-version: 11
- - name: Replace maven's wagon-http version
- if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
- run: sudo ./build/replace_maven-wagon-http-version.sh
-
- name: run unit test 'OTHER'
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: ./build/run_unit_group.sh OTHER
diff --git a/build/replace_maven-wagon-http-version.sh b/build/replace_maven-wagon-http-version.sh
deleted file mode 100755
index 09b4ae3..0000000
--- a/build/replace_maven-wagon-http-version.sh
+++ /dev/null
@@ -1,28 +0,0 @@
-#!/bin/bash -xe
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-# patches installed maven version to get fix for https://issues.apache.org/jira/browse/HTTPCORE-634
-
-MAVEN_HOME=$(mvn -v |grep 'Maven home:' | awk '{ print $3 }')
-if [ -d "$MAVEN_HOME" ]; then
- cd "$MAVEN_HOME/lib"
- rm wagon-http-*-shaded.jar
- curl -O https://repo1.maven.org/maven2/org/apache/maven/wagon/wagon-http/3.4.3/wagon-http-3.4.3-shaded.jar
-fi
\ No newline at end of file
[pulsar] 03/07: [Enhancement] avoid duplicate deletion of schema
(#11640)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 5344b822edf2c5cfcacc8ef51bf99cf7e975560e
Author: Zhanpeng Wu <zh...@qq.com>
AuthorDate: Sat Aug 21 19:32:38 2021 +0800
[Enhancement] avoid duplicate deletion of schema (#11640)
### Motivation
Currently when I need to delete a namespace forcedly, the `NamespacesBase#internalDeleteNamespaceForcefully` will firstly list all the topics under the namespace and then delete them concurrently without distinguishing wheather the topic is partitioned or non-partitioned. This behaivor will cause duplicate deletion of schema when the topic list contains partitioned topics, because all the partitions of the partitioned topic share the same schema.
(cherry picked from commit e28f69ecf82d40b20aed16ad03f7831bbca721b4)
---
.../pulsar/broker/admin/impl/NamespacesBase.java | 34 +++++++++++--
.../apache/pulsar/broker/admin/AdminApiTest2.java | 56 ++++++++++++++++++++++
2 files changed, 87 insertions(+), 3 deletions(-)
diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
index 1a08e6c..2ba0a09 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/NamespacesBase.java
@@ -414,14 +414,42 @@ public abstract class NamespacesBase extends AdminResource {
try {
// firstly remove all topics including system topics
if (!topics.isEmpty()) {
+ Set<String> partitionedTopics = new HashSet<>();
+ Set<String> nonPartitionedTopics = new HashSet<>();
+
for (String topic : topics) {
try {
- futures.add(pulsar().getAdminClient().topics().deleteAsync(topic, true, true));
+ TopicName topicName = TopicName.get(topic);
+ if (topicName.isPartitioned()) {
+ String partitionedTopic = topicName.getPartitionedTopicName();
+ if (!partitionedTopics.contains(partitionedTopic)) {
+ // Distinguish partitioned topic to avoid duplicate deletion of the same schema
+ futures.add(pulsar().getAdminClient().topics().deletePartitionedTopicAsync(
+ partitionedTopic, true, true));
+ partitionedTopics.add(partitionedTopic);
+ }
+ } else {
+ futures.add(pulsar().getAdminClient().topics().deleteAsync(
+ topic, true, true));
+ nonPartitionedTopics.add(topic);
+ }
} catch (Exception e) {
- log.error("[{}] Failed to force delete topic {}", clientAppId(), topic, e);
- asyncResponse.resume(new RestException(e));
+ String errorMessage = String.format("Failed to force delete topic %s, "
+ + "but the previous deletion command of partitioned-topics:%s "
+ + "and non-partitioned-topics:%s have been sent out asynchronously. "
+ + "Reason: %s",
+ topic, partitionedTopics, nonPartitionedTopics, e.getCause());
+ log.error("[{}] {}", clientAppId(), errorMessage, e);
+ asyncResponse.resume(new RestException(Status.INTERNAL_SERVER_ERROR, errorMessage));
+ return;
}
}
+
+ if (log.isDebugEnabled()) {
+ log.debug("Successfully send deletion command of partitioned-topics:{} "
+ + "and non-partitioned-topics:{} in namespace:{}.",
+ partitionedTopics, nonPartitionedTopics, namespaceName);
+ }
}
// forcefully delete namespace bundles
NamespaceBundles bundles = pulsar().getNamespaceService().getNamespaceBundleFactory()
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest2.java b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest2.java
index 3eac264..e43cf4f 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest2.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest2.java
@@ -37,6 +37,7 @@ import java.lang.reflect.Field;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
@@ -45,6 +46,7 @@ import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.UUID;
+import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.TimeUnit;
import javax.ws.rs.core.Response.Status;
import lombok.Cleanup;
@@ -1472,6 +1474,60 @@ public class AdminApiTest2 extends MockedPulsarServiceBaseTest {
}
@Test
+ public void testDistinguishTopicTypeWhenForceDeleteNamespace() throws Exception {
+ conf.setForceDeleteNamespaceAllowed(true);
+ final String ns = "prop-xyz/distinguish-topic-type-ns";
+ final String exNs = "prop-xyz/ex-distinguish-topic-type-ns";
+ admin.namespaces().createNamespace(ns, 2);
+ admin.namespaces().createNamespace(exNs, 2);
+
+ final String p1 = "persistent://" + ns + "/p1";
+ final String p5 = "persistent://" + ns + "/p5";
+ final String np = "persistent://" + ns + "/np";
+
+ admin.topics().createPartitionedTopic(p1, 1);
+ admin.topics().createPartitionedTopic(p5, 5);
+ admin.topics().createNonPartitionedTopic(np);
+
+ final String exNp = "persistent://" + exNs + "/np";
+ admin.topics().createNonPartitionedTopic(exNp);
+ // insert an invalid topic name
+ pulsar.getLocalMetadataStore().put(
+ "/managed-ledgers/" + exNs + "/persistent/", "".getBytes(), Optional.empty()).join();
+
+ List<String> topics = pulsar.getNamespaceService().getFullListOfTopics(NamespaceName.get(ns)).get();
+ List<String> exTopics = pulsar.getNamespaceService().getFullListOfTopics(NamespaceName.get(exNs)).get();
+
+ // ensure that the topic list contains all the topics
+ List<String> allTopics = new ArrayList<>(Arrays.asList(np, TopicName.get(p1).getPartition(0).toString()));
+ for (int i = 0; i < 5; i++) {
+ allTopics.add(TopicName.get(p5).getPartition(i).toString());
+ }
+ Assert.assertEquals(allTopics.stream().filter(t -> !topics.contains(t)).count(), 0);
+ Assert.assertTrue(exTopics.contains("persistent://" + exNs + "/"));
+ // partition num = p1 + p5 + np
+ Assert.assertEquals(topics.size(), 1 + 5 + 1);
+ Assert.assertEquals(exTopics.size(), 1 + 1);
+
+ admin.namespaces().deleteNamespace(ns, true);
+ Arrays.asList(p1, p5, np).forEach(t -> {
+ try {
+ admin.schemas().getSchemaInfo(t);
+ } catch (PulsarAdminException e) {
+ // all the normal topics' schemas have been deleted
+ Assert.assertEquals(e.getStatusCode(), 404);
+ }
+ });
+
+ try {
+ admin.namespaces().deleteNamespace(exNs, true);
+ fail("Should fail due to invalid topic");
+ } catch (Exception e) {
+ //ok
+ }
+ }
+
+ @Test
public void testUpdateClusterWithProxyUrl() throws Exception {
ClusterData cluster = ClusterData.builder().serviceUrl(pulsar.getWebServiceAddress()).build();
String clusterName = "test2";
[pulsar] 04/07: [Tests] Use TestRetrySupport for
BaseMetadataStoreTests to cleanup state between retries (#11771)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit e4cdfe574516eb7d82b2130df022fcbc83521dfe
Author: Lari Hotari <lh...@users.noreply.github.com>
AuthorDate: Wed Aug 25 11:22:13 2021 +0300
[Tests] Use TestRetrySupport for BaseMetadataStoreTests to cleanup state between retries (#11771)
* [Tests] Use TestRetrySupport for BaseMetadataStoreTests to cleanup state between retries
- mitigates flaky tests in LockManagerTest #11690 and ZkSessionTest #11032
* [Tests] Add notes about the correct usage of TestRetrySupport
(cherry picked from commit 7de9992b6c047fbdda7ab47ba83b97156f881d12)
---
.../java/org/apache/pulsar/tests/TestRetrySupport.java | 17 +++++++++++++++++
.../apache/pulsar/metadata/BaseMetadataStoreTest.java | 13 ++++++++-----
2 files changed, 25 insertions(+), 5 deletions(-)
diff --git a/buildtools/src/main/java/org/apache/pulsar/tests/TestRetrySupport.java b/buildtools/src/main/java/org/apache/pulsar/tests/TestRetrySupport.java
index c4b9fc6..a9bf4b4 100644
--- a/buildtools/src/main/java/org/apache/pulsar/tests/TestRetrySupport.java
+++ b/buildtools/src/main/java/org/apache/pulsar/tests/TestRetrySupport.java
@@ -33,6 +33,10 @@ import org.testng.annotations.BeforeMethod;
* This is useful for making test retries to work on classes which use BeforeClass
* and AfterClass methods to setup a test environment that is shared across all test methods in the test
* class.
+ *
+ * The setup method implementation must call incrementSetupNumber method and the cleanup method must call
+ * markCurrentSetupNumberCleaned method. This is required by the state tracking logic.
+ *
*/
public abstract class TestRetrySupport {
private static final Logger LOG = LoggerFactory.getLogger(TestRetrySupport.class);
@@ -83,12 +87,25 @@ public abstract class TestRetrySupport {
LOG.debug("currentSetupNumber={}", currentSetupNumber);
}
+ /**
+ * This method should be called in the cleanup method of the concrete class.
+ */
protected final void markCurrentSetupNumberCleaned() {
cleanedUpSetupNumber = currentSetupNumber;
LOG.debug("cleanedUpSetupNumber={}", cleanedUpSetupNumber);
}
+ /**
+ * Initializes the test environment state.
+ *
+ * The implementation of this method must call incrementSetupNumber method.
+ */
protected abstract void setup() throws Exception;
+ /**
+ * Cleans up the state of the environment.
+ *
+ * The implementation of this method must call the markCurrentSetupNumberCleaned method.
+ */
protected abstract void cleanup() throws Exception;
}
diff --git a/pulsar-metadata/src/test/java/org/apache/pulsar/metadata/BaseMetadataStoreTest.java b/pulsar-metadata/src/test/java/org/apache/pulsar/metadata/BaseMetadataStoreTest.java
index 49c916a..afd6f3a 100644
--- a/pulsar-metadata/src/test/java/org/apache/pulsar/metadata/BaseMetadataStoreTest.java
+++ b/pulsar-metadata/src/test/java/org/apache/pulsar/metadata/BaseMetadataStoreTest.java
@@ -19,23 +19,26 @@
package org.apache.pulsar.metadata;
import static org.testng.Assert.assertTrue;
-
import java.util.concurrent.CompletionException;
-
+import org.apache.pulsar.tests.TestRetrySupport;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.DataProvider;
-public abstract class BaseMetadataStoreTest {
+public abstract class BaseMetadataStoreTest extends TestRetrySupport {
protected TestZKServer zks;
@BeforeClass(alwaysRun = true)
- void setup() throws Exception {
+ @Override
+ protected void setup() throws Exception {
+ incrementSetupNumber();
zks = new TestZKServer();
}
@AfterClass(alwaysRun = true)
- void teardown() throws Exception {
+ @Override
+ protected void cleanup() throws Exception {
+ markCurrentSetupNumberCleaned();
zks.close();
}
[pulsar] 01/07: [Broker] Support disabling non-TLS service ports
(#11681)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 39786572b168950679d26b0a945adf671e3f7cf4
Author: Lari Hotari <lh...@users.noreply.github.com>
AuthorDate: Thu Aug 19 11:36:37 2021 +0300
[Broker] Support disabling non-TLS service ports (#11681)
* Support disabling non-tls service ports
* Add docs for disabling non-TLS ports
* Update site2/docs/security-tls-keystore.md
Co-authored-by: Anonymitaet <50...@users.noreply.github.com>
(cherry picked from commit 50b6e79d7cc350efb2208b4aa89f684e133e31c0)
---
.../org/apache/pulsar/broker/PulsarService.java | 7 +++--
.../pulsar/broker/loadbalance/NoopLoadManager.java | 10 +++++--
.../loadbalance/impl/SimpleLoadManagerImpl.java | 2 +-
.../broker/auth/MockedPulsarServiceBaseTest.java | 8 ++++--
.../pulsar/broker/service/BrokerServiceTest.java | 32 ++++++++++++++++++++++
.../common/naming/ServiceConfigurationTest.java | 9 ++++++
.../functions/worker/PulsarWorkerService.java | 17 ++++++++++--
.../pulsar/websocket/service/ProxyServer.java | 6 +++-
site2/docs/security-tls-keystore.md | 13 +++++++++
9 files changed, 93 insertions(+), 11 deletions(-)
diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
index c09ec45..8d78d40 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
@@ -1541,12 +1541,15 @@ public class PulsarService implements AutoCloseable {
AuthorizationService authorizationService)
throws Exception {
if (functionWorkerService.isPresent()) {
- if (workerConfig.isUseTls()) {
+ if (workerConfig.isUseTls() || brokerServiceUrl == null) {
workerConfig.setPulsarServiceUrl(brokerServiceUrlTls);
+ } else {
+ workerConfig.setPulsarServiceUrl(brokerServiceUrl);
+ }
+ if (workerConfig.isUseTls() || webServiceAddress == null) {
workerConfig.setPulsarWebServiceUrl(webServiceAddressTls);
workerConfig.setFunctionWebServiceUrl(webServiceAddressTls);
} else {
- workerConfig.setPulsarServiceUrl(brokerServiceUrl);
workerConfig.setPulsarWebServiceUrl(webServiceAddress);
workerConfig.setFunctionWebServiceUrl(webServiceAddress);
}
diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/NoopLoadManager.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/NoopLoadManager.java
index db017c1..85071ff 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/NoopLoadManager.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/NoopLoadManager.java
@@ -51,8 +51,7 @@ public class NoopLoadManager implements LoadManager {
@Override
public void start() throws PulsarServerException {
- lookupServiceAddress = pulsar.getAdvertisedAddress() + ":"
- + pulsar.getConfiguration().getWebServicePort().get();
+ lookupServiceAddress = getBrokerAddress();
localResourceUnit = new SimpleResourceUnit(String.format("http://%s", lookupServiceAddress),
new PulsarResourceDescription());
@@ -71,6 +70,13 @@ public class NoopLoadManager implements LoadManager {
}
}
+ private String getBrokerAddress() {
+ return String.format("%s:%s", pulsar.getAdvertisedAddress(),
+ pulsar.getConfiguration().getWebServicePort().isPresent()
+ ? pulsar.getConfiguration().getWebServicePort().get()
+ : pulsar.getConfiguration().getWebServicePortTls().get());
+ }
+
@Override
public boolean isCentralized() {
return false;
diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/SimpleLoadManagerImpl.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/SimpleLoadManagerImpl.java
index fe5eddf..b14a018 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/SimpleLoadManagerImpl.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/SimpleLoadManagerImpl.java
@@ -1118,7 +1118,7 @@ public class SimpleLoadManagerImpl implements LoadManager, Consumer<Notification
return String.format("%s:%s", pulsar.getAdvertisedAddress(),
pulsar.getConfiguration().getWebServicePort().isPresent()
? pulsar.getConfiguration().getWebServicePort().get()
- : pulsar.getConfiguration().getWebServicePortTls());
+ : pulsar.getConfiguration().getWebServicePortTls().get());
}
@Override
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
index ddb76e8..fb23eb3 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/broker/auth/MockedPulsarServiceBaseTest.java
@@ -264,13 +264,15 @@ public abstract class MockedPulsarServiceBaseTest extends TestRetrySupport {
}
this.pulsar = startBroker(conf);
- brokerUrl = new URL(pulsar.getWebServiceAddress());
- brokerUrlTls = new URL(pulsar.getWebServiceAddressTls());
+ brokerUrl = pulsar.getWebServiceAddress() != null ? new URL(pulsar.getWebServiceAddress()) : null;
+ brokerUrlTls = pulsar.getWebServiceAddressTls() != null ? new URL(pulsar.getWebServiceAddressTls()) : null;
if (admin != null) {
admin.close();
}
- PulsarAdminBuilder pulsarAdminBuilder = PulsarAdmin.builder().serviceHttpUrl(brokerUrl.toString());
+ PulsarAdminBuilder pulsarAdminBuilder = PulsarAdmin.builder().serviceHttpUrl(brokerUrl != null
+ ? brokerUrl.toString()
+ : brokerUrlTls.toString());
customizeNewPulsarAdminBuilder(pulsarAdminBuilder);
admin = spy(pulsarAdminBuilder.build());
}
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
index a0e3a9e..ffd474f 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceTest.java
@@ -567,6 +567,38 @@ public class BrokerServiceTest extends BrokerTestBase {
}
}
+ @Test
+ public void testTlsEnabledWithoutNonTlsServicePorts() throws Exception {
+ final String topicName = "persistent://prop/ns-abc/newTopic";
+ final String subName = "newSub";
+
+ conf.setAuthenticationEnabled(false);
+ conf.setBrokerServicePort(Optional.empty());
+ conf.setBrokerServicePortTls(Optional.of(0));
+ conf.setWebServicePort(Optional.empty());
+ conf.setWebServicePortTls(Optional.of(0));
+ conf.setTlsCertificateFilePath(TLS_SERVER_CERT_FILE_PATH);
+ conf.setTlsKeyFilePath(TLS_SERVER_KEY_FILE_PATH);
+ conf.setNumExecutorThreadPoolSize(5);
+ restartBroker();
+
+ // Access with TLS (Allow insecure TLS connection)
+ try {
+ pulsarClient = PulsarClient.builder().serviceUrl(brokerUrlTls.toString()).enableTls(true)
+ .allowTlsInsecureConnection(true).statsInterval(0, TimeUnit.SECONDS)
+ .operationTimeout(1000, TimeUnit.MILLISECONDS).build();
+
+ @Cleanup
+ Consumer<byte[]> consumer = pulsarClient.newConsumer().topic(topicName).subscriptionName(subName)
+ .subscribe();
+
+ } catch (Exception e) {
+ fail("should not fail");
+ } finally {
+ pulsarClient.close();
+ }
+ }
+
@SuppressWarnings("deprecation")
@Test
public void testTlsAuthAllowInsecure() throws Exception {
diff --git a/pulsar-broker/src/test/java/org/apache/pulsar/common/naming/ServiceConfigurationTest.java b/pulsar-broker/src/test/java/org/apache/pulsar/common/naming/ServiceConfigurationTest.java
index 6b7df2c..c5807e3 100644
--- a/pulsar-broker/src/test/java/org/apache/pulsar/common/naming/ServiceConfigurationTest.java
+++ b/pulsar-broker/src/test/java/org/apache/pulsar/common/naming/ServiceConfigurationTest.java
@@ -78,6 +78,15 @@ public class ServiceConfigurationTest {
assertEquals(config.getLoadBalancerOverrideBrokerNicSpeedGbps(), Optional.of(5.0));
}
+ @Test
+ public void testServicePortsEmpty() throws Exception {
+ String confFile = "brokerServicePort=\nwebServicePort=\n";
+ InputStream stream = new ByteArrayInputStream(confFile.getBytes());
+ final ServiceConfiguration config = PulsarConfigurationLoader.create(stream, ServiceConfiguration.class);
+ assertEquals(config.getBrokerServicePort(), Optional.empty());
+ assertEquals(config.getWebServicePort(), Optional.empty());
+ }
+
/**
* test {@link ServiceConfiguration} with incorrect values.
*
diff --git a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/PulsarWorkerService.java b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/PulsarWorkerService.java
index 85c1152..93d4263 100644
--- a/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/PulsarWorkerService.java
+++ b/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/PulsarWorkerService.java
@@ -141,7 +141,13 @@ public class PulsarWorkerService implements WorkerService {
workerConfig.isTlsAllowInsecureConnection(),
workerConfig.isTlsEnableHostnameVerification());
} else {
- return WorkerUtils.getPulsarAdminClient(pulsarServiceUrl);
+ return WorkerUtils.getPulsarAdminClient(
+ pulsarServiceUrl,
+ null,
+ null,
+ null,
+ workerConfig.isTlsAllowInsecureConnection(),
+ workerConfig.isTlsEnableHostnameVerification());
}
}
@@ -158,7 +164,14 @@ public class PulsarWorkerService implements WorkerService {
workerConfig.isTlsAllowInsecureConnection(),
workerConfig.isTlsEnableHostnameVerification());
} else {
- return WorkerUtils.getPulsarClient(pulsarServiceUrl);
+ return WorkerUtils.getPulsarClient(
+ pulsarServiceUrl,
+ null,
+ null,
+ null,
+ null,
+ workerConfig.isTlsAllowInsecureConnection(),
+ workerConfig.isTlsEnableHostnameVerification());
}
}
};
diff --git a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/service/ProxyServer.java b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/service/ProxyServer.java
index ae5fd8d..c616db0 100644
--- a/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/service/ProxyServer.java
+++ b/pulsar-websocket/src/main/java/org/apache/pulsar/websocket/service/ProxyServer.java
@@ -22,10 +22,12 @@ import com.google.common.collect.Lists;
import java.net.MalformedURLException;
import java.util.ArrayList;
+import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import java.util.TimeZone;
+import java.util.stream.Collectors;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import javax.websocket.DeploymentException;
@@ -120,7 +122,9 @@ public class ProxyServer {
}
public void start() throws PulsarServerException {
- log.info("Starting web socket proxy at port {}", conf.getWebServicePort().get());
+ log.info("Starting web socket proxy at port {}", Arrays.stream(server.getConnectors())
+ .map(ServerConnector.class::cast).map(ServerConnector::getPort).map(Object::toString)
+ .collect(Collectors.joining(",")));
RequestLogHandler requestLogHandler = new RequestLogHandler();
Slf4jRequestLog requestLog = new Slf4jRequestLog();
requestLog.setExtended(true);
diff --git a/site2/docs/security-tls-keystore.md b/site2/docs/security-tls-keystore.md
index 4f432aa..fd5a75a 100644
--- a/site2/docs/security-tls-keystore.md
+++ b/site2/docs/security-tls-keystore.md
@@ -131,6 +131,19 @@ brokerClientTlsTrustStorePassword=clientpw
NOTE: it is important to restrict access to the store files via filesystem permissions.
+If you have configured TLS on the broker, to disable non-TLS ports, you can set the values of the following configurations to empty as below.
+```
+brokerServicePort=
+webServicePort=
+```
+In this case, you need to set the following configurations.
+
+```conf
+brokerClientTlsEnabled=true // Set this to true
+brokerClientTlsEnabledWithKeyStore=true // Set this to true
+brokerClientTlsTrustStore= // Set this to your desired value
+brokerClientTlsTrustStorePassword= // Set this to your desired value
+
Optional settings that may worth consider:
1. tlsClientAuthentication=false: Enable/Disable using TLS for authentication. This config when enabled will authenticate the other end
[pulsar] 02/07: Failed update partition of topic (#11683)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit f96f967b8da8e65a60877deab80fc87cda3c4733
Author: gaozhangmin <ga...@qq.com>
AuthorDate: Mon Aug 30 11:39:50 2021 +0800
Failed update partition of topic (#11683)
Fixes #11682
### Motivation
1、`tryCreatePartitionsAsync` would create znode under `managed-ledgers/public/default/persistent/` after success invoke
2、there are chance, `updatePartitionedTopic` would failed, and managed-ledgers znode would stay unclean. If We tried update partition once again , We would fail, because of managed-ledgers znode exists
https://github.com/apache/pulsar/blob/80171a733ab4799f912a8935f03c19554b9ca3b1/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java#L473-L485
### Modifications
clean up managed-ledger znode after we failed update partition.
(cherry picked from commit 90c7db4dadddf4900565b9b3315cb86666aa1648)
---
.../pulsar/broker/admin/impl/PersistentTopicsBase.java | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
index f63d318..a64e0b0 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
@@ -3584,6 +3584,21 @@ public class PersistentTopicsBase extends AdminResource {
p -> new PartitionedTopicMetadata(numPartitions));
updatePartition.complete(null);
} catch (Exception e) {
+ getPartitionedTopicMetadataAsync(topicName, false, false).thenAccept(metadata -> {
+ int oldPartition = metadata.partitions;
+ for (int i = oldPartition; i < numPartitions; i++) {
+ String managedLedgerPath = ZkAdminPaths.managedLedgerPath(topicName.getPartition(i));
+ namespaceResources().getPartitionedTopicResources()
+ .deleteAsync(managedLedgerPath).exceptionally(ex1 -> {
+ log.warn("[{}] Failed to clean up managedLedger znode {}", clientAppId(),
+ managedLedgerPath, ex1.getCause());
+ return null;
+ });
+ }
+ }).exceptionally(ex -> {
+ log.warn("[{}] Failed to clean up managedLedger znode", clientAppId(), ex.getCause());
+ return null;
+ });
updatePartition.completeExceptionally(e);
}
}).exceptionally(ex -> {
[pulsar] 07/07: [Functions] ConcurrentHashMap should be used for
caching producers (#11820)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 7384b7a313737161f13dbdf14df38768eca23163
Author: Lari Hotari <lh...@users.noreply.github.com>
AuthorDate: Mon Aug 30 22:32:57 2021 +0300
[Functions] ConcurrentHashMap should be used for caching producers (#11820)
(cherry picked from commit 3231caa894a24fa9048adf4628316d280d02c679)
---
.../src/main/java/org/apache/pulsar/functions/instance/ContextImpl.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/ContextImpl.java b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/ContextImpl.java
index 3170236..fc806c1 100644
--- a/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/ContextImpl.java
+++ b/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/ContextImpl.java
@@ -155,7 +155,7 @@ class ContextImpl implements Context, SinkContext, SourceContext, AutoCloseable
if (useThreadLocalProducers) {
tlPublishProducers = new ThreadLocal<>();
} else {
- publishProducers = new HashMap<>();
+ publishProducers = new ConcurrentHashMap<>();
}
if (config.getFunctionDetails().getUserConfig().isEmpty()) {
[pulsar] 05/07: Fix build from submodules (broker,
transaction coordinator) (#11795)
Posted by pe...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
penghui pushed a commit to branch branch-2.8
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit ff23ae9c4b9dc779d3b22259057997cc87db1f3d
Author: Nicolò Boschi <bo...@gmail.com>
AuthorDate: Thu Aug 26 13:47:56 2021 +0200
Fix build from submodules (broker, transaction coordinator) (#11795)
(cherry picked from commit ea1e0f4d7b0ac0d88a6c755f24910e239ac0c5f0)
---
pom.xml | 6 ++++++
pulsar-broker/pom.xml | 25 ++++++++++++++++++++++++-
pulsar-transaction/coordinator/pom.xml | 29 ++++++++++++++++++++++++++---
3 files changed, 56 insertions(+), 4 deletions(-)
diff --git a/pom.xml b/pom.xml
index e5456e0..005850e 100644
--- a/pom.xml
+++ b/pom.xml
@@ -227,6 +227,7 @@ flexible messaging model and an intuitive client API.</description>
<maven-dependency-plugin.version>3.1.2</maven-dependency-plugin.version>
<maven-shade-plugin>3.2.4</maven-shade-plugin>
<maven-antrun-plugin.version>3.0.0</maven-antrun-plugin.version>
+ <properties-maven-plugin.version>1.0.0</properties-maven-plugin.version>
<nifi-nar-maven-plugin.version>1.2.0</nifi-nar-maven-plugin.version>
<maven-checkstyle-plugin.version>3.1.2</maven-checkstyle-plugin.version>
<git-commit-id-plugin.version>4.0.2</git-commit-id-plugin.version>
@@ -1692,6 +1693,11 @@ flexible messaging model and an intuitive client API.</description>
</dependency>
</dependencies>
</plugin>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>properties-maven-plugin</artifactId>
+ <version>${properties-maven-plugin.version}</version>
+ </plugin>
</plugins>
</pluginManagement>
<extensions>
diff --git a/pulsar-broker/pom.xml b/pulsar-broker/pom.xml
index 18d420c..b596640 100644
--- a/pulsar-broker/pom.xml
+++ b/pulsar-broker/pom.xml
@@ -474,7 +474,30 @@
</execution>
</executions>
</plugin>
-
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>properties-maven-plugin</artifactId>
+ <executions>
+ <execution>
+ <phase>initialize</phase>
+ <goals>
+ <goal>set-system-properties</goal>
+ </goals>
+ <configuration>
+ <properties>
+ <property>
+ <name>proto_path</name>
+ <value>${project.parent.basedir}</value>
+ </property>
+ <property>
+ <name>proto_search_strategy</name>
+ <value>2</value>
+ </property>
+ </properties>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
<plugin>
<groupId>com.github.splunk.lightproto</groupId>
<artifactId>lightproto-maven-plugin</artifactId>
diff --git a/pulsar-transaction/coordinator/pom.xml b/pulsar-transaction/coordinator/pom.xml
index 1bd6a8b..f6c4112 100644
--- a/pulsar-transaction/coordinator/pom.xml
+++ b/pulsar-transaction/coordinator/pom.xml
@@ -57,7 +57,6 @@
<dependency>
<groupId>org.awaitility</groupId>
<artifactId>awaitility</artifactId>
- <version>${awaitility.version}</version>
<scope>test</scope>
</dependency>
@@ -65,8 +64,32 @@
<build>
<plugins>
- <plugin>
- <groupId>com.github.splunk.lightproto</groupId>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>properties-maven-plugin</artifactId>
+ <executions>
+ <execution>
+ <phase>initialize</phase>
+ <goals>
+ <goal>set-system-properties</goal>
+ </goals>
+ <configuration>
+ <properties>
+ <property>
+ <name>proto_path</name>
+ <value>${project.parent.parent.basedir}</value>
+ </property>
+ <property>
+ <name>proto_search_strategy</name>
+ <value>2</value>
+ </property>
+ </properties>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>com.github.splunk.lightproto</groupId>
<artifactId>lightproto-maven-plugin</artifactId>
<version>${lightproto-maven-plugin.version}</version>
<executions>