You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by rx...@apache.org on 2019/11/20 12:43:39 UTC
[pulsar] branch branch-2.4 updated (cc74632 -> 59cf167)
This is an automated email from the ASF dual-hosted git repository.
rxl pushed a change to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/pulsar.git.
from cc74632 Fix：PulsarKafkaProducer is not thread safe (#4745)
new 1a427f2 ### Motivation (#4999)
new 1b31ace add different cache flags for size() and toString() (#5040)
new c0054f2 Reload zk cache asynchronously (#5049)
new 30bf216 [bookkeeper] Don't return DEFAULT_RACK if ZkBookieRackAffinityMapping can't resolve network location (#5067)
new 43bfae3 modify BatcherBuilder interface and it's subs to implement java.io.Serializable (#5068)
new e05c095 Use "info" as the default root logger level (#5079)
new 6a59450 Add the schema admin api (#4800)
new 4c98dba Add checkstyle validation and fix style violations in the common module (#4989)
new 2992ca7 [admin] DeleteBookieRack should remove the rack info from zookeeper (#5084)
new 88ab1e2 [bookkeeper] allow configuring region aware placement related settings (#5100)
new cbcd298 Remove failed stale producer from the connection (#4741)
new f17ada4 Test cleanup and simplification (#4799)
new 3e8b33c Fix storage size always 0 without subscription. (#5108)
new f777fd1 Fix: windowed functions were broken when we changed java function instance to use classloaders (#5117)
new dc61cdf Throw an error if the key was not specified for querying state (#5145)
new f0571b1 Broker fails to start with function worker enabled and broker client using TLS (#5151)
new cb6fcbd Fixed spark receiver to account for all the consumer config options (#5152)
new b1651a9 [pulsar-broker] Close RateLimiter instance (#5155)
new f19d2f5 Fix exception type check order bug (#5174)
new 10bc14f [pulsar-broker] Fix bug that message delivery stops after resetting cursor for failover subscription (#5185)
new 1775513 Get schema info with topic partition (#4751)
new 45eb10a Add schema admin api get schema info with schema version (#4877)
new 85a5b18 Modify rest api post schema compatibility strategy (#4950)
new 97627f2 Process requests asynchronously on some REST APIs (1) (#4765)
new f24c1b0 Process requests asynchronously on some REST APIs (2) (#4778)
new e52b15b [Issue #4756][broker] Process requests asynchronously on some REST APIs (3) (#4795)
new 0481904 Add more config for auto-topic-creation (#4963)
new d546f67 Ensure getting list of topics for namespace is handled asynchronously (#5188)
new 850b123 Fix bug that fails to search namespace bundle due to NPE (#5191)
new 90ec8e1 Make some member variables of Dispatcher volatile (#5193)
new 73abddf Added missing enum value KeyShared on python wrapper (#5196)
new e9c0195 Ensure consumer background tasks are cancelled after subscribe failures (#5204)
new 1168709 Upgrade dependencies for security fixes (#5232)
new 0150b1f Bug in Message Deduplication that may cause incorrect client/broker interaction (#5243)
new 34c645c [Issue #5234][pulsar-client-cpp] Fix memory leak caused by deadline_timer holding object reference (#5246)
new aec7cc0 Fix client backoff (#5261)
new a4a742d Fix Cmake to build _pulsar for osx (#5263)
new f6d78b3 Improve error handling logic for effectively once (#5271)
new e70924e fix behavior when getting a key from functions state that doesn't exist (#5272)
new e12ac7f Fixed race condition while triggering message redelivery after an ack-timeout event (#5276)
new b0c6a5f [Java-Client] Fix producer blocked after send an over size message while batch enabled #5282
new 8b33ff2 Fix memory leak caused by not being executed ClientConnection destructor (#5286)
new 63b09e7 [pulsar-broker] Close previous dispatcher when subscription type changes (#5288)
new 85af2f0 Don't attempt to append on read-only cursor ledger (#5297)
new 8ca502c [pulsar-broker] Fix: invalidate cache on zk-cache timeout (#5298)
new 2c32db6 [doc] Add a documentation page for metrics reference (#4910)
new 6e02e9c Doc release 2.4.0 (#4666)
new 37d7de0 update website for 2.4.1 release (#5080)
new 5c614d8 fix typo in metrics (#5299)
new 4f55878 [pulsar-proxy] add debug log + fix thread-factory name (#5302)
new dcc7a93 Fix bad_weak_ptr error when closing producer (#5315)
new ce8fc4f Merge Request for #4809: provide a convenient method for C++ client producer batch container (#4885)
new f08c936 [Issue: 3161][Go client] Return Message ID for send (#4811)
new 37c093e [Cpp] Fixed negative ack tracker constructor sequence (#5453)
new c89fb48 Allow to specify delivery delay in C++ client (#5317)
new a358009 [pulsar-broker] Disable stickyRead by default (#5321)
new a4a8eeb Fix bk write failure part 2 (#5322)
new edbb353 Fix instability in Pulsar Function window integration test (#5337)
new 4a6f1d3 If cursor is not durable, close dispatcher when all consumers are removed from subscription (#5340)
new df6a902 [TIEREDSTORAGE] Don't require both region and endpoint to be specified (#5355)
new 10a4501 [functions][Issue:5350]Fix pulsar can't load the customized SerDe (#5357)
new 0884d7c Set default ensemble size to 2 in service conf, to match broker.conf (#5359)
new 381fd6d [proxy] Fix proxy to be able to re-send request body (#5361)
new dedbe71 fix potential deadlock that can occur in addConsumer (#5371)
new d92da4e make skip all messages async. (#5375)
new ae864d5 Trim messages which less than mark delete position for message redelivery (#5378)
new d126b1d [pulsar-broker] Fix: race condition : Failed to read-more entries on dispatcher (#5391)
new 7e06034 [pulsar-broker] Fix bug that namespace policies does not take effect due to NPE (#5408)
new f222dad Revert "Add default loader for latest pyyaml (#4974)" (#5432)
new 370bfc3 Fix couple functions related integration tests (#5434)
new 6b3c7b7 Add support for partitioned topic consumer seek by time. (#5435)
new b7522df [pulsar-perf] Introduce number of threads in perf producer program (#5036)
new e11f195 [pulsar-perf] record message failure and avoid exiting from process on publish failure (#5441)
new f12d97c [pulsar-client] Avoid leak on publish failure on batch message (#5442)
new 5716cce [pulsar-client] Fix message corruption on OOM for batch messages (#5443)
new c7fcf8a [pulsar-client] print unknown exception stacktrace (#5444)
new 7d1c6ee Fixed StringSchema static initialization (#5445)
new 4f247a4 [pulsar-broker] fix broken custom auth-provider that uses authenticationData (#5462)
new 7914eb3 make some method of ManagedLedger public (#5472)
new bd0752e Fix wrongly report "3600 messages have timed-out" (#5477)
new 53ac344 Rewrite portions of retention/message expiry cookbook (#4780)
new 6162a71 [Issue 5309][docs] Clarify wording around retention policies (#5482)
new 153d190 Add is_read_compacted to create_reader() in python API (#5483)
new 4094d45 Python API: fix reader_listener option (#5487)
new e30bbd7 [docs] Add thread/process modes in Configure Functions runtime (#4828)
new b1bf579 [site2] fix Function classname importing (#4924)
new e1d9295 [docs] Update some links on Pulsar website (#5492)
new 2dba8b6 Efficiency improvements for delay delivery tracker (#5498)
new 297c606 Trim deleted entries after recover cursor. (#4987)
new 203b0fd Fixed dispatcher skipping delivery of a batch during concurrent replays (#5499)
new fcb7a5d Return after triggering callback with empty result (#5500)
new 9a9add7 Fix list non-persistent topics shows the persistent topics (#5502)
new 9671090 fix: Functions unnecessarily restart during FunctionRuntimeManager init phase (#5527)
new 9c275a6 Add subscribe position param for consumer of sink (#5532)
new 6b8591c fix repeated initialization of connectorsManager (#5545)
new 68f351b Fix docs about reset cursor (#5551)
new 8ff212f Fix protobuf schema of java function error (#5569)
new 663d24f [Issue 5575] fix typo in BrokersBase file (#5596)
new 1fae168 Fix: don't set interrupt flag again after catching interrupt exception in Pulsar Client (#5643)
new e935a4b Update client-libraries-go.md (#5382)
new d946a8f [Docs] Fix the go client docs missing in website (#5595)
new b190874 Data is not deleted after expiration due to connected readers (#5621)
new c414068 Format code for branch-2.4.2
new d4e953f expose getLastMessageId method in ConsumerImpl (#4911)
new fed5252 Release 2.4.2
new 59cf167 Update logic for picking active consumer for failover subscription on non-partitioned topic. (#4604)
The 106 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
bin/pulsar | 2 +-
buildtools/pom.xml | 2 +-
conf/broker.conf | 6 +
conf/log4j2.yaml | 3 +-
conf/standalone.conf | 9 +
dashboard/pom.xml | 2 +-
distribution/io/pom.xml | 2 +-
distribution/io/src/assemble/io.xml | 3 +
distribution/offloaders/pom.xml | 2 +-
distribution/pom.xml | 2 +-
distribution/server/pom.xml | 2 +-
distribution/server/src/assemble/LICENSE.bin.txt | 49 +-
docker/grafana/pom.xml | 2 +-
docker/pom.xml | 2 +-
docker/pulsar-all/pom.xml | 2 +-
docker/pulsar-standalone/pom.xml | 2 +-
docker/pulsar/pom.xml | 2 +-
docker/pulsar/scripts/gen-yml-from-env.py | 2 +-
examples/flink/pom.xml | 2 +-
examples/pom.xml | 2 +-
examples/spark/pom.xml | 2 +-
jclouds-shaded/pom.xml | 13 +-
managed-ledger-shaded/pom.xml | 2 +-
managed-ledger/pom.xml | 2 +-
.../apache/bookkeeper/mledger/ManagedCursor.java | 21 +
.../apache/bookkeeper/mledger/ManagedLedger.java | 5 +
.../bookkeeper/mledger/impl/EntryCacheImpl.java | 6 +
.../mledger/impl/ManagedCursorContainer.java | 2 +-
.../bookkeeper/mledger/impl/ManagedCursorImpl.java | 55 +-
.../bookkeeper/mledger/impl/ManagedLedgerImpl.java | 70 +-
.../mledger/impl/NonDurableCursorImpl.java | 8 +-
.../apache/bookkeeper/mledger/impl/OpAddEntry.java | 2 +-
.../mledger/impl/EntryCacheManagerTest.java | 9 +-
.../mledger/impl/ManagedCursorConcurrencyTest.java | 14 +-
.../mledger/impl/ManagedCursorContainerTest.java | 29 +-
.../bookkeeper/mledger/impl/ManagedCursorTest.java | 119 +-
.../mledger/impl/ManagedLedgerErrorsTest.java | 56 +-
.../impl/ManagedLedgerSingleBookieTest.java | 6 +-
.../mledger/impl/ManagedLedgerTerminationTest.java | 10 +-
.../bookkeeper/mledger/impl/ManagedLedgerTest.java | 63 +-
.../mledger/impl/NonDurableCursorTest.java | 72 +-
.../mledger/impl/OffloadLedgerDeleteTest.java | 1 -
.../bookkeeper/mledger/impl/OffloadPrefixTest.java | 232 +--
.../bookkeeper/mledger/impl/PositionTest.java | 8 +-
.../bookkeeper/mledger/util/RangeCacheTest.java | 21 +-
.../java/org/apache/zookeeper/MockZooKeeper.java | 81 +-
pom.xml | 11 +-
protobuf-shaded/pom.xml | 4 +-
pulsar-broker-auth-athenz/pom.xml | 2 +-
pulsar-broker-auth-sasl/pom.xml | 2 +-
.../authentication/SaslAuthenticateTest.java | 20 +-
pulsar-broker-common/pom.xml | 2 +-
.../apache/pulsar/broker/ServiceConfiguration.java | 21 +-
pulsar-broker-shaded/pom.xml | 2 +-
pulsar-broker/pom.xml | 2 +-
.../org/apache/pulsar/PulsarBrokerStarter.java | 2 +-
.../java/org/apache/pulsar/PulsarStandalone.java | 2 +-
.../pulsar/broker/BookKeeperClientFactoryImpl.java | 44 +-
.../org/apache/pulsar/broker/PulsarService.java | 3 +-
.../apache/pulsar/broker/admin/AdminResource.java | 148 +-
.../pulsar/broker/admin/impl/BrokersBase.java | 9 +-
.../pulsar/broker/admin/impl/NamespacesBase.java | 266 ++--
.../broker/admin/impl/PersistentTopicsBase.java | 753 +++++++---
.../apache/pulsar/broker/admin/v1/Namespaces.java | 93 +-
.../broker/admin/v1/NonPersistentTopics.java | 90 +-
.../pulsar/broker/admin/v1/PersistentTopics.java | 155 +-
.../org/apache/pulsar/broker/admin/v2/Bookies.java | 23 +-
.../apache/pulsar/broker/admin/v2/Namespaces.java | 104 +-
.../broker/admin/v2/NonPersistentTopics.java | 130 +-
.../pulsar/broker/admin/v2/PersistentTopics.java | 114 +-
.../pulsar/broker/admin/v2/SchemasResource.java | 276 +++-
.../delayed/InMemoryDelayedDeliveryTracker.java | 12 +-
.../broker/loadbalance/impl/LoadManagerShared.java | 66 +-
.../loadbalance/impl/ModularLoadManagerImpl.java | 18 +-
.../loadbalance/impl/SimpleLoadManagerImpl.java | 15 +-
.../pulsar/broker/namespace/NamespaceService.java | 60 +-
.../broker/service/AbstractBaseDispatcher.java | 4 +
.../AbstractDispatcherMultipleConsumers.java | 6 +-
.../AbstractDispatcherSingleActiveConsumer.java | 66 +-
.../pulsar/broker/service/BrokerService.java | 5 +-
.../org/apache/pulsar/broker/service/Consumer.java | 30 +-
.../apache/pulsar/broker/service/Dispatcher.java | 8 +
.../apache/pulsar/broker/service/ServerCnx.java | 42 +-
.../NonPersistentDispatcherMultipleConsumers.java | 6 +
...onPersistentDispatcherSingleActiveConsumer.java | 16 +-
.../nonpersistent/NonPersistentSubscription.java | 41 +-
.../service/persistent/MessageDeduplication.java | 75 +-
.../PersistentDispatcherMultipleConsumers.java | 107 +-
.../PersistentDispatcherSingleActiveConsumer.java | 30 +-
.../service/persistent/PersistentReplicator.java | 29 +
.../service/persistent/PersistentSubscription.java | 80 +-
.../broker/service/persistent/PersistentTopic.java | 185 ++-
.../schema/DefaultSchemaRegistryService.java | 10 +
.../broker/service/schema/SchemaRegistry.java | 4 +
.../service/schema/SchemaRegistryService.java | 1 +
.../service/schema/SchemaRegistryServiceImpl.java | 16 +-
...hemaRegistryServiceWithSchemaDataValidator.java | 10 +
.../pulsar/broker/stats/prometheus/TopicStats.java | 4 +-
.../apache/pulsar/client/impl/RawReaderImpl.java | 5 +-
.../org/apache/pulsar/PulsarBrokerStarterTest.java | 126 +-
.../broker/BookKeeperClientFactoryImplTest.java | 150 ++
.../broker/admin/AdminApiGetLastMessageIdTest.java | 6 -
.../pulsar/broker/admin/AdminApiSchemaTest.java | 74 +
.../admin/AdminApiSchemaValidationEnforced.java | 42 +-
.../apache/pulsar/broker/admin/AdminApiTest.java | 74 +-
.../apache/pulsar/broker/admin/AdminApiTest2.java | 11 +-
.../org/apache/pulsar/broker/admin/AdminTest.java | 28 +-
.../apache/pulsar/broker/admin/BookiesApiTest.java | 115 ++
.../apache/pulsar/broker/admin/NamespacesTest.java | 147 +-
.../pulsar/broker/admin/PersistentTopicsTest.java | 128 +-
.../pulsar/broker/admin/v1/V1_AdminApiTest.java | 7 +-
.../pulsar/broker/admin/v1/V1_AdminApiTest2.java | 5 +-
.../pulsar/broker/auth/AuthorizationTest.java | 154 +-
.../broker/auth/MockedPulsarServiceBaseTest.java | 1 +
.../delayed/InMemoryDeliveryTrackerTest.java | 47 +-
.../AntiAffinityNamespaceGroupTest.java | 19 +-
.../broker/loadbalance/LoadBalancerTest.java | 15 +-
.../loadbalance/ModularLoadManagerImplTest.java | 1 -
.../loadbalance/SimpleLoadManagerImplTest.java | 14 +-
.../loadbalance/impl/LoadManagerSharedTest.java | 86 ++
.../broker/lookup/http/HttpTopicLookupv2Test.java | 1 -
.../broker/namespace/NamespaceServiceTest.java | 14 +-
.../broker/namespace/OwnershipCacheTest.java | 22 +-
.../broker/service/BacklogQuotaManagerTest.java | 43 +-
.../pulsar/broker/service/BatchMessageTest.java | 32 +-
.../broker/service/BrokerBkEnsemblesTests.java | 1 +
.../broker/service/BrokerBookieIsolationTest.java | 5 +-
.../BrokerServiceAutoTopicCreationTest.java | 120 ++
.../pulsar/broker/service/BrokerServiceTest.java | 16 +
.../service/BrokerServiceThrottlingTest.java | 2 -
.../pulsar/broker/service/PeerReplicatorTest.java | 3 +-
.../PersistentDispatcherFailoverConsumerTest.java | 90 +-
.../broker/service/PersistentFailoverE2ETest.java | 66 +-
.../service/PersistentMessageFinderTest.java | 10 +-
.../service/PersistentTopicConcurrentTest.java | 13 +-
.../broker/service/PersistentTopicE2ETest.java | 6 +-
.../pulsar/broker/service/PersistentTopicTest.java | 72 +-
.../pulsar/broker/service/ReplicatorTest.java | 4 +-
.../pulsar/broker/service/ReplicatorTestBase.java | 3 +
.../pulsar/broker/service/ResendRequestTest.java | 7 +-
.../pulsar/broker/service/ServerCnxTest.java | 17 +-
.../broker/service/SubscriptionSeekTest.java | 57 +-
.../service/persistent/DelayedDeliveryTest.java | 68 +-
.../service/persistent/MessageDuplicationTest.java | 322 +++++
.../persistent/PersistentSubscriptionTest.java | 2 +-
.../broker/service/schema/SchemaServiceTest.java | 35 +
.../stats/BookieClientsStatsGeneratorTest.java | 4 +-
.../pulsar/broker/stats/PrometheusMetricsTest.java | 5 +-
.../zookeeper/ZooKeeperClientAspectJTest.java | 3 +-
.../pulsar/client/api/BrokerServiceLookupTest.java | 6 +-
.../client/api/ClientDeduplicationFailureTest.java | 418 ++++++
.../apache/pulsar/client/api/ClientErrorsTest.java | 2 +-
.../client/api/MessageDispatchThrottlingTest.java | 46 +
.../pulsar/client/api/NonPersistentTopicTest.java | 3 +
.../api/PartitionedProducerConsumerTest.java | 4 +-
.../client/api/SimpleProducerConsumerStatTest.java | 3 +-
.../client/api/SimpleProducerConsumerTest.java | 12 +-
.../SubscriptionMessageDispatchThrottlingTest.java | 47 +
.../client/api/v1/V1_ProducerConsumerTest.java | 17 +-
.../client/impl/BrokerClientIntegrationTest.java | 45 +-
.../pulsar/client/impl/MessageRedeliveryTest.java | 43 +-
.../client/impl/PatternTopicsConsumerImplTest.java | 12 +-
.../pulsar/client/impl/TopicsConsumerImplTest.java | 109 +-
.../impl/UnAcknowledgedMessagesTimeoutTest.java | 7 +-
.../pulsar/common/naming/NamespaceBundleTest.java | 23 +-
.../pulsar/common/naming/NamespaceBundlesTest.java | 10 +-
.../worker/PulsarFunctionE2ESecurityTest.java | 5 +-
.../worker/PulsarFunctionLocalRunTest.java | 16 +-
.../worker/PulsarFunctionPublishTest.java | 5 +-
.../functions/worker/PulsarFunctionStateTest.java | 45 +-
.../worker/PulsarWorkerAssignmentTest.java | 2 +-
.../apache/pulsar/io/PulsarFunctionAdminTest.java | 3 +-
.../apache/pulsar/io/PulsarFunctionE2ETest.java | 8 +-
.../websocket/proxy/ProxyAuthorizationTest.java | 28 +-
.../websocket/proxy/ProxyPublishConsumeTest.java | 90 +-
.../proxy/ProxyPublishConsumeTlsTest.java | 2 -
.../configurations/pulsar_broker_test.conf | 3 +
pulsar-client-1x-base/pom.xml | 2 +-
pulsar-client-1x-base/pulsar-client-1x/pom.xml | 2 +-
.../pulsar/client/impl/v1/ConsumerV1Impl.java | 7 +
.../pulsar-client-2x-shaded/pom.xml | 2 +-
pulsar-client-admin-shaded/pom.xml | 2 +-
pulsar-client-admin/pom.xml | 2 +-
.../org/apache/pulsar/client/admin/Namespaces.java | 62 +-
.../org/apache/pulsar/client/admin/Schemas.java | 60 +
.../client/admin/internal/NamespacesImpl.java | 94 +-
.../pulsar/client/admin/internal/SchemasImpl.java | 137 +-
pulsar-client-all/pom.xml | 2 +-
pulsar-client-api/pom.xml | 2 +-
.../apache/pulsar/client/api/BatcherBuilder.java | 4 +-
.../org/apache/pulsar/client/api/Consumer.java | 14 +
.../pulsar/client/api/PulsarClientException.java | 5 +-
.../client/internal/DefaultImplementation.java | 40 +
.../common/schema/SchemaInfoWithVersion.java | 17 +-
pulsar-client-auth-athenz/pom.xml | 2 +-
.../client/impl/auth/AuthenticationAthenzTest.java | 7 +-
pulsar-client-auth-sasl/pom.xml | 2 +-
pulsar-client-cpp/CMakeLists.txt | 8 +-
pulsar-client-cpp/examples/SampleAsyncProducer.cc | 4 +-
.../include/pulsar/ConsumerConfiguration.h | 6 +
pulsar-client-cpp/include/pulsar/MessageBuilder.h | 16 +
.../include/pulsar/ProducerConfiguration.h | 2 +-
pulsar-client-cpp/include/pulsar/c/message.h | 15 +
pulsar-client-cpp/include/pulsar/c/producer.h | 2 +-
pulsar-client-cpp/lib/BatchMessageContainer.cc | 9 +-
pulsar-client-cpp/lib/BatchMessageContainer.h | 6 +-
pulsar-client-cpp/lib/ClientConnection.cc | 23 +-
pulsar-client-cpp/lib/ClientImpl.cc | 1 +
pulsar-client-cpp/lib/ConnectionPool.cc | 13 +
pulsar-client-cpp/lib/ConnectionPool.h | 2 +
pulsar-client-cpp/lib/ConsumerConfiguration.cc | 10 +-
pulsar-client-cpp/lib/ConsumerConfigurationImpl.h | 3 +-
pulsar-client-cpp/lib/ConsumerImpl.cc | 13 +-
pulsar-client-cpp/lib/ConsumerImpl.h | 2 +-
pulsar-client-cpp/lib/HandlerBase.cc | 11 +-
pulsar-client-cpp/lib/HandlerBase.h | 3 -
pulsar-client-cpp/lib/MessageBuilder.cc | 11 +
pulsar-client-cpp/lib/NegativeAcksTracker.cc | 18 +-
pulsar-client-cpp/lib/NegativeAcksTracker.h | 2 -
pulsar-client-cpp/lib/PartitionedConsumerImpl.cc | 2 +-
pulsar-client-cpp/lib/PartitionedProducerImpl.cc | 2 +-
pulsar-client-cpp/lib/Producer.cc | 10 +-
pulsar-client-cpp/lib/ProducerImpl.cc | 84 +-
pulsar-client-cpp/lib/ProducerImpl.h | 9 +-
.../lib/TimeUtils.cc | 16 +-
.../lib/TimeUtils.h | 24 +-
pulsar-client-cpp/lib/c/c_Message.cc | 8 +
pulsar-client-cpp/lib/c/c_Producer.cc | 16 +-
pulsar-client-cpp/perf/PerfProducer.cc | 5 +-
pulsar-client-cpp/python/pulsar/__init__.py | 9 +-
pulsar-client-cpp/python/src/config.cc | 2 +-
pulsar-client-cpp/python/src/enums.cc | 1 +
pulsar-client-cpp/python/src/producer.cc | 4 +-
pulsar-client-cpp/test-conf/standalone-ssl.conf | 9 +
pulsar-client-cpp/test-conf/standalone.conf | 9 +
pulsar-client-cpp/tests/AuthPluginTest.cc | 7 +-
pulsar-client-cpp/tests/BasicEndToEndTest.cc | 107 +-
pulsar-client-cpp/tests/BatchMessageTest.cc | 14 +-
pulsar-client-cpp/tests/ProducerTest.cc | 8 +-
pulsar-client-cpp/tests/standalone.conf | 9 +
.../sendWithMsgID/producer.go} | 41 +-
pulsar-client-go/pulsar/c_go_pulsar.h | 7 +
pulsar-client-go/pulsar/c_message.go | 6 +
pulsar-client-go/pulsar/c_producer.go | 67 +
pulsar-client-go/pulsar/producer.go | 13 +
pulsar-client-go/pulsar/producer_test.go | 28 +
pulsar-client-go/pulsar/testhelps.go | 10 +
pulsar-client-kafka-compat/pom.xml | 2 +-
.../pulsar-client-kafka-shaded/pom.xml | 2 +-
.../pulsar-client-kafka-tests/pom.xml | 2 +-
.../pulsar-client-kafka/pom.xml | 2 +-
pulsar-client-shaded/pom.xml | 2 +-
pulsar-client-tools-test/pom.xml | 2 +-
.../apache/pulsar/admin/cli/CmdFunctionsTest.java | 23 +
pulsar-client-tools/pom.xml | 2 +-
.../org/apache/pulsar/admin/cli/CmdFunctions.java | 5 +-
.../org/apache/pulsar/admin/cli/CmdSchemas.java | 7 +-
.../java/org/apache/pulsar/admin/cli/CmdSinks.java | 8 +
.../org/apache/pulsar/admin/cli/CmdTopics.java | 2 +-
pulsar-client/pom.xml | 2 +-
.../org/apache/pulsar/client/impl/Backoff.java | 32 +-
.../apache/pulsar/client/impl/BackoffBuilder.java | 16 +-
.../client/impl/BatchMessageContainerImpl.java | 36 +-
.../client/impl/BatchMessageKeyBasedContainer.java | 31 +-
.../client/impl/BinaryProtoLookupService.java | 2 -
.../pulsar/client/impl/ClientBuilderImpl.java | 2 +-
.../apache/pulsar/client/impl/ConsumerBase.java | 29 +-
.../apache/pulsar/client/impl/ConsumerImpl.java | 72 +-
.../pulsar/client/impl/DefaultBatcherBuilder.java | 2 +
.../pulsar/client/impl/HttpLookupService.java | 3 +-
.../pulsar/client/impl/KeyBasedBatcherBuilder.java | 2 +
.../pulsar/client/impl/MultiMessageIdImpl.java | 111 ++
.../client/impl/MultiTopicsConsumerImpl.java | 50 +-
.../PersistentAcknowledgmentsGroupingTracker.java | 2 +-
.../apache/pulsar/client/impl/ProducerImpl.java | 21 +-
.../pulsar/client/impl/PulsarClientImpl.java | 4 +-
.../org/apache/pulsar/client/impl/ReaderImpl.java | 5 +-
.../pulsar/client/impl/UnAckedMessageTracker.java | 2 +-
.../pulsar/client/impl/ZeroQueueConsumerImpl.java | 2 +-
.../client/impl/conf/ClientConfigurationData.java | 4 +-
.../pulsar/client/impl/schema/SchemaUtils.java | 69 +-
.../pulsar/client/impl/schema/StringSchema.java | 36 +-
.../apache/pulsar/client/api/ConsumerIdTest.java | 20 +-
.../org/apache/pulsar/client/impl/BackoffTest.java | 4 +-
.../pulsar/client/impl/BatchMessageIdImplTest.java | 40 +-
.../apache/pulsar/client/impl/BuildersTest.java | 24 +-
.../client/impl/ClientInitializationTest.java | 3 +-
.../pulsar/client/impl/ConnectionTimeoutTest.java | 2 -
.../pulsar/client/impl/ConsumerImplTest.java | 23 +-
.../pulsar/client/impl/MessageIdCompareToTest.java | 246 +++-
.../client/impl/TypedMessageBuilderImplTest.java | 6 +-
.../impl/conf/ConfigurationDataUtilsTest.java | 8 +-
.../client/impl/schema/DefaultSchemasTest.java | 4 +-
.../client/impl/schema/DoubleSchemaTest.java | 2 -
.../client/impl/schema/KeyValueSchemaInfoTest.java | 18 +-
.../pulsar/client/impl/schema/SchemaTestUtils.java | 11 +
pulsar-common/pom.xml | 22 +-
.../client/api/url/DataURLStreamHandler.java | 6 +
.../api/url/PulsarURLStreamHandlerFactory.java | 4 +
.../java/org/apache/pulsar/client/api/url/URL.java | 3 +
.../api/url/package-info.java} | 9 +-
.../common/allocator/PulsarByteBufAllocator.java | 3 +
.../package-info.java} | 9 +-
.../pulsar/common/api/raw/MessageParser.java | 20 +-
.../apache/pulsar/common/api/raw/RawMessage.java | 18 +-
.../apache/pulsar/common/api/raw/RawMessageId.java | 3 +
.../common/api/raw/ReferenceCountedObject.java | 5 +
.../raw/{RawMessageId.java => package-info.java} | 9 +-
.../common/compression/CompressionCodec.java | 9 +-
.../common/compression/CompressionCodecLZ4.java | 8 +-
.../common/compression/CompressionCodecNone.java | 6 +-
.../compression/CompressionCodecProvider.java | 9 +
.../common/compression/CompressionCodecSnappy.java | 7 +-
.../common/compression/CompressionCodecZLib.java | 2 +-
.../common/compression/CompressionCodecZstd.java | 2 +-
.../package-info.java} | 9 +-
.../common/conf/InternalConfigurationData.java | 3 +
.../RawMessageId.java => conf/package-info.java} | 9 +-
.../pulsar/common/functions/ConsumerConfig.java | 9 +-
.../pulsar/common/functions/FunctionConfig.java | 25 +-
.../pulsar/common/functions/FunctionState.java | 15 +-
.../apache/pulsar/common/functions/Resources.java | 18 +-
.../pulsar/common/functions/UpdateOptions.java | 4 +-
.../org/apache/pulsar/common/functions/Utils.java | 15 +-
.../pulsar/common/functions/WindowConfig.java | 8 +-
.../apache/pulsar/common/functions/WorkerInfo.java | 5 +-
.../package-info.java} | 9 +-
.../pulsar/common/io/ConnectorDefinition.java | 15 +-
.../org/apache/pulsar/common/io/SinkConfig.java | 17 +-
.../org/apache/pulsar/common/io/SourceConfig.java | 17 +-
.../raw/RawMessageId.java => io/package-info.java} | 9 +-
.../pulsar/common/lookup/data/LookupData.java | 23 +-
.../data/package-info.java} | 9 +-
.../org/apache/pulsar/common/naming/Constants.java | 3 +
.../org/apache/pulsar/common/naming/Metadata.java | 3 +
.../apache/pulsar/common/naming/NamespaceName.java | 12 +-
.../apache/pulsar/common/naming/ServiceUnitId.java | 13 +-
.../apache/pulsar/common/naming/TopicDomain.java | 3 +
.../org/apache/pulsar/common/naming/TopicName.java | 48 +-
.../RawMessageId.java => naming/package-info.java} | 9 +-
.../org/apache/pulsar/common/nar/FileUtils.java | 10 +-
.../apache/pulsar/common/nar/NarClassLoader.java | 8 +-
.../org/apache/pulsar/common/nar/NarUnpacker.java | 5 +-
.../RawMessageId.java => nar/package-info.java} | 9 +-
.../RawMessageId.java => net/package-info.java} | 9 +-
.../common/partition/PartitionedTopicMetadata.java | 3 +
.../package-info.java} | 9 +-
.../pulsar/common/policies/AutoFailoverPolicy.java | 7 +-
.../common/policies/NamespaceIsolationPolicy.java | 23 +-
.../pulsar/common/policies/data/AuthAction.java | 8 +-
.../pulsar/common/policies/data/AuthPolicies.java | 9 +-
.../policies/data/AutoFailoverPolicyData.java | 8 +-
.../policies/data/AutoFailoverPolicyType.java | 7 +-
.../pulsar/common/policies/data/BacklogQuota.java | 21 +-
.../policies/data/BookieAffinityGroupData.java | 3 +
.../pulsar/common/policies/data/BookieInfo.java | 3 +
.../policies/data/BookiesRackConfiguration.java | 21 +-
.../common/policies/data/BrokerAssignment.java | 3 +
.../data/BrokerNamespaceIsolationData.java | 6 +-
.../pulsar/common/policies/data/BrokerStatus.java | 3 +
.../pulsar/common/policies/data/BundlesData.java | 6 +-
.../pulsar/common/policies/data/ClusterData.java | 6 +-
.../pulsar/common/policies/data/ConsumerStats.java | 31 +-
.../pulsar/common/policies/data/DispatchRate.java | 6 +-
.../pulsar/common/policies/data/ErrorData.java | 3 +
.../common/policies/data/ExceptionInformation.java | 3 +
.../pulsar/common/policies/data/FailureDomain.java | 3 +
.../pulsar/common/policies/data/FunctionStats.java | 110 +-
.../common/policies/data/FunctionStatus.java | 14 +-
.../pulsar/common/policies/data/LocalPolicies.java | 3 +
.../policies/data/NamespaceIsolationData.java | 7 +-
.../policies/data/NamespaceOwnershipStatus.java | 6 +
.../policies/data/NonPersistentPublisherStats.java | 3 +-
.../data/NonPersistentReplicatorStats.java | 5 +-
.../data/NonPersistentSubscriptionStats.java | 5 +-
.../policies/data/NonPersistentTopicStats.java | 18 +-
.../data/PartitionedTopicInternalStats.java | 7 +-
.../policies/data/PartitionedTopicStats.java | 7 +-
.../common/policies/data/PersistencePolicies.java | 6 +-
.../policies/data/PersistentOfflineTopicStats.java | 29 +-
.../data/PersistentTopicInternalStats.java | 12 +-
.../common/policies/data/PersistentTopicStats.java | 2 +-
.../pulsar/common/policies/data/Policies.java | 28 +-
.../common/policies/data/PublisherStats.java | 25 +-
.../common/policies/data/ReplicatorStats.java | 27 +-
.../pulsar/common/policies/data/ResourceQuota.java | 54 +-
.../common/policies/data/RetentionPolicies.java | 1 +
.../SchemaAutoUpdateCompatibilityStrategy.java | 2 +-
.../pulsar/common/policies/data/SinkStatus.java | 14 +-
.../pulsar/common/policies/data/SourceStatus.java | 14 +-
.../pulsar/common/policies/data/SubscribeRate.java | 3 +
.../common/policies/data/SubscriptionAuthMode.java | 6 +-
.../common/policies/data/SubscriptionStats.java | 29 +-
.../pulsar/common/policies/data/TenantInfo.java | 7 +-
.../pulsar/common/policies/data/TopicStats.java | 33 +-
.../policies/data/WorkerFunctionInstanceStats.java | 8 +-
.../data/package-info.java} | 9 +-
.../policies/impl/AutoFailoverPolicyFactory.java | 3 +
.../common/policies/impl/MinAvailablePolicy.java | 9 +-
.../policies/impl/NamespaceIsolationPolicies.java | 39 +-
.../impl/NamespaceIsolationPolicyImpl.java | 25 +-
.../impl/package-info.java} | 9 +-
.../package-info.java} | 9 +-
.../apache/pulsar/common/protocol/ByteBufPair.java | 8 +-
.../pulsar/common/protocol/CommandUtils.java | 6 +-
.../apache/pulsar/common/protocol/Commands.java | 56 +-
.../org/apache/pulsar/common/protocol/Markers.java | 9 +-
.../pulsar/common/protocol/PulsarDecoder.java | 7 +-
.../pulsar/common/protocol/PulsarHandler.java | 9 +-
.../package-info.java} | 9 +-
.../common/protocol/schema/BytesSchemaVersion.java | 22 +-
.../protocol/schema/DeleteSchemaResponse.java | 3 +
.../common/protocol/schema/EmptyVersion.java | 3 +
...onse.java => GetAllVersionsSchemaResponse.java} | 8 +-
.../common/protocol/schema/GetSchemaResponse.java | 3 +
...aResponse.java => IsCompatibilityResponse.java} | 9 +-
.../common/protocol/schema/LatestVersion.java | 3 +
...esponse.java => LongSchemaVersionResponse.java} | 7 +-
.../common/protocol/schema/PostSchemaPayload.java | 3 +
.../common/protocol/schema/PostSchemaResponse.java | 3 +
.../pulsar/common/protocol/schema/SchemaData.java | 5 +-
.../common/protocol/schema/SchemaInfoUtil.java | 7 +-
.../common/protocol/schema/SchemaVersion.java | 3 +
.../schema/package-info.java} | 9 +-
.../common/sasl/JAASCredentialsContainer.java | 10 +-
.../apache/pulsar/common/sasl/KerberosName.java | 51 +-
.../pulsar/common/sasl/TGTRefreshThread.java | 5 +-
.../RawMessageId.java => sasl/package-info.java} | 9 +-
.../apache/pulsar/common/stats/AllocatorStats.java | 15 +
.../common/stats/JvmDefaultGCMetricsLogger.java | 9 +-
.../pulsar/common/stats/JvmG1GCMetricsLogger.java | 3 +
.../pulsar/common/stats/JvmGCMetricsLogger.java | 6 +-
.../org/apache/pulsar/common/stats/JvmMetrics.java | 12 +-
.../org/apache/pulsar/common/stats/Metrics.java | 19 +-
.../RawMessageId.java => stats/package-info.java} | 9 +-
.../common/util/ClientSslContextRefresher.java | 5 +-
.../apache/pulsar/common/util/DateFormatter.java | 2 +-
.../common/util/DefaultSslContextBuilder.java | 1 +
.../org/apache/pulsar/common/util/FieldParser.java | 56 +-
.../common/util/FileModifiedTimeUpdater.java | 7 +-
.../org/apache/pulsar/common/util/FutureUtil.java | 5 +-
.../java/org/apache/pulsar/common/util/Hash.java | 7 +-
.../apache/pulsar/common/util/KeyStoreHolder.java | 5 +
.../apache/pulsar/common/util/Murmur3_32Hash.java | 7 +-
.../util/NamespaceBundleStatsComparator.java | 1 -
.../pulsar/common/util/NettySslContextBuilder.java | 10 +-
.../apache/pulsar/common/util/NumberFormat.java | 3 +
.../pulsar/common/util/ObjectMapperFactory.java | 1 +
.../org/apache/pulsar/common/util/RateLimiter.java | 50 +-
.../pulsar/common/util/RelativeTimeUtil.java | 5 +-
.../apache/pulsar/common/util/SecurityUtility.java | 28 +-
.../common/util/SslContextAutoRefreshBuilder.java | 8 +-
.../common/util/collections/ConcurrentBitSet.java | 3 +
.../util/collections/ConcurrentLongHashMap.java | 20 +-
.../util/collections/ConcurrentLongPairSet.java | 44 +-
.../util/collections/ConcurrentOpenHashMap.java | 17 +-
.../util/collections/ConcurrentOpenHashSet.java | 19 +-
.../ConcurrentOpenLongPairRangeSet.java | 55 +-
.../collections/ConcurrentSortedLongPairSet.java | 9 +-
.../collections/GrowableArrayBlockingQueue.java | 7 +-
.../collections/GrowablePriorityLongPairQueue.java | 39 +-
.../common/util/collections/LongPairRangeSet.java | 95 +-
.../common/util/collections/LongPairSet.java | 34 +-
.../util/collections/TripleLongPriorityQueue.java | 26 +-
.../collections/package-info.java} | 9 +-
.../pulsar/common/util/netty/EventLoopUtil.java | 6 +-
.../netty/package-info.java} | 9 +-
.../RawMessageId.java => util/package-info.java} | 9 +-
.../util/protobuf/ByteBufCodedInputStream.java | 5 +-
.../util/protobuf/ByteBufCodedOutputStream.java | 5 +-
.../protobuf/package-info.java} | 9 +-
.../policies/data/loadbalancer/BrokerUsage.java | 14 +-
.../policies/data/loadbalancer/JSONWritable.java | 7 +-
.../policies/data/loadbalancer/JvmUsage.java | 2 +-
.../data/loadbalancer/LoadManagerReport.java | 43 +-
.../policies/data/loadbalancer/LoadReport.java | 13 +-
.../data/loadbalancer/LoadReportDeserializer.java | 9 +-
.../data/loadbalancer/LocalBrokerData.java | 12 +-
.../data/loadbalancer/NamespaceBundleStats.java | 18 +-
.../policies/data/loadbalancer/NamespaceUsage.java | 28 +-
.../data/loadbalancer/ResourceUnitRanking.java | 53 +-
.../policies/data/loadbalancer/ResourceUsage.java | 5 +-
.../data/loadbalancer/ServiceLookupData.java | 12 +-
.../data/loadbalancer/SystemResourceUsage.java | 4 +-
.../data/loadbalancer/package-info.java} | 9 +-
.../common/compression/CompressorCodecTest.java | 4 +-
.../pulsar/common/naming/NamespaceNameTest.java | 12 +-
.../apache/pulsar/common/naming/TopicNameTest.java | 9 +-
.../policies/data/AutoFailoverPolicyDataTest.java | 12 +-
.../common/policies/data/ClusterDataTest.java | 21 +-
.../common/policies/data/ConsumerStatsTest.java | 40 +-
.../common/policies/data/LocalPolicesTest.java | 11 +-
.../policies/data/NamespaceIsolationDataTest.java | 13 +-
.../policies/data/PersistencePoliciesTest.java | 9 +-
.../common/policies/data/PoliciesDataTest.java | 14 +-
.../common/policies/data/PublisherStatsTest.java | 58 +-
.../common/policies/data/RetentionPolicesTest.java | 13 +-
.../policies/impl/MinAvailablePolicyTest.java | 12 +-
.../impl/NamespaceIsolationPoliciesTest.java | 29 +-
.../impl/NamespaceIsolationPolicyImplTest.java | 40 +-
.../apache/pulsar/common/util/RateLimiterTest.java | 12 +-
.../collections/ConcurrentLongHashMapTest.java | 12 +-
.../collections/ConcurrentLongPairSetTest.java | 8 +-
.../collections/ConcurrentOpenHashMapTest.java | 18 +-
.../collections/ConcurrentOpenHashSetTest.java | 6 +-
.../ConcurrentOpenLongPairRangeSetTest.java | 40 +-
.../ConcurrentSortedLongPairSetTest.java | 6 +-
.../GrowableArrayBlockingQueueTest.java | 12 +-
.../GrowablePriorityLongPairQueueTest.java | 7 +-
.../collections/TripleLongPriorityQueueTest.java | 7 +-
pulsar-discovery-service/pom.xml | 2 +-
.../service/web/ZookeeperCacheLoaderTest.java | 1 -
pulsar-flink/pom.xml | 2 +-
.../serialization/AvroSerializationSchemaTest.java | 5 +-
.../pulsar/PulsarConsumerSourceTests.java | 51 +-
pulsar-functions/api-java/pom.xml | 2 +-
pulsar-functions/instance/pom.xml | 2 +-
.../pulsar/functions/instance/ContextImpl.java | 2 +-
.../functions/instance/JavaInstanceRunnable.java | 25 +-
.../functions/instance/state/StateContextImpl.java | 18 +-
.../apache/pulsar/functions/sink/PulsarSink.java | 2 +-
.../pulsar/functions/source/PulsarSource.java | 2 +
.../functions/source/PulsarSourceConfig.java | 2 +
.../pulsar/functions/source/TopicSchema.java | 18 +-
.../instance/state/StateContextImplTest.java | 12 +
.../pulsar/functions/sink/PulsarSinkTest.java | 10 +-
.../pulsar/functions/source/PulsarSourceTest.java | 12 +-
.../windowing/WindowFunctionExecutorTest.java | 1 -
pulsar-functions/java-examples/pom.xml | 2 +-
pulsar-functions/localrun-shaded/pom.xml | 2 +-
pulsar-functions/localrun/pom.xml | 2 +-
pulsar-functions/pom.xml | 2 +-
pulsar-functions/proto/pom.xml | 2 +-
.../proto/src/main/proto/Function.proto | 6 +
pulsar-functions/runtime-all/pom.xml | 2 +-
pulsar-functions/runtime/pom.xml | 2 +-
.../functions/runtime/KubernetesRuntime.java | 5 +-
.../pulsar/functions/runtime/ProcessRuntime.java | 11 +-
.../runtime/KubernetesRuntimeFactoryTest.java | 2 -
pulsar-functions/secrets/pom.xml | 2 +-
.../ClearTextSecretsProviderTest.java | 13 +-
.../EnvironmentBasedSecretsProviderTest.java | 11 +-
pulsar-functions/utils/pom.xml | 2 +-
.../pulsar/functions/utils/SinkConfigUtils.java | 7 +
.../pulsar/functions/utils/ReflectionsTest.java | 5 +-
pulsar-functions/worker/pom.xml | 2 +-
.../functions/worker/FunctionRuntimeManager.java | 61 +-
.../pulsar/functions/worker/WorkerService.java | 6 +-
.../functions/worker/FunctionActionerTest.java | 11 +-
.../worker/FunctionRuntimeManagerTest.java | 199 +--
.../functions/worker/MembershipManagerTest.java | 15 +-
.../worker/rest/api/FunctionsImplTest.java | 26 +-
pulsar-io/aerospike/pom.xml | 2 +-
pulsar-io/canal/pom.xml | 2 +-
pulsar-io/cassandra/pom.xml | 2 +-
.../pulsar/io/cassandra/CassandraAbstractSink.java | 3 +-
pulsar-io/common/pom.xml | 2 +-
.../apache/pulsar/io/common/IOConfigUtilsTest.java | 1 -
pulsar-io/core/pom.xml | 2 +-
pulsar-io/data-generator/pom.xml | 2 +-
pulsar-io/debezium/core/pom.xml | 2 +-
pulsar-io/debezium/mysql/pom.xml | 2 +-
pulsar-io/debezium/pom.xml | 2 +-
pulsar-io/debezium/postgres/pom.xml | 2 +-
pulsar-io/docs/pom.xml | 2 +-
pulsar-io/elastic-search/pom.xml | 2 +-
pulsar-io/file/pom.xml | 2 +-
pulsar-io/flume/pom.xml | 2 +-
.../node/TestAbstractConfigurationProvider.java | 83 +-
pulsar-io/hbase/pom.xml | 2 +-
pulsar-io/hdfs2/pom.xml | 2 +-
pulsar-io/hdfs3/pom.xml | 2 +-
pulsar-io/influxdb/pom.xml | 2 +-
pulsar-io/jdbc/pom.xml | 2 +-
.../apache/pulsar/io/jdbc/JdbcAbstractSink.java | 1 -
pulsar-io/kafka-connect-adaptor/pom.xml | 2 +-
.../io/kafka/connect/KafkaConnectSourceTest.java | 8 +-
pulsar-io/kafka/pom.xml | 2 +-
pulsar-io/kinesis/pom.xml | 2 +-
.../org/apache/pulsar/io/kinesis/UtilsTest.java | 45 +-
pulsar-io/mongo/pom.xml | 2 +-
pulsar-io/netty/pom.xml | 2 +-
pulsar-io/pom.xml | 2 +-
pulsar-io/rabbitmq/pom.xml | 2 +-
pulsar-io/redis/pom.xml | 2 +-
pulsar-io/solr/pom.xml | 2 +-
pulsar-io/twitter/pom.xml | 2 +-
pulsar-log4j2-appender/pom.xml | 2 +-
pulsar-proxy/pom.xml | 2 +-
.../pulsar/proxy/server/AdminProxyHandler.java | 43 +-
.../pulsar/proxy/server/LookupProxyHandler.java | 7 +-
.../apache/pulsar/proxy/server/ProxyService.java | 4 +-
.../org/apache/pulsar/proxy/server/ProxyTest.java | 1 -
pulsar-spark/pom.xml | 2 +-
.../pulsar/spark/SparkStreamingPulsarReceiver.java | 22 +-
pulsar-sql/pom.xml | 2 +-
pulsar-sql/presto-distribution/LICENSE | 49 +-
pulsar-sql/presto-distribution/pom.xml | 52 +-
pulsar-sql/presto-pulsar-plugin/pom.xml | 3 +-
pulsar-sql/presto-pulsar/pom.xml | 2 +-
pulsar-storm/pom.xml | 2 +-
.../java/org/apache/pulsar/storm/PulsarSpout.java | 4 +-
pulsar-testclient/pom.xml | 2 +-
.../pulsar/testclient/PerformanceProducer.java | 370 +++--
pulsar-transaction/buffer/pom.xml | 2 +-
pulsar-transaction/common/pom.xml | 2 +-
pulsar-transaction/coordinator/pom.xml | 2 +-
pulsar-transaction/pom.xml | 2 +-
pulsar-websocket/pom.xml | 2 +-
pulsar-zookeeper-utils/pom.xml | 2 +-
.../pulsar/zookeeper/GlobalZooKeeperCache.java | 2 +-
.../pulsar/zookeeper/LocalBookkeeperEnsemble.java | 37 +-
.../pulsar/zookeeper/LocalZooKeeperCache.java | 2 +-
.../zookeeper/ZkBookieRackAffinityMapping.java | 10 +-
.../apache/pulsar/zookeeper/ZooKeeperCache.java | 161 ++-
.../pulsar/zookeeper/ZooKeeperChildrenCache.java | 43 +-
.../pulsar/zookeeper/ZooKeeperDataCache.java | 32 +-
.../LocalZooKeeperConnectionServiceTest.java | 21 +-
.../zookeeper/ZkBookieRackAffinityMappingTest.java | 21 +-
...kIsolatedBookieEnsemblePlacementPolicyTest.java | 3 +-
.../pulsar/zookeeper/ZookeeperCacheTest.java | 88 +-
pulsar-zookeeper/pom.xml | 2 +-
site2/docs/client-libraries-go.md | 32 +-
site2/docs/concepts-overview.md | 2 +-
site2/docs/cookbooks-retention-expiry.md | 38 +-
site2/docs/deploy-bare-metal-multi-cluster.md | 2 +-
site2/docs/functions-api.md | 12 +-
site2/docs/functions-overview.md | 6 +-
site2/docs/functions-runtime.md | 4 +
site2/docs/getting-started-clients.md | 2 +-
site2/docs/getting-started-standalone.md | 2 +-
site2/docs/reference-configuration.md | 2 +
site2/docs/reference-metrics.md | 243 ++++
site2/docs/reference-pulsar-admin.md | 6 +-
site2/website/releases.json | 1 +
site2/website/sidebars.json | 3 +-
.../versioned_docs/version-2.4.0/adaptors-kafka.md | 263 ++++
.../version-2.4.0/admin-api-overview.md | 93 ++
.../version-2.4.0/administration-zk-bk.md | 330 +++++
.../version-2.4.0}/client-libraries-go.md | 3 +-
.../version-2.4.0/client-libraries-java.md | 666 +++++++++
.../version-2.4.0/concepts-messaging.md | 394 +++++
.../version-2.4.0/concepts-tiered-storage.md | 18 +
.../version-2.4.0/cookbooks-compaction.md | 132 ++
.../version-2.4.0/cookbooks-tiered-storage.md | 235 +++
.../version-2.4.0/deploy-bare-metal.md | 446 ++++++
.../version-2.4.0/developing-binary-protocol.md | 553 +++++++
.../versioned_docs/version-2.4.0/developing-cpp.md | 101 ++
.../versioned_docs/version-2.4.0}/functions-api.md | 3 +-
.../version-2.4.0/functions-debugging.md | 452 ++++++
.../version-2.4.0}/functions-overview.md | 3 +-
.../version-2.4.0/functions-quickstart.md | 390 +++++
.../version-2.4.0}/functions-runtime.md | 18 +-
.../version-2.4.0/functions-worker.md | 242 ++++
.../version-2.4.0}/getting-started-standalone.md | 3 +-
.../versioned_docs/version-2.4.0/io-cdc-canal.md | 175 +++
.../version-2.4.0/io-cdc-debezium.md | 261 ++++
.../versioned_docs/version-2.4.0/io-managing.md | 162 +++
.../versioned_docs/version-2.4.0/io-quickstart.md | 746 ++++++++++
.../version-2.4.0/reference-cli-tools.md | 699 +++++++++
.../version-2.4.0}/reference-configuration.md | 5 +-
.../version-2.4.0}/reference-pulsar-admin.md | 3 +-
.../version-2.4.0/security-kerberos.md | 325 +++++
.../version-2.4.0/security-tls-authentication.md | 149 ++
.../version-2.4.0/security-token-admin.md | 159 +++
.../version-2.4.0/sql-getting-started.md | 143 ++
.../admin-api-non-persistent-topics.md | 264 ++++
.../version-2.4.1/admin-api-overview.md | 89 ++
.../version-2.4.1/admin-api-partitioned-topics.md | 377 +++++
.../version-2.4.1/administration-dashboard.md | 60 +
.../version-2.4.1/administration-upgrade.md | 153 ++
.../version-2.4.1/client-libraries-java.md | 708 +++++++++
.../version-2.4.1/concepts-messaging.md | 415 ++++++
.../version-2.4.1/concepts-tiered-storage.md | 18 +
.../version-2.4.1/cookbooks-compaction.md | 127 ++
.../versioned_docs/version-2.4.1}/functions-api.md | 27 +-
.../versioned_docs/version-2.4.1/functions-cli.md | 196 +++
.../version-2.4.1/functions-debug.md | 448 ++++++
.../version-2.4.1/functions-deploy.md | 211 +++
.../version-2.4.1/functions-develop.md | 617 ++++++++
.../version-2.4.1/functions-overview-new.md | 190 +++
.../version-2.4.1}/functions-overview.md | 7 +-
.../version-2.4.1}/functions-runtime.md | 22 +-
.../version-2.4.1}/getting-started-standalone.md | 8 +-
.../versioned_docs/version-2.4.1/io-cdc-canal.md | 175 +++
.../version-2.4.1/io-cdc-debezium.md | 261 ++++
.../versioned_docs/version-2.4.1/io-connectors.md | 31 +
.../versioned_docs/version-2.4.1/io-debug.md | 329 +++++
.../versioned_docs/version-2.4.1/io-develop.md | 196 +++
.../versioned_docs/version-2.4.1/io-jdbc.md | 24 +
.../versioned_docs/version-2.4.1/io-netty.md | 148 ++
.../website/versioned_docs/version-2.4.1/io-use.md | 1505 ++++++++++++++++++++
.../version-2.4.1}/reference-configuration.md | 3 +-
.../version-2.4.1/reference-connector-admin.md | 601 ++++++++
.../version-2.4.1/reference-metrics.md | 244 ++++
.../version-2.4.1}/reference-pulsar-admin.md | 526 +------
.../schema-evolution-compatibility.md | 784 ++++++++++
.../version-2.4.1/schema-get-started.md | 91 ++
.../versioned_docs/version-2.4.1/schema-manage.md | 786 ++++++++++
.../version-2.4.1/schema-understand.md | 576 ++++++++
.../version-2.4.1/security-athenz.md | 89 ++
.../version-2.4.1/security-authorization.md | 100 ++
.../version-2.4.1/security-kerberos.md | 391 +++++
.../version-2.4.1/security-overview.md | 38 +
.../version-2.4.1/security-tls-authentication.md | 147 ++
.../version-2.4.1/security-tls-transport.md | 230 +++
.../version-2.4.1/security-token-client.md | 123 ++
.../versioned_sidebars/version-2.4.0-sidebars.json | 129 ++
.../versioned_sidebars/version-2.4.1-sidebars.json | 154 ++
site2/website/versions.json | 2 +
tests/bc_2_0_0/pom.xml | 2 +-
tests/bc_2_0_1/pom.xml | 2 +-
tests/docker-images/java-test-functions/pom.xml | 2 +-
tests/docker-images/latest-version-image/pom.xml | 2 +-
tests/docker-images/pom.xml | 2 +-
tests/integration/pom.xml | 2 +-
.../tests/integration/cli/AdminMultiHostTest.java | 1 -
.../pulsar/tests/integration/cli/CLITest.java | 23 +
.../{functions => cli}/FunctionsCLITest.java | 12 +-
.../integration/functions/PulsarFunctionsTest.java | 289 +++-
.../functions/PulsarFunctionsTestBase.java | 5 +
.../pulsar/tests/integration/proxy/TestProxy.java | 24 +
.../tests/integration/suites/PulsarTestSuite.java | 24 +-
.../integration/src/test/resources/pulsar-cli.xml | 1 +
.../{pulsar-cli.xml => pulsar-function-state.xml} | 9 +-
tests/integration/src/test/resources/pulsar.xml | 1 +
tests/pom.xml | 2 +-
tests/pulsar-kafka-compat-client-test/pom.xml | 2 +-
tests/pulsar-spark-test/pom.xml | 2 +-
.../spark/SparkStreamingPulsarReceiverTest.java | 50 +-
tests/pulsar-storm-test/pom.xml | 2 +-
.../org/apache/pulsar/storm/PulsarSpoutTest.java | 101 +-
tiered-storage/file-system/pom.xml | 112 ++
.../impl/FileSystemManagedLedgerOffloaderTest.java | 159 +++
tiered-storage/jcloud/pom.xml | 2 +-
.../impl/BlobStoreManagedLedgerOffloader.java | 4 +-
tiered-storage/pom.xml | 2 +-
737 files changed, 29377 insertions(+), 5000 deletions(-)
create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/BookKeeperClientFactoryImplTest.java
create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/BookiesApiTest.java
create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/impl/LoadManagerSharedTest.java
create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceAutoTopicCreationTest.java
create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/service/persistent/MessageDuplicationTest.java
create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/client/api/ClientDeduplicationFailureTest.java
copy pulsar-common/src/main/java/org/apache/pulsar/common/protocol/schema/DeleteSchemaResponse.java => pulsar-client-api/src/main/java/org/apache/pulsar/common/schema/SchemaInfoWithVersion.java (73%)
copy pulsar-common/src/main/java/org/apache/pulsar/policies/data/loadbalancer/ServiceLookupData.java => pulsar-client-cpp/lib/TimeUtils.cc (71%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/partition/PartitionedTopicMetadata.java => pulsar-client-cpp/lib/TimeUtils.h (69%)
copy pulsar-client-go/{pulsar/testhelps.go => examples/sendWithMsgID/producer.go} (60%)
create mode 100644 pulsar-client/src/main/java/org/apache/pulsar/client/impl/MultiMessageIdImpl.java
copy pulsar-common/src/main/java/org/apache/pulsar/{common/api/raw/RawMessageId.java => client/api/url/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => allocator/package-info.java} (89%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/api/raw/{RawMessageId.java => package-info.java} (89%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => compression/package-info.java} (88%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => conf/package-info.java} (90%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => functions/package-info.java} (89%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => io/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => lookup/data/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => naming/package-info.java} (89%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => nar/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => net/package-info.java} (90%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => partition/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => policies/data/package-info.java} (87%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => policies/impl/package-info.java} (90%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => policies/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => protocol/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/protocol/schema/{DeleteSchemaResponse.java => GetAllVersionsSchemaResponse.java} (86%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/protocol/schema/{DeleteSchemaResponse.java => IsCompatibilityResponse.java} (84%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/protocol/schema/{DeleteSchemaResponse.java => LongSchemaVersionResponse.java} (89%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => protocol/schema/package-info.java} (89%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => sasl/package-info.java} (88%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => stats/package-info.java} (88%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => util/collections/package-info.java} (86%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => util/netty/package-info.java} (85%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => util/package-info.java} (91%)
copy pulsar-common/src/main/java/org/apache/pulsar/common/{api/raw/RawMessageId.java => util/protobuf/package-info.java} (87%)
copy pulsar-common/src/main/java/org/apache/pulsar/{common/api/raw/RawMessageId.java => policies/data/loadbalancer/package-info.java} (87%)
create mode 100644 site2/docs/reference-metrics.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/adaptors-kafka.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/admin-api-overview.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/administration-zk-bk.md
copy site2/{docs => website/versioned_docs/version-2.4.0}/client-libraries-go.md (99%)
create mode 100644 site2/website/versioned_docs/version-2.4.0/client-libraries-java.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/concepts-messaging.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/concepts-tiered-storage.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/cookbooks-compaction.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/cookbooks-tiered-storage.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/deploy-bare-metal.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/developing-binary-protocol.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/developing-cpp.md
copy site2/{docs => website/versioned_docs/version-2.4.0}/functions-api.md (99%)
create mode 100644 site2/website/versioned_docs/version-2.4.0/functions-debugging.md
copy site2/{docs => website/versioned_docs/version-2.4.0}/functions-overview.md (99%)
create mode 100644 site2/website/versioned_docs/version-2.4.0/functions-quickstart.md
copy site2/{docs => website/versioned_docs/version-2.4.0}/functions-runtime.md (90%)
create mode 100644 site2/website/versioned_docs/version-2.4.0/functions-worker.md
copy site2/{docs => website/versioned_docs/version-2.4.0}/getting-started-standalone.md (99%)
create mode 100644 site2/website/versioned_docs/version-2.4.0/io-cdc-canal.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/io-cdc-debezium.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/io-managing.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/io-quickstart.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/reference-cli-tools.md
copy site2/{docs => website/versioned_docs/version-2.4.0}/reference-configuration.md (99%)
copy site2/{docs => website/versioned_docs/version-2.4.0}/reference-pulsar-admin.md (99%)
create mode 100644 site2/website/versioned_docs/version-2.4.0/security-kerberos.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/security-tls-authentication.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/security-token-admin.md
create mode 100644 site2/website/versioned_docs/version-2.4.0/sql-getting-started.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/admin-api-non-persistent-topics.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/admin-api-overview.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/admin-api-partitioned-topics.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/administration-dashboard.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/administration-upgrade.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/client-libraries-java.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/concepts-messaging.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/concepts-tiered-storage.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/cookbooks-compaction.md
copy site2/{docs => website/versioned_docs/version-2.4.1}/functions-api.md (95%)
create mode 100644 site2/website/versioned_docs/version-2.4.1/functions-cli.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/functions-debug.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/functions-deploy.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/functions-develop.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/functions-overview-new.md
copy site2/{docs => website/versioned_docs/version-2.4.1}/functions-overview.md (99%)
copy site2/{docs => website/versioned_docs/version-2.4.1}/functions-runtime.md (86%)
copy site2/{docs => website/versioned_docs/version-2.4.1}/getting-started-standalone.md (95%)
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-cdc-canal.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-cdc-debezium.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-connectors.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-debug.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-develop.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-jdbc.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-netty.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/io-use.md
copy site2/{docs => website/versioned_docs/version-2.4.1}/reference-configuration.md (99%)
create mode 100644 site2/website/versioned_docs/version-2.4.1/reference-connector-admin.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/reference-metrics.md
copy site2/{docs => website/versioned_docs/version-2.4.1}/reference-pulsar-admin.md (74%)
create mode 100644 site2/website/versioned_docs/version-2.4.1/schema-evolution-compatibility.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/schema-get-started.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/schema-manage.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/schema-understand.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/security-athenz.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/security-authorization.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/security-kerberos.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/security-overview.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/security-tls-authentication.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/security-tls-transport.md
create mode 100644 site2/website/versioned_docs/version-2.4.1/security-token-client.md
create mode 100644 site2/website/versioned_sidebars/version-2.4.0-sidebars.json
create mode 100644 site2/website/versioned_sidebars/version-2.4.1-sidebars.json
rename tests/integration/src/test/java/org/apache/pulsar/tests/integration/{functions => cli}/FunctionsCLITest.java (89%)
copy tests/integration/src/test/resources/{pulsar-cli.xml => pulsar-function-state.xml} (66%)
create mode 100644 tiered-storage/file-system/pom.xml
create mode 100644 tiered-storage/file-system/src/test/java/org/apache/bookkeeper/mledger/offload/filesystem/impl/FileSystemManagedLedgerOffloaderTest.java