You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by ur...@apache.org on 2022/09/02 12:01:42 UTC
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#386cd4d)
This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new e92f22c6d80 Docs sync done from apache/pulsar(#386cd4d)
e92f22c6d80 is described below
commit e92f22c6d80aad5f6b53688e24163dffe7ddd8a5
Author: Pulsar Site Updater <de...@pulsar.apache.org>
AuthorDate: Fri Sep 2 12:01:37 2022 +0000
Docs sync done from apache/pulsar(#386cd4d)
---
.../blog/2019-12-04-Apache-Pulsar-2-4-2.md | 2 +-
.../blog/2020-06-18-Apache-Pulsar-2-6-0.md | 2 +-
.../blog/2021-06-12-Apache-Pulsar-2-8-0.md | 16 +-
site2/website-next/docs/about.md | 4 +-
site2/website-next/docs/adaptors-kafka.md | 22 +-
site2/website-next/docs/adaptors-spark.md | 10 +-
site2/website-next/docs/adaptors-storm.md | 6 -
site2/website-next/docs/admin-api-brokers.md | 43 +-
site2/website-next/docs/admin-api-clusters.md | 54 +-
site2/website-next/docs/admin-api-functions.md | 230 ++----
site2/website-next/docs/admin-api-namespaces.md | 315 +++-----
site2/website-next/docs/admin-api-overview.md | 10 +-
site2/website-next/docs/admin-api-packages.md | 82 +-
site2/website-next/docs/admin-api-permissions.md | 59 +-
site2/website-next/docs/admin-api-tenants.md | 44 +-
site2/website-next/docs/admin-api-topics.md | 458 ++++-------
site2/website-next/docs/admin-api-transactions.md | 112 +--
.../website-next/docs/administration-dashboard.md | 42 +-
site2/website-next/docs/administration-geo.md | 105 +--
.../docs/administration-isolation-bookie.md | 15 +-
.../docs/administration-isolation-broker.md | 4 +-
.../website-next/docs/administration-isolation.md | 8 +-
.../docs/administration-load-balance.md | 37 +-
.../docs/administration-metadata-store.md | 20 +-
site2/website-next/docs/administration-proxy.md | 20 +-
.../docs/administration-pulsar-manager.md | 56 +-
.../docs/administration-pulsar-shell.md | 11 +-
site2/website-next/docs/administration-upgrade.md | 13 +-
site2/website-next/docs/administration-zk-bk.md | 101 +--
site2/website-next/docs/client-libraries-cpp.md | 200 ++---
site2/website-next/docs/client-libraries-dotnet.md | 66 +-
site2/website-next/docs/client-libraries-go.md | 86 +--
site2/website-next/docs/client-libraries-java.md | 300 +++-----
site2/website-next/docs/client-libraries-node.md | 70 +-
site2/website-next/docs/client-libraries-python.md | 31 +-
site2/website-next/docs/client-libraries-rest.md | 8 -
.../docs/client-libraries-websocket.md | 91 +--
site2/website-next/docs/client-libraries.md | 4 +-
.../docs/concepts-architecture-overview.md | 30 +-
site2/website-next/docs/concepts-clients.md | 24 +-
site2/website-next/docs/concepts-messaging.md | 140 +---
site2/website-next/docs/concepts-multi-tenancy.md | 4 +-
.../docs/concepts-multiple-advertised-listeners.md | 6 +-
site2/website-next/docs/concepts-overview.md | 2 +-
.../docs/concepts-proxy-sni-routing.md | 29 +-
site2/website-next/docs/concepts-replication.md | 18 +-
site2/website-next/docs/concepts-tiered-storage.md | 4 +-
.../website-next/docs/concepts-topic-compaction.md | 2 +-
site2/website-next/docs/concepts-transactions.md | 6 +-
site2/website-next/docs/cookbooks-compaction.md | 42 +-
site2/website-next/docs/cookbooks-deduplication.md | 28 +-
site2/website-next/docs/cookbooks-encryption.md | 29 +-
site2/website-next/docs/cookbooks-message-queue.md | 10 +-
.../website-next/docs/cookbooks-non-persistent.md | 24 +-
.../docs/cookbooks-retention-expiry.md | 120 +--
.../website-next/docs/cookbooks-tiered-storage.md | 81 +-
site2/website-next/docs/deploy-aws.md | 106 +--
.../docs/deploy-bare-metal-multi-cluster.md | 130 +---
site2/website-next/docs/deploy-bare-metal.md | 177 ++---
site2/website-next/docs/deploy-dcos.md | 32 +-
site2/website-next/docs/deploy-docker.md | 15 +-
site2/website-next/docs/deploy-monitoring.md | 38 +-
site2/website-next/docs/develop-load-manager.md | 47 +-
site2/website-next/docs/develop-plugin.md | 18 +-
site2/website-next/docs/develop-schema.md | 8 +-
site2/website-next/docs/develop-tools.md | 23 +-
.../docs/developing-binary-protocol.md | 34 +-
site2/website-next/docs/functions-concepts.md | 12 +-
site2/website-next/docs/functions-debug-cli.md | 25 -
.../website-next/docs/functions-debug-localrun.md | 8 +-
.../website-next/docs/functions-debug-log-topic.md | 4 -
.../website-next/docs/functions-debug-unit-test.md | 8 -
.../docs/functions-deploy-arguments.md | 2 -
.../docs/functions-deploy-cluster-encryption.md | 2 -
.../docs/functions-deploy-cluster-package.md | 4 +-
.../docs/functions-deploy-cluster-parallelism.md | 8 +-
.../docs/functions-deploy-cluster-resource.md | 2 -
.../website-next/docs/functions-deploy-cluster.md | 6 +-
.../website-next/docs/functions-deploy-localrun.md | 2 -
.../website-next/docs/functions-deploy-trigger.md | 10 -
.../docs/functions-develop-admin-api.md | 4 -
site2/website-next/docs/functions-develop-api.md | 20 +-
site2/website-next/docs/functions-develop-log.md | 21 +-
.../website-next/docs/functions-develop-metrics.md | 6 -
.../docs/functions-develop-security.md | 8 +-
site2/website-next/docs/functions-develop-serde.md | 18 +-
site2/website-next/docs/functions-develop-state.md | 38 +-
.../docs/functions-develop-tutorial.md | 13 +-
.../docs/functions-develop-user-defined-configs.md | 16 -
site2/website-next/docs/functions-overview.md | 2 +-
site2/website-next/docs/functions-package-go.md | 8 -
site2/website-next/docs/functions-package-java.md | 8 -
.../website-next/docs/functions-package-python.md | 36 +-
site2/website-next/docs/functions-quickstart.md | 94 ---
.../docs/functions-runtime-java-options.md | 4 +-
.../docs/functions-runtime-kubernetes.md | 9 +-
.../website-next/docs/functions-runtime-process.md | 2 -
.../website-next/docs/functions-runtime-thread.md | 4 -
site2/website-next/docs/functions-runtime.md | 2 +-
site2/website-next/docs/functions-worker-corun.md | 8 +-
.../docs/functions-worker-for-geo-replication.md | 4 -
.../docs/functions-worker-run-separately.md | 26 +-
.../website-next/docs/functions-worker-stateful.md | 10 +-
.../docs/functions-worker-troubleshooting.md | 4 -
site2/website-next/docs/getting-started-docker.md | 28 +-
site2/website-next/docs/getting-started-helm.md | 72 +-
site2/website-next/docs/getting-started-pulsar.md | 8 +-
.../docs/getting-started-standalone.md | 74 +-
site2/website-next/docs/helm-deploy.md | 66 +-
site2/website-next/docs/helm-overview.md | 8 +-
site2/website-next/docs/helm-prepare.md | 10 +-
site2/website-next/docs/helm-tools.md | 4 +-
site2/website-next/docs/helm-upgrade.md | 4 -
site2/website-next/docs/io-canal-source.md | 74 +-
site2/website-next/docs/io-cassandra-sink.md | 4 -
site2/website-next/docs/io-cdc-debezium.md | 143 +---
site2/website-next/docs/io-cli.md | 135 +---
site2/website-next/docs/io-debezium-source.md | 180 ++---
site2/website-next/docs/io-debug.md | 40 +-
site2/website-next/docs/io-develop.md | 56 +-
site2/website-next/docs/io-dynamodb-source.md | 6 +-
site2/website-next/docs/io-elasticsearch-sink.md | 44 +-
site2/website-next/docs/io-file-source.md | 42 +-
site2/website-next/docs/io-flume-sink.md | 4 -
site2/website-next/docs/io-flume-source.md | 4 -
site2/website-next/docs/io-hbase-sink.md | 7 +-
site2/website-next/docs/io-hdfs2-sink.md | 7 +-
site2/website-next/docs/io-hdfs3-sink.md | 7 +-
site2/website-next/docs/io-influxdb-sink.md | 11 +-
site2/website-next/docs/io-jdbc-sink.md | 33 +-
site2/website-next/docs/io-kafka-sink.md | 7 +-
site2/website-next/docs/io-kafka-source.md | 55 +-
site2/website-next/docs/io-kinesis-sink.md | 8 +-
site2/website-next/docs/io-kinesis-source.md | 11 +-
site2/website-next/docs/io-mongo-sink.md | 7 +-
site2/website-next/docs/io-netty-source.md | 86 +--
site2/website-next/docs/io-overview.md | 40 +-
site2/website-next/docs/io-quickstart.md | 167 +----
site2/website-next/docs/io-rabbitmq-sink.md | 7 +-
site2/website-next/docs/io-rabbitmq-source.md | 7 +-
site2/website-next/docs/io-redis-sink.md | 39 +-
site2/website-next/docs/io-solr-sink.md | 7 +-
site2/website-next/docs/io-use.md | 232 ++----
site2/website-next/docs/kubernetes-helm.md | 72 +-
site2/website-next/docs/performance-pulsar-perf.md | 64 +-
site2/website-next/docs/reference-cli-tools.md | 235 ++----
.../docs/reference-configuration-bookkeeper.md | 2 +-
.../docs/reference-configuration-zookeeper.md | 2 +-
site2/website-next/docs/reference-metrics.md | 120 +--
site2/website-next/docs/reference-pulsar-admin.md | 834 ++++++---------------
site2/website-next/docs/reference-terminology.md | 12 +-
.../docs/schema-evolution-compatibility.md | 10 +-
site2/website-next/docs/schema-get-started.md | 14 +-
site2/website-next/docs/schema-manage.md | 90 +--
site2/website-next/docs/schema-understand.md | 42 +-
site2/website-next/docs/security-athenz.md | 4 -
site2/website-next/docs/security-authorization.md | 41 +-
site2/website-next/docs/security-basic-auth.md | 18 +-
site2/website-next/docs/security-bouncy-castle.md | 40 +-
site2/website-next/docs/security-encryption.md | 34 +-
site2/website-next/docs/security-extending.md | 10 +-
site2/website-next/docs/security-jwt.md | 66 +-
site2/website-next/docs/security-kerberos.md | 152 ++--
site2/website-next/docs/security-oauth2.md | 26 +-
site2/website-next/docs/security-overview.md | 6 +-
.../docs/security-policy-and-supported-versions.md | 15 +-
.../docs/security-tls-authentication.md | 64 +-
site2/website-next/docs/security-tls-keystore.md | 89 +--
site2/website-next/docs/security-tls-transport.md | 54 +-
site2/website-next/docs/security-token-admin.md | 68 +-
.../docs/sql-deployment-configurations.md | 50 +-
site2/website-next/docs/sql-getting-started.md | 24 +-
site2/website-next/docs/sql-overview.md | 6 +-
site2/website-next/docs/sql-rest-api.md | 6 +-
site2/website-next/docs/standalone-docker.md | 33 +-
site2/website-next/docs/standalone.md | 76 +-
site2/website-next/docs/tiered-storage-aliyun.md | 36 +-
site2/website-next/docs/tiered-storage-aws.md | 44 +-
site2/website-next/docs/tiered-storage-azure.md | 32 +-
.../website-next/docs/tiered-storage-filesystem.md | 146 +---
site2/website-next/docs/tiered-storage-gcs.md | 49 +-
site2/website-next/docs/tiered-storage-overview.md | 12 +-
site2/website-next/docs/tiered-storage-s3.md | 32 +-
site2/website-next/docs/transaction-api.md | 60 +-
site2/website-next/docs/txn-how.md | 10 +-
site2/website-next/docs/txn-use.md | 36 +-
site2/website-next/docs/txn-what.md | 6 +-
site2/website-next/docs/txn-why.md | 4 +-
.../website-next/docs/window-functions-context.md | 59 +-
.../administration-proxy.md | 2 +-
.../reference-configuration.md | 12 +-
.../reference-pulsar-admin.md | 12 +-
.../administration-proxy.md | 2 +-
.../client-libraries-go.md | 2 +-
.../client-libraries-java.md | 12 +-
.../version-2.1.1-incubating/concepts-messaging.md | 2 +-
.../reference-configuration.md | 12 +-
.../reference-pulsar-admin.md | 12 +-
.../security-encryption.md | 2 +-
.../client-libraries-go.md | 2 +-
.../client-libraries-java.md | 10 +-
.../concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 10 +-
.../security-encryption.md | 2 +-
.../security-policy-and-supported-versions.md | 2 +-
.../sql-deployment-configurations.md | 2 +-
.../version-2.10.0-deprecated/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../client-libraries-java.md | 10 +-
.../concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 10 +-
.../security-encryption.md | 2 +-
.../security-policy-and-supported-versions.md | 2 +-
.../sql-deployment-configurations.md | 2 +-
.../version-2.10.1-deprecated/txn-use.md | 2 +-
.../version-2.10.x/admin-api-clusters.md | 2 +-
.../version-2.10.x/admin-api-namespaces.md | 18 +-
.../version-2.10.x/admin-api-packages.md | 2 +-
.../version-2.10.x/administration-zk-bk.md | 2 +-
.../version-2.10.x/client-libraries-cpp.md | 26 +-
.../version-2.10.x/client-libraries-dotnet.md | 2 +-
.../version-2.10.x/client-libraries-go.md | 2 +-
.../version-2.10.x/client-libraries-java.md | 20 +-
.../version-2.10.x/client-libraries-node.md | 14 +-
.../version-2.10.x/client-libraries-websocket.md | 6 +-
.../concepts-architecture-overview.md | 14 +-
.../version-2.10.x/concepts-clients.md | 8 +-
.../version-2.10.x/concepts-messaging.md | 8 +-
.../version-2.10.x/concepts-multi-tenancy.md | 2 +-
.../version-2.10.x/concepts-proxy-sni-routing.md | 8 +-
.../versioned_docs/version-2.10.x/deploy-aws.md | 4 +-
.../deploy-bare-metal-multi-cluster.md | 16 +-
.../versioned_docs/version-2.10.x/deploy-dcos.md | 2 +-
.../version-2.10.x/getting-started-docker.md | 28 +-
.../version-2.10.x/getting-started-helm.md | 84 +--
.../version-2.10.x/getting-started-standalone.md | 83 +-
.../version-2.10.x/reference-configuration.md | 10 +-
.../version-2.10.x/reference-metrics.md | 18 +-
.../version-2.10.x/security-encryption.md | 2 +-
.../security-policy-and-supported-versions.md | 2 +-
.../sql-deployment-configurations.md | 2 +-
.../versioned_docs/version-2.10.x/txn-use.md | 2 +-
.../version-2.2.0/client-libraries-java.md | 12 +-
.../version-2.2.0/concepts-messaging.md | 2 +-
.../version-2.2.0/reference-configuration.md | 12 +-
.../version-2.2.0/reference-pulsar-admin.md | 18 +-
.../version-2.2.0/security-encryption.md | 2 +-
.../version-2.2.1/concepts-messaging.md | 2 +-
.../version-2.2.1/reference-configuration.md | 12 +-
.../version-2.2.1/reference-pulsar-admin.md | 18 +-
.../version-2.2.1/security-encryption.md | 2 +-
.../version-2.2.1/sql-deployment-configurations.md | 2 +-
.../version-2.3.0/administration-proxy.md | 2 +-
.../version-2.3.0/client-libraries-java.md | 12 +-
.../version-2.3.0/reference-configuration.md | 12 +-
.../version-2.3.0/reference-pulsar-admin.md | 18 +-
.../version-2.3.0/security-encryption.md | 2 +-
.../version-2.3.0/sql-deployment-configurations.md | 2 +-
.../version-2.3.1/reference-configuration.md | 12 +-
.../version-2.3.1/reference-pulsar-admin.md | 18 +-
.../version-2.3.1/security-encryption.md | 2 +-
.../version-2.3.1/sql-deployment-configurations.md | 2 +-
.../version-2.3.2/client-libraries-go.md | 2 +-
.../version-2.3.2/client-libraries-java.md | 12 +-
.../version-2.3.2/reference-configuration.md | 12 +-
.../version-2.3.2/security-encryption.md | 2 +-
.../version-2.3.2/sql-deployment-configurations.md | 2 +-
.../version-2.4.0/reference-configuration.md | 12 +-
.../version-2.4.0/security-encryption.md | 2 +-
.../version-2.4.0/sql-deployment-configurations.md | 2 +-
.../version-2.4.1/client-libraries-go.md | 2 +-
.../version-2.4.1/reference-configuration.md | 12 +-
.../version-2.4.1/schema-understand.md | 10 +-
.../version-2.4.1/security-encryption.md | 2 +-
.../version-2.4.1/sql-deployment-configurations.md | 2 +-
.../version-2.4.2/client-libraries-go.md | 2 +-
.../version-2.4.2/reference-configuration.md | 12 +-
.../version-2.4.2/schema-understand.md | 10 +-
.../version-2.4.2/security-encryption.md | 2 +-
.../version-2.4.2/sql-deployment-configurations.md | 2 +-
.../version-2.5.0/administration-proxy.md | 2 +-
.../version-2.5.0/reference-configuration.md | 12 +-
.../version-2.5.1/client-libraries-go.md | 2 +-
.../version-2.5.1/reference-configuration.md | 12 +-
.../version-2.5.1/security-encryption.md | 2 +-
.../version-2.5.1/sql-deployment-configurations.md | 2 +-
.../version-2.5.2/client-libraries-go.md | 2 +-
.../version-2.5.2/reference-configuration.md | 12 +-
.../version-2.5.2/security-encryption.md | 2 +-
.../version-2.5.2/sql-deployment-configurations.md | 2 +-
.../version-2.6.0/administration-proxy.md | 2 +-
.../version-2.6.0/client-libraries-go.md | 2 +-
.../version-2.6.0/reference-configuration.md | 12 +-
.../version-2.6.1/administration-proxy.md | 2 +-
.../version-2.6.1/client-libraries-go.md | 2 +-
.../version-2.6.1/concepts-proxy-sni-routing.md | 4 +-
.../version-2.6.1/reference-configuration.md | 12 +-
.../version-2.6.2/administration-proxy.md | 2 +-
.../version-2.6.2/client-libraries-go.md | 2 +-
.../version-2.6.2/concepts-proxy-sni-routing.md | 4 +-
.../version-2.6.2/reference-configuration.md | 12 +-
.../version-2.6.3/administration-proxy.md | 2 +-
.../version-2.6.3/client-libraries-go.md | 2 +-
.../version-2.6.3/concepts-proxy-sni-routing.md | 4 +-
.../version-2.6.3/reference-configuration.md | 12 +-
.../version-2.6.4/administration-proxy.md | 2 +-
.../version-2.6.4/client-libraries-go.md | 2 +-
.../version-2.6.4/concepts-proxy-sni-routing.md | 4 +-
.../version-2.6.4/reference-configuration.md | 12 +-
.../version-2.7.0/client-libraries-go.md | 2 +-
.../version-2.7.0/concepts-proxy-sni-routing.md | 4 +-
.../version-2.7.0/reference-configuration.md | 12 +-
.../version-2.7.1/client-libraries-go.md | 2 +-
.../version-2.7.1/concepts-proxy-sni-routing.md | 4 +-
.../version-2.7.1/reference-configuration.md | 12 +-
.../version-2.7.2/client-libraries-go.md | 2 +-
.../version-2.7.2/concepts-proxy-sni-routing.md | 4 +-
.../version-2.7.2/reference-configuration.md | 12 +-
.../version-2.7.3/client-libraries-go.md | 2 +-
.../version-2.7.3/concepts-proxy-sni-routing.md | 4 +-
.../version-2.7.3/reference-configuration.md | 12 +-
.../version-2.7.4/client-libraries-go.md | 2 +-
.../version-2.7.4/concepts-proxy-sni-routing.md | 4 +-
.../version-2.7.4/reference-configuration.md | 12 +-
.../client-libraries-go.md | 2 +-
.../version-2.8.0-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 12 +-
.../version-2.8.0-deprecated/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../version-2.8.1-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 12 +-
.../version-2.8.1-deprecated/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../version-2.8.2-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 12 +-
.../security-encryption.md | 2 +-
.../version-2.8.2-deprecated/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../version-2.8.3-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 12 +-
.../security-encryption.md | 2 +-
.../version-2.8.3-deprecated/txn-use.md | 2 +-
.../version-2.8.x/admin-api-clusters.md | 2 +-
.../version-2.8.x/admin-api-namespaces.md | 22 +-
.../version-2.8.x/administration-zk-bk.md | 2 +-
.../version-2.8.x/client-libraries-cpp.md | 10 +-
.../version-2.8.x/client-libraries-dotnet.md | 2 +-
.../version-2.8.x/client-libraries-go.md | 2 +-
.../version-2.8.x/client-libraries-java.md | 10 +-
.../version-2.8.x/client-libraries-node.md | 14 +-
.../version-2.8.x/client-libraries-websocket.md | 6 +-
.../concepts-architecture-overview.md | 12 +-
.../version-2.8.x/concepts-clients.md | 8 +-
.../version-2.8.x/concepts-messaging.md | 2 +-
.../version-2.8.x/concepts-multi-tenancy.md | 2 +-
.../version-2.8.x/concepts-proxy-sni-routing.md | 8 +-
.../versioned_docs/version-2.8.x/deploy-aws.md | 4 +-
.../deploy-bare-metal-multi-cluster.md | 26 +-
.../versioned_docs/version-2.8.x/deploy-dcos.md | 6 +-
.../version-2.8.x/getting-started-docker.md | 28 +-
.../version-2.8.x/getting-started-helm.md | 71 +-
.../version-2.8.x/getting-started-standalone.md | 74 +-
.../version-2.8.x/reference-configuration.md | 12 +-
.../version-2.8.x/reference-metrics.md | 16 +-
.../versioned_docs/version-2.8.x/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../version-2.9.0-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 10 +-
.../version-2.9.0-deprecated/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../version-2.9.1-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 10 +-
.../version-2.9.1-deprecated/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../version-2.9.2-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 10 +-
.../version-2.9.2-deprecated/txn-use.md | 2 +-
.../client-libraries-go.md | 2 +-
.../version-2.9.3-deprecated/concepts-messaging.md | 2 +-
.../concepts-proxy-sni-routing.md | 4 +-
.../reference-configuration.md | 10 +-
.../version-2.9.3-deprecated/txn-use.md | 2 +-
.../version-2.9.x/admin-api-clusters.md | 2 +-
.../version-2.9.x/admin-api-namespaces.md | 20 +-
.../version-2.9.x/administration-zk-bk.md | 2 +-
.../version-2.9.x/client-libraries-cpp.md | 20 +-
.../version-2.9.x/client-libraries-dotnet.md | 2 +-
.../version-2.9.x/client-libraries-go.md | 2 +-
.../version-2.9.x/client-libraries-java.md | 10 +-
.../version-2.9.x/client-libraries-node.md | 14 +-
.../version-2.9.x/client-libraries-websocket.md | 8 +-
.../concepts-architecture-overview.md | 10 +-
.../version-2.9.x/concepts-clients.md | 8 +-
.../version-2.9.x/concepts-messaging.md | 2 +-
.../version-2.9.x/concepts-multi-tenancy.md | 2 +-
.../version-2.9.x/concepts-proxy-sni-routing.md | 8 +-
.../versioned_docs/version-2.9.x/deploy-aws.md | 4 +-
.../deploy-bare-metal-multi-cluster.md | 18 +-
.../versioned_docs/version-2.9.x/deploy-dcos.md | 2 +-
.../version-2.9.x/getting-started-docker.md | 29 +-
.../version-2.9.x/getting-started-helm.md | 70 --
.../version-2.9.x/getting-started-standalone.md | 70 +-
.../version-2.9.x/reference-configuration.md | 10 +-
.../version-2.9.x/reference-metrics.md | 14 +-
.../versioned_docs/version-2.9.x/txn-use.md | 2 +-
414 files changed, 3103 insertions(+), 7861 deletions(-)
diff --git a/site2/website-next/blog/2019-12-04-Apache-Pulsar-2-4-2.md b/site2/website-next/blog/2019-12-04-Apache-Pulsar-2-4-2.md
index 297f864f0ce..c0ae39855cc 100644
--- a/site2/website-next/blog/2019-12-04-Apache-Pulsar-2-4-2.md
+++ b/site2/website-next/blog/2019-12-04-Apache-Pulsar-2-4-2.md
@@ -26,7 +26,7 @@ In Pulsar 2.4.2, we can start Broker with Functions worker when broker client is
In Pulsar Functions, BookKeeper is supported to store the state of Functions. When users attempt to fetch a key that does not exist from function state, an NPE(NullPointerException) error occurs. In Pulsar 2.4.2, we add error code and error message for the case when a key does not exist.
## Deduplication
-Deduplication removes messages based on the the largest sequence ID that pre-persisted. If an error is persisted to BookKeeper, a retry attempt is “deduplicated” with no message ever getting persisted. In version 2.4.2, we fix the issue from the following two aspects:
+Deduplication removes messages based on the the largest sequence ID that pre-persisted. If an error is persisted to BookKeeper, a retry attempt is "deduplicated" with no message ever getting persisted. In version 2.4.2, we fix the issue from the following two aspects:
- Double check the pending messages and return error to the producer when the duplication status is uncertain. For example, when a message is still pending.
- Sync back the lastPushed map with the lastStored map after failures.
diff --git a/site2/website-next/blog/2020-06-18-Apache-Pulsar-2-6-0.md b/site2/website-next/blog/2020-06-18-Apache-Pulsar-2-6-0.md
index 0eaa97bc36a..9bd36133fd0 100644
--- a/site2/website-next/blog/2020-06-18-Apache-Pulsar-2-6-0.md
+++ b/site2/website-next/blog/2020-06-18-Apache-Pulsar-2-6-0.md
@@ -257,7 +257,7 @@ For more information about implementation details, see [PR-6077](https://github.
### Add a flag to skip broker shutdown on transient OOM
-A high dispatch rate on one of the topics may cause a broker to go OOM temporarily. It is a transient error and the broker can recover within a few seconds as soon as some memory gets released. However, in 2.4 release ([#4196](https://github.com/apache/pulsar/pull/4196)), the “restarted broker on OOM” feature can cause huge instability in a cluster, where a topic moves from one broker to another and restarts multiple brokers and disrupts other topics as well. So this PR provides a dynami [...]
+A high dispatch rate on one of the topics may cause a broker to go OOM temporarily. It is a transient error and the broker can recover within a few seconds as soon as some memory gets released. However, in 2.4 release ([#4196](https://github.com/apache/pulsar/pull/4196)), the "restarted broker on OOM" feature can cause huge instability in a cluster, where a topic moves from one broker to another and restarts multiple brokers and disrupts other topics as well. So this PR provides a dynami [...]
For more information about implementation details, see [PR-6634](https://github.com/apache/pulsar/pull/6634).
diff --git a/site2/website-next/blog/2021-06-12-Apache-Pulsar-2-8-0.md b/site2/website-next/blog/2021-06-12-Apache-Pulsar-2-8-0.md
index d010aeee40e..a0620d59048 100644
--- a/site2/website-next/blog/2021-06-12-Apache-Pulsar-2-8-0.md
+++ b/site2/website-next/blog/2021-06-12-Apache-Pulsar-2-8-0.md
@@ -23,29 +23,29 @@ The key features and updates in this release are:
## Exclusive Producer
-By default, the Pulsar producer API provides a “multi-writer” semantic to append messages to a topic. However, there are several use cases that require exclusive access for a single writer, such as ensuring a linear non-interleaved history of messages or providing a mechanism for leader election.
+By default, the Pulsar producer API provides a "multi-writer" semantic to append messages to a topic. However, there are several use cases that require exclusive access for a single writer, such as ensuring a linear non-interleaved history of messages or providing a mechanism for leader election.
-This new feature allows applications to require exclusive producer access in order to achieve a “single-writer” situation. It guarantees that there should be 1 single writer in any combination of errors. If the producer loses its exclusive access, no more messages from it can be published on the topic.
+This new feature allows applications to require exclusive producer access in order to achieve a "single-writer" situation. It guarantees that there should be 1 single writer in any combination of errors. If the producer loses its exclusive access, no more messages from it can be published on the topic.
-One use case for this feature is the metadata controller in Pulsar Functions. In order to write a single linear history of all the functions metadata updates, the metadata controller requires to elect one leader and that all the “decisions” made by this leader be written on the metadata topic. By leveraging the exclusive producer feature, Pulsar guarantees that the metadata topic contains different segments of updates, one per each successive leader, and there is no interleaving across d [...]
+One use case for this feature is the metadata controller in Pulsar Functions. In order to write a single linear history of all the functions metadata updates, the metadata controller requires to elect one leader and that all the "decisions" made by this leader be written on the metadata topic. By leveraging the exclusive producer feature, Pulsar guarantees that the metadata topic contains different segments of updates, one per each successive leader, and there is no interleaving across d [...]
## Package Management API
-Since its introduction in version 2.0, the Functions API has become hugely popular among Pulsar users. While it offers many benefits, there are a number of ways to improve the user experience. For example, today, if a function is deployed multiple times, the function package ends up being uploaded multiple times. Also, there is no version management in Pulsar for Functions and IO connectors. The newly introduced package management API provides an easier way to manage the packages for Fun [...]
+Since its introduction in version 2.0, the Functions API has become hugely popular among Pulsar users. While it offers many benefits, there are a number of ways to improve the user experience. For example, today, if a function is deployed multiple times, the function package ends up being uploaded multiple times. Also, there is no version management in Pulsar for Functions and IO connectors. The newly introduced package management API provides an easier way to manage the packages for Fun [...]
## Simplified Client Memory Limit Settings
Prior to 2.8, there are multiple settings in producers and consumers that allow controlling the sizes of the internal message queues. These settings ultimately control the amount of memory the Pulsar client uses. However, there are few issues with this approach that make it complicated to select an overall configuration that controls the total usage of memory.
-For example, the settings are based on the “number of messages”, so the expected message size must be adjusted per producer or consumer. If an application has a large (or unknown) number of producers or consumers, it’s very difficult to select an appropriate value for queue sizes. The same is true for topics that have many partitions.
+For example, the settings are based on the "number of messages", so the expected message size must be adjusted per producer or consumer. If an application has a large (or unknown) number of producers or consumers, it’s very difficult to select an appropriate value for queue sizes. The same is true for topics that have many partitions.
-In 2.8, we introduced a new API to set the memory limit. This single `memoryLimit` setting specifies a maximum amount of memory on a given Pulsar client. The producers and consumers compete for the memory assigned. It ensures the memory used by the Pulsar client will not go beyond the set limit. Read “[PIP-74: Pulsar client memory limits](https://github.com/apache/pulsar/wiki/PIP-74%3A-Pulsar-client-memory-limits)” for more details.
+In 2.8, we introduced a new API to set the memory limit. This single `memoryLimit` setting specifies a maximum amount of memory on a given Pulsar client. The producers and consumers compete for the memory assigned. It ensures the memory used by the Pulsar client will not go beyond the set limit. Read "[PIP-74: Pulsar client memory limits](https://github.com/apache/pulsar/wiki/PIP-74%3A-Pulsar-client-memory-limits)" for more details.
## Broker Entry Metadata
Pulsar messages define a very comprehensive set of metadata properties. However, to add a new property, the `MessageMetadata` definition in Pulsar protocol must change to inform both broker and client of the newly introduced property.
-But in certain cases, this metadata property might need to be added from the broker side, or need to be retrieved by the broker at a very low cost. To prevent deserializing these properties from the message metadata, we introduced “Broker Entry Metadata” in 2.8.0 to provide a lightweight approach to add additional metadata properties without serializing and deserializing the protobuf-encoded `MessageMetadata`.
+But in certain cases, this metadata property might need to be added from the broker side, or need to be retrieved by the broker at a very low cost. To prevent deserializing these properties from the message metadata, we introduced "Broker Entry Metadata" in 2.8.0 to provide a lightweight approach to add additional metadata properties without serializing and deserializing the protobuf-encoded `MessageMetadata`.
This feature unblocks a new set of capabilities for Pulsar. For example, we can leverage broker entry metadata to generate broker publish time for the messages appended to the Pulsar topic. The other example is to generate a monotonically increasing sequence-id for messages produced to a Pulsar topic. We use this feature in Kafka-on-Pulsar to implement Kafka offset.
@@ -117,7 +117,7 @@ The serverless function concepts were adopted into stream processing and then bu
## Step 6: Infinite storage for Pulsar via Tiered Storage
-As adoption of Apache Pulsar continued and the amount of data stored in Pulsar increased, users eventually hit a “retention cliff”, at which point it became significantly more expensive to store, manage, and retrieve data in Apache BookKeeper. To work around this, operators and application developers typically use an external store like AWS S3 as a sink for long-term storage. This means you lose most of the benefits of Pulsar’s immutable stream and ordering semantics, and instead end up [...]
+As adoption of Apache Pulsar continued and the amount of data stored in Pulsar increased, users eventually hit a "retention cliff", at which point it became significantly more expensive to store, manage, and retrieve data in Apache BookKeeper. To work around this, operators and application developers typically use an external store like AWS S3 as a sink for long-term storage. This means you lose most of the benefits of Pulsar’s immutable stream and ordering semantics, and instead end up [...]
The introduction of Tiered Storage allows Pulsar to offload the majority of the data to a remote cloud-native storage. This cheaper form of storage readily scales with the volume of data. More importantly, with the addition of Tiered Storage, Pulsar provides the batch storage capabilities needed to support batch processing when integrating with a unified batch and stream processor like Flink. The unified batch and stream processing capabilities integrated with Pulsar enable companies to [...]
diff --git a/site2/website-next/docs/about.md b/site2/website-next/docs/about.md
index 5d21cce7204..9faaf7ec0b7 100644
--- a/site2/website-next/docs/about.md
+++ b/site2/website-next/docs/about.md
@@ -19,7 +19,7 @@ This portal holds a variety of topics, tutorials, guides, and reference material
Select one of the content blocks below to begin your Pulsar journey. If you ...
* Are new to Pulsar, start with **About Pulsar** to learn about features and concepts.
* Want to jump to the quickstart, select **Get Started**.
-* Are an operator responsible for architecting and supporting Apache Pulsar. start with **Install, Deploy, Upgrade**.
+* Are an operator responsible for architecting and supporting Apache Pulsar, start with **Install, Deploy, Upgrade**.
* Are a developer who wants to master Apache Pulsar, select **Pulsar for Developers**.
* Want to try out Pulsar, select **How To** for access to the "hello world" tutorial.
* An experienced Pulsar coder looking for API, metrics, or configuration documentation, go to **Reference**.
@@ -56,7 +56,7 @@ You’ll notice an Edit button at the bottom and top of each page. Click it to o
## Join the Community!
***
-The Pulsar community on github is active, passionate, and knowledgeable. Join discussions, voice opinions, suggest features, and dive into the code itself. Find your Pulsar family here at [apache/pulsar](https://github.com/apache/pulsar).
+The Pulsar community on GitHub is active, passionate, and knowledgeable. Join discussions, voice opinions, suggest features, and dive into the code itself. Find your Pulsar family here at [apache/pulsar](https://github.com/apache/pulsar).
An equally passionate community can be found in the [Pulsar Slack channel](https://apache-pulsar.slack.com/). You’ll need an invitation to join, but many Github Pulsar community members are Slack members too. Join, hang out, learn, and make some new friends.
diff --git a/site2/website-next/docs/adaptors-kafka.md b/site2/website-next/docs/adaptors-kafka.md
index c51fe8b7ca8..49761504779 100644
--- a/site2/website-next/docs/adaptors-kafka.md
+++ b/site2/website-next/docs/adaptors-kafka.md
@@ -7,50 +7,44 @@ sidebar_label: "Kafka client wrapper"
Pulsar provides an easy option for applications that are currently written using the [Apache Kafka](http://kafka.apache.org) Java client API.
-## Using the Pulsar Kafka compatibility wrapper
+## Use the Pulsar Kafka compatibility wrapper
In an existing application, change the regular Kafka client dependency and replace it with the Pulsar Kafka wrapper. Remove the following dependency in `pom.xml`:
```xml
-
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.2.1</version>
</dependency>
-
```
Then include this dependency for the Pulsar Kafka wrapper:
```xml
-
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka</artifactId>
<version>@pulsar:version@</version>
</dependency>
-
```
With the new dependency, the existing code works without any changes. You need to adjust the configuration, and make sure it points the
producers and consumers to Pulsar service rather than Kafka, and uses a particular
Pulsar topic.
-## Using the Pulsar Kafka compatibility wrapper together with existing kafka client
+## Use the Pulsar Kafka compatibility wrapper together with existing Kafka client
-When migrating from Kafka to Pulsar, the application might use the original kafka client
-and the pulsar kafka wrapper together during migration. You should consider using the
-unshaded pulsar kafka client wrapper.
+When migrating from Kafka to Pulsar, the application might use the original Kafka client
+and the Pulsar Kafka wrapper together during migration. You should consider using the
+unshaded Pulsar Kafka client wrapper.
```xml
-
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-original</artifactId>
<version>@pulsar:version@</version>
</dependency>
-
```
When using this dependency, construct producers using `org.apache.kafka.clients.producer.PulsarKafkaProducer`
@@ -59,7 +53,6 @@ instead of `org.apache.kafka.clients.producer.KafkaProducer` and `org.apache.kaf
## Producer example
```java
-
// Topic needs to be a regular Pulsar topic
String topic = "persistent://public/default/my-topic";
@@ -78,13 +71,11 @@ for (int i = 0; i < 10; i++) {
}
producer.close();
-
```
## Consumer example
```java
-
String topic = "persistent://public/default/my-topic";
Properties props = new Properties();
@@ -107,7 +98,6 @@ while (true) {
// Commit last offset
consumer.commitSync();
}
-
```
## Complete Examples
@@ -116,7 +106,7 @@ You can find the complete producer and consumer examples [here](https://github.c
## Compatibility matrix
-Currently the Pulsar Kafka wrapper supports most of the operations offered by the Kafka API.
+Currently, the Pulsar Kafka wrapper supports most of the operations offered by the Kafka API.
### Producer
diff --git a/site2/website-next/docs/adaptors-spark.md b/site2/website-next/docs/adaptors-spark.md
index afa5a7ef827..d97d2349669 100644
--- a/site2/website-next/docs/adaptors-spark.md
+++ b/site2/website-next/docs/adaptors-spark.md
@@ -18,7 +18,6 @@ To use the receiver, include a dependency for the `pulsar-spark` library in your
If you're using Maven, add this to your `pom.xml`:
```xml
-
<!-- in your <properties> block -->
<pulsar.version>@pulsar:version@</pulsar.version>
@@ -28,7 +27,6 @@ If you're using Maven, add this to your `pom.xml`:
<artifactId>pulsar-spark</artifactId>
<version>${pulsar.version}</version>
</dependency>
-
```
#### Gradle
@@ -36,13 +34,11 @@ If you're using Maven, add this to your `pom.xml`:
If you're using Gradle, add this to your `build.gradle` file:
```groovy
-
def pulsarVersion = "@pulsar:version@"
dependencies {
compile group: 'org.apache.pulsar', name: 'pulsar-spark', version: pulsarVersion
}
-
```
### Usage
@@ -50,7 +46,6 @@ dependencies {
Pass an instance of `SparkStreamingPulsarReceiver` to the `receiverStream` method in `JavaStreamingContext`:
```java
-
String serviceUrl = "pulsar://localhost:6650/";
String topic = "persistent://public/default/test_src";
String subs = "test_sub";
@@ -72,19 +67,16 @@ Pass an instance of `SparkStreamingPulsarReceiver` to the `receiverStream` metho
new AuthenticationDisabled());
JavaReceiverInputDStream<byte[]> lineDStream = jsc.receiverStream(pulsarReceiver);
-
```
For a complete example, click [here](https://github.com/apache/pulsar-adapters/blob/master/examples/spark/src/main/java/org/apache/spark/streaming/receiver/example/SparkStreamingPulsarReceiverExample.java). In this example, the number of messages that contain the string "Pulsar" in received messages is counted.
-Note that if needed, other Pulsar authentication classes can be used. For example, in order to use a token during authentication the following parameters for the `SparkStreamingPulsarReceiver` constructor can be set:
+Note that if needed, other Pulsar authentication classes can be used. For example, to use a token during authentication the following parameters for the `SparkStreamingPulsarReceiver` constructor can be set:
```java
-
SparkStreamingPulsarReceiver pulsarReceiver = new SparkStreamingPulsarReceiver(
serviceUrl,
pulsarConf,
new AuthenticationToken("token:<secret-JWT-token>"));
-
```
diff --git a/site2/website-next/docs/adaptors-storm.md b/site2/website-next/docs/adaptors-storm.md
index 9df907680dd..7f05e79dde1 100644
--- a/site2/website-next/docs/adaptors-storm.md
+++ b/site2/website-next/docs/adaptors-storm.md
@@ -13,13 +13,11 @@ An application can inject data into a Storm topology via a generic Pulsar spout,
Include dependency for Pulsar Storm Adaptor:
```xml
-
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-storm</artifactId>
<version>${pulsar.version}</version>
</dependency>
-
```
## Pulsar Spout
@@ -29,7 +27,6 @@ The Pulsar Spout allows for the data published on a topic to be consumed by a St
The tuples that fail to be processed by the downstream bolts will be re-injected by the spout with an exponential backoff, within a configurable timeout (the default is 60 seconds) or a configurable number of retries, whichever comes first, after which it is acknowledged by the consumer. Here's an example construction of a spout:
```java
-
MessageToValuesMapper messageToValuesMapper = new MessageToValuesMapper() {
@Override
@@ -53,7 +50,6 @@ spoutConf.setMessageToValuesMapper(messageToValuesMapper);
// Create a Pulsar Spout
PulsarSpout spout = new PulsarSpout(spoutConf);
-
```
For a complete example, click [here](https://github.com/apache/pulsar-adapters/blob/master/pulsar-storm/src/test/java/org/apache/pulsar/storm/PulsarSpoutTest.java).
@@ -65,7 +61,6 @@ The Pulsar bolt allows data in a Storm topology to be published on a topic. It p
A partitioned topic can also be used to publish messages on different topics. In the implementation of the `TupleToMessageMapper`, a "key" will need to be provided in the message which will send the messages with the same key to the same topic. Here's an example bolt:
```java
-
TupleToMessageMapper tupleToMessageMapper = new TupleToMessageMapper() {
@Override
@@ -90,6 +85,5 @@ boltConf.setTupleToMessageMapper(tupleToMessageMapper);
// Create a Pulsar Bolt
PulsarBolt bolt = new PulsarBolt(boltConf);
-
```
diff --git a/site2/website-next/docs/admin-api-brokers.md b/site2/website-next/docs/admin-api-brokers.md
index 72206fe8dcf..9d20b06a5c5 100644
--- a/site2/website-next/docs/admin-api-brokers.md
+++ b/site2/website-next/docs/admin-api-brokers.md
@@ -35,7 +35,7 @@ Pulsar brokers consist of two components:
In addition to being configurable when you start them up, brokers can also be [dynamically configured](#dynamic-broker-configuration).
-> See the [Configuration](reference-configuration.md#broker) page for a full listing of broker-specific configuration parameters.
+For a full listing of broker-specific configuration parameters, see the [Configuration](reference-configuration.md#broker) page.
## Brokers resources
@@ -50,10 +50,8 @@ Fetch all available active brokers that are serving traffic with cluster name.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin brokers list use
+pulsar-admin brokers list use
broker1.use.org.com:8080
-
```
</TabItem>
@@ -65,9 +63,7 @@ broker1.use.org.com:8080
<TabItem value="Java">
```java
-
admin.brokers().getActiveBrokers(clusterName)
-
```
</TabItem>
@@ -86,10 +82,8 @@ Fetch the information of the leader broker, for example, the service url.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin brokers leader-broker
+pulsar-admin brokers leader-broker
BrokerInfo(serviceUrl=broker1.use.org.com:8080)
-
```
</TabItem>
@@ -101,9 +95,7 @@ BrokerInfo(serviceUrl=broker1.use.org.com:8080)
<TabItem value="Java">
```java
-
admin.brokers().getLeaderBroker()
-
```
For the detail of the code above, see [here](https://github.com/apache/pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/BrokersImpl.java#L80)
@@ -124,9 +116,8 @@ It finds all namespaces which are owned and served by a given broker.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin brokers namespaces use \
- --url broker1.use.org.com:8080
+pulsar-admin brokers namespaces use \
+--url broker1.use.org.com:8080
{
"my-property/use/my-ns/0x00000000_0xffffffff": {
@@ -135,7 +126,6 @@ $ pulsar-admin brokers namespaces use \
"is_active": true
}
}
-
```
</TabItem>
@@ -147,9 +137,7 @@ $ pulsar-admin brokers namespaces use \
<TabItem value="Java">
```java
-
admin.brokers().getOwnedNamespaces(cluster,brokerUrl);
-
```
</TabItem>
@@ -174,12 +162,10 @@ But since all broker configuration in Pulsar is stored in ZooKeeper, configurati
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-The [`update-dynamic-config`](/tools/pulsar-admin/) subcommand will update existing configuration. It takes two arguments: the name of the parameter and the new value using the `config` and `value` flag respectively. Here's an example for the [`brokerShutdownTimeoutMs`](reference-configuration.md#broker-brokerShutdownTimeoutMs) parameter:
+The [`update-dynamic-config`](/tools/pulsar-admin/) subcommand will update existing configuration. It takes two arguments: the name of the parameter and the new value using the `config` and `value` flag respectively. Here's an example of the [`brokerShutdownTimeoutMs`](reference-configuration.md#broker-brokerShutdownTimeoutMs) parameter:
```shell
-
-$ pulsar-admin brokers update-dynamic-config --config brokerShutdownTimeoutMs --value 100
-
+pulsar-admin brokers update-dynamic-config --config brokerShutdownTimeoutMs --value 100
```
</TabItem>
@@ -191,9 +177,7 @@ $ pulsar-admin brokers update-dynamic-config --config brokerShutdownTimeoutMs --
<TabItem value="Java">
```java
-
admin.brokers().updateDynamicConfiguration(configName, configValue);
-
```
</TabItem>
@@ -204,6 +188,7 @@ admin.brokers().updateDynamicConfiguration(configName, configValue);
### List updated values
Fetch a list of all potentially updatable configuration parameters.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -211,10 +196,8 @@ Fetch a list of all potentially updatable configuration parameters.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin brokers list-dynamic-config
+pulsar-admin brokers list-dynamic-config
brokerShutdownTimeoutMs
-
```
</TabItem>
@@ -226,9 +209,7 @@ brokerShutdownTimeoutMs
<TabItem value="Java">
```java
-
admin.brokers().getDynamicConfigurationNames();
-
```
</TabItem>
@@ -247,10 +228,8 @@ Fetch a list of all parameters that have been dynamically updated.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin brokers get-all-dynamic-config
+pulsar-admin brokers get-all-dynamic-config
brokerShutdownTimeoutMs:100
-
```
</TabItem>
@@ -262,9 +241,7 @@ brokerShutdownTimeoutMs:100
<TabItem value="Java">
```java
-
admin.brokers().getAllDynamicConfigurations();
-
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-clusters.md b/site2/website-next/docs/admin-api-clusters.md
index 41925c6ce57..97faab5f20c 100644
--- a/site2/website-next/docs/admin-api-clusters.md
+++ b/site2/website-next/docs/admin-api-clusters.md
@@ -37,7 +37,11 @@ Clusters can be managed via:
New clusters can be provisioned using the admin interface.
-> Please note that this operation requires superuser privileges.
+:::note
+
+This operation requires superuser privileges.
+
+:::
````mdx-code-block
<Tabs groupId="api-choice"
@@ -48,11 +52,9 @@ New clusters can be provisioned using the admin interface.
You can provision a new cluster using the [`create`](/tools/pulsar-admin/) subcommand. Here's an example:
```shell
-
-$ pulsar-admin clusters create cluster-1 \
- --url http://my-cluster.org.com:8080 \
- --broker-url pulsar://my-cluster.org.com:6650
-
+pulsar-admin clusters create cluster-1 \
+--url http://my-cluster.org.com:8080 \
+--broker-url pulsar://my-cluster.org.com:6650
```
</TabItem>
@@ -64,7 +66,6 @@ $ pulsar-admin clusters create cluster-1 \
<TabItem value="Java">
```java
-
ClusterData clusterData = new ClusterData(
serviceUrl,
serviceUrlTls,
@@ -72,7 +73,6 @@ ClusterData clusterData = new ClusterData(
brokerServiceUrlTls
);
admin.clusters().createCluster(clusterName, clusterData);
-
```
</TabItem>
@@ -82,7 +82,7 @@ admin.clusters().createCluster(clusterName, clusterData);
### Initialize cluster metadata
-When provision a new cluster, you need to initialize that cluster's [metadata](concepts-architecture-overview.md#metadata-store). When initializing cluster metadata, you need to specify all of the following:
+When provisioning a new cluster, you need to initialize that cluster's [metadata](concepts-architecture-overview.md#metadata-store). When initializing cluster metadata, you need to specify all of the following:
* The name of the cluster
* The local metadata store connection string for the cluster
@@ -102,7 +102,6 @@ You must initialize cluster metadata *before* starting up any [brokers](admin-ap
Here's an example cluster metadata initialization command:
```shell
-
bin/pulsar initialize-cluster-metadata \
--cluster us-west \
--metadata-store zk:zk1.us-west.example.com:2181,zk2.us-west.example.com:2181/my-chroot-path \
@@ -111,7 +110,6 @@ bin/pulsar initialize-cluster-metadata \
--web-service-url-tls https://pulsar.us-west.example.com:8443/ \
--broker-service-url pulsar://pulsar.us-west.example.com:6650/ \
--broker-service-url-tls pulsar+ssl://pulsar.us-west.example.com:6651/
-
```
You'll need to use `--*-tls` flags only if you're using [TLS authentication](security-tls-authentication.md) in your instance.
@@ -129,8 +127,7 @@ You can fetch the [configuration](reference-configuration.md) for an existing cl
Use the [`get`](/tools/pulsar-admin/) subcommand and specify the name of the cluster. Here's an example:
```shell
-
-$ pulsar-admin clusters get cluster-1
+pulsar-admin clusters get cluster-1
{
"serviceUrl": "http://my-cluster.org.com:8080/",
"serviceUrlTls": null,
@@ -138,7 +135,6 @@ $ pulsar-admin clusters get cluster-1
"brokerServiceUrlTls": null
"peerClusterNames": null
}
-
```
</TabItem>
@@ -150,9 +146,7 @@ $ pulsar-admin clusters get cluster-1
<TabItem value="Java">
```java
-
admin.clusters().getCluster(clusterName);
-
```
</TabItem>
@@ -173,11 +167,9 @@ You can update the configuration for an existing cluster at any time.
Use the [`update`](/tools/pulsar-admin/) subcommand and specify new configuration values using flags.
```shell
-
-$ pulsar-admin clusters update cluster-1 \
- --url http://my-cluster.org.com:4081 \
- --broker-url pulsar://my-cluster.org.com:3350
-
+pulsar-admin clusters update cluster-1 \
+--url http://my-cluster.org.com:4081 \
+--broker-url pulsar://my-cluster.org.com:3350
```
</TabItem>
@@ -189,7 +181,6 @@ $ pulsar-admin clusters update cluster-1 \
<TabItem value="Java">
```java
-
ClusterData clusterData = new ClusterData(
serviceUrl,
serviceUrlTls,
@@ -197,7 +188,6 @@ ClusterData clusterData = new ClusterData(
brokerServiceUrlTls
);
admin.clusters().updateCluster(clusterName, clusterData);
-
```
</TabItem>
@@ -218,9 +208,7 @@ Clusters can be deleted from a Pulsar [instance](reference-terminology.md#instan
Use the [`delete`](/tools/pulsar-admin/) subcommand and specify the name of the cluster.
```
-
-$ pulsar-admin clusters delete cluster-1
-
+pulsar-admin clusters delete cluster-1
```
</TabItem>
@@ -232,9 +220,7 @@ $ pulsar-admin clusters delete cluster-1
<TabItem value="Java">
```java
-
admin.clusters().deleteCluster(clusterName);
-
```
</TabItem>
@@ -255,11 +241,9 @@ You can fetch a list of all clusters in a Pulsar [instance](reference-terminolog
Use the [`list`](/tools/pulsar-admin/) subcommand.
```shell
-
-$ pulsar-admin clusters list
+pulsar-admin clusters list
cluster-1
cluster-2
-
```
</TabItem>
@@ -271,9 +255,7 @@ cluster-2
<TabItem value="Java">
```java
-
admin.clusters().getClusters();
-
```
</TabItem>
@@ -294,9 +276,7 @@ Peer clusters can be configured for a given cluster in a Pulsar [instance](refer
Use the [`update-peer-clusters`](/tools/pulsar-admin/) subcommand and specify the list of peer-cluster names.
```
-
-$ pulsar-admin update-peer-clusters cluster-1 --peer-clusters cluster-2
-
+pulsar-admin update-peer-clusters cluster-1 --peer-clusters cluster-2
```
</TabItem>
@@ -308,9 +288,7 @@ $ pulsar-admin update-peer-clusters cluster-1 --peer-clusters cluster-2
<TabItem value="Java">
```java
-
admin.clusters().updatePeerClusterNames(clusterName, peerClusterList);
-
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-functions.md b/site2/website-next/docs/admin-api-functions.md
index 467b3f79ff5..799fa0c54b6 100644
--- a/site2/website-next/docs/admin-api-functions.md
+++ b/site2/website-next/docs/admin-api-functions.md
@@ -55,16 +55,14 @@ Use the [`create`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions create \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --inputs test-input-topic \
- --output persistent://public/default/test-output-topic \
- --classname org.apache.pulsar.functions.api.examples.ExclamationFunction \
- --jar /examples/api-examples.jar
-
+pulsar-admin functions create \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--inputs test-input-topic \
+--output persistent://public/default/test-output-topic \
+--classname org.apache.pulsar.functions.api.examples.ExclamationFunction \
+--jar /examples/api-examples.jar
```
</TabItem>
@@ -76,7 +74,6 @@ $ pulsar-admin functions create \
<TabItem value="Java Admin API">
```java
-
FunctionConfig functionConfig = new FunctionConfig();
functionConfig.setTenant(tenant);
functionConfig.setNamespace(namespace);
@@ -90,7 +87,6 @@ functionConfig.setSubName(subscriptionName);
functionConfig.setAutoAck(true);
functionConfig.setOutput(sinkTopic);
admin.functions().createFunction(functionConfig, fileName);
-
```
</TabItem>
@@ -113,14 +109,12 @@ Use the [`update`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions update \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --output persistent://public/default/update-output-topic \
- # other options
-
+pulsar-admin functions update \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--output persistent://public/default/update-output-topic \
+# other options
```
</TabItem>
@@ -132,7 +126,6 @@ $ pulsar-admin functions update \
<TabItem value="Java Admin API">
```java
-
FunctionConfig functionConfig = new FunctionConfig();
functionConfig.setTenant(tenant);
functionConfig.setNamespace(namespace);
@@ -143,7 +136,6 @@ functionConfig.setClassName("org.apache.pulsar.functions.api.examples.Exclamatio
UpdateOptions updateOptions = new UpdateOptions();
updateOptions.setUpdateAuthData(updateAuthData);
admin.functions().updateFunction(functionConfig, userCodeFile, updateOptions);
-
```
</TabItem>
@@ -164,13 +156,11 @@ You can start a stopped function instance with `instance-id` using Admin CLI, RE
Use the [`start`](/tools/pulsar-admin/) subcommand.
```shell
-
-$ pulsar-admin functions start \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --instance-id 1
-
+pulsar-admin functions start \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--instance-id 1
```
</TabItem>
@@ -182,9 +172,7 @@ $ pulsar-admin functions start \
<TabItem value="Java Admin API">
```java
-
admin.functions().startFunction(tenant, namespace, functionName, Integer.parseInt(instanceId));
-
```
</TabItem>
@@ -207,12 +195,10 @@ Use the [`start`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions start \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
-
+pulsar-admin functions start \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
```
</TabItem>
@@ -224,9 +210,7 @@ $ pulsar-admin functions start \
<TabItem value="Java Admin API">
```java
-
admin.functions().startFunction(tenant, namespace, functionName);
-
```
</TabItem>
@@ -249,13 +233,11 @@ Use the [`stop`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions stop \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --instance-id 1
-
+pulsar-admin functions stop \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--instance-id 1
```
</TabItem>
@@ -267,9 +249,7 @@ $ pulsar-admin functions stop \
<TabItem value="Java Admin API">
```java
-
admin.functions().stopFunction(tenant, namespace, functionName, Integer.parseInt(instanceId));
-
```
</TabItem>
@@ -292,12 +272,10 @@ Use the [`stop`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions stop \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
-
+pulsar-admin functions stop \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
```
</TabItem>
@@ -309,9 +287,7 @@ $ pulsar-admin functions stop \
<TabItem value="Java Admin API">
```java
-
admin.functions().stopFunction(tenant, namespace, functionName);
-
```
</TabItem>
@@ -334,12 +310,11 @@ Use the [`restart`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions restart \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --instance-id 1
+pulsar-admin functions restart \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--instance-id 1
```
@@ -352,9 +327,7 @@ $ pulsar-admin functions restart \
<TabItem value="Java Admin API">
```java
-
admin.functions().restartFunction(tenant, namespace, functionName, Integer.parseInt(instanceId));
-
```
</TabItem>
@@ -377,12 +350,10 @@ Use the [`restart`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions restart \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
-
+pulsar-admin functions restart \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
```
</TabItem>
@@ -394,9 +365,7 @@ $ pulsar-admin functions restart \
<TabItem value="Java Admin API">
```java
-
admin.functions().restartFunction(tenant, namespace, functionName);
-
```
</TabItem>
@@ -419,11 +388,9 @@ Use the [`list`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions list \
- --tenant public \
- --namespace default
-
+pulsar-admin functions list \
+--tenant public \
+--namespace default
```
</TabItem>
@@ -435,9 +402,7 @@ $ pulsar-admin functions list \
<TabItem value="Java Admin API">
```java
-
admin.functions().getFunctions(tenant, namespace);
-
```
</TabItem>
@@ -460,12 +425,10 @@ Use the [`delete`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions delete \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions)
-
+pulsar-admin functions delete \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions)
```
</TabItem>
@@ -477,9 +440,7 @@ $ pulsar-admin functions delete \
<TabItem value="Java Admin API">
```java
-
admin.functions().deleteFunction(tenant, namespace, functionName);
-
```
</TabItem>
@@ -502,12 +463,10 @@ Use the [`get`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions get \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions)
-
+pulsar-admin functions get \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions)
```
</TabItem>
@@ -519,9 +478,7 @@ $ pulsar-admin functions get \
<TabItem value="Java Admin API">
```java
-
admin.functions().getFunction(tenant, namespace, functionName);
-
```
</TabItem>
@@ -543,13 +500,11 @@ Use the [`status`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions status \
+pulsar-admin functions status \
--tenant public \
--namespace default \
--name (the name of Pulsar Functions) \
--instance-id 1
-
```
</TabItem>
@@ -561,9 +516,7 @@ $ pulsar-admin functions status \
<TabItem value="Java Admin API">
```java
-
admin.functions().getFunctionStatus(tenant, namespace, functionName, Integer.parseInt(instanceId));
-
```
</TabItem>
@@ -586,12 +539,10 @@ Use the [`status`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions status \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions)
-
+pulsar-admin functions status \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions)
```
</TabItem>
@@ -603,9 +554,7 @@ $ pulsar-admin functions status \
<TabItem value="Java Admin API">
```java
-
admin.functions().getFunctionStatus(tenant, namespace, functionName);
-
```
</TabItem>
@@ -616,6 +565,7 @@ admin.functions().getFunctionStatus(tenant, namespace, functionName);
### Get stats of an instance of a function
You can get the current stats of a Pulsar Function instance with `instance-id` using Admin CLI, REST API or Java admin API.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
@@ -627,13 +577,11 @@ Use the [`stats`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions stats \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --instance-id 1
-
+pulsar-admin functions stats \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--instance-id 1
```
</TabItem>
@@ -645,9 +593,7 @@ $ pulsar-admin functions stats \
<TabItem value="Java Admin API">
```java
-
admin.functions().getFunctionStats(tenant, namespace, functionName, Integer.parseInt(instanceId));
-
```
</TabItem>
@@ -670,12 +616,10 @@ Use the [`stats`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions stats \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions)
-
+pulsar-admin functions stats \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions)
```
</TabItem>
@@ -687,9 +631,7 @@ $ pulsar-admin functions stats \
<TabItem value="Java Admin API">
```java
-
admin.functions().getFunctionStats(tenant, namespace, functionName);
-
```
</TabItem>
@@ -712,15 +654,13 @@ Use the [`trigger`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions trigger \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --topic (the name of input topic) \
- --trigger-value \"hello pulsar\"
- # or --trigger-file (the path of trigger file)
-
+pulsar-admin functions trigger \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--topic (the name of input topic) \
+--trigger-value \"hello pulsar\"
+# or --trigger-file (the path of trigger file)
```
</TabItem>
@@ -732,9 +672,7 @@ $ pulsar-admin functions trigger \
<TabItem value="Java Admin API">
```java
-
admin.functions().triggerFunction(tenant, namespace, functionName, topic, triggerValue, triggerFile);
-
```
</TabItem>
@@ -757,13 +695,11 @@ Use the [`putstate`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions putstate \
+pulsar-admin functions putstate \
--tenant public \
--namespace default \
--name (the name of Pulsar Functions) \
--state "{\"key\":\"pulsar\", \"stringValue\":\"hello pulsar\"}"
-
```
</TabItem>
@@ -775,11 +711,9 @@ $ pulsar-admin functions putstate \
<TabItem value="Java Admin API">
```java
-
TypeReference<FunctionState> typeRef = new TypeReference<FunctionState>() {};
FunctionState stateRepr = ObjectMapperFactory.getThreadLocal().readValue(state, typeRef);
admin.functions().putFunctionState(tenant, namespace, functionName, stateRepr);
-
```
</TabItem>
@@ -802,13 +736,11 @@ Use the [`querystate`](/tools/pulsar-admin/) subcommand.
**Example**
```shell
-
-$ pulsar-admin functions querystate \
- --tenant public \
- --namespace default \
- --name (the name of Pulsar Functions) \
- --key (the key of state)
-
+pulsar-admin functions querystate \
+--tenant public \
+--namespace default \
+--name (the name of Pulsar Functions) \
+--key (the key of state)
```
</TabItem>
@@ -820,9 +752,7 @@ $ pulsar-admin functions querystate \
<TabItem value="Java Admin API">
```java
-
admin.functions().getFunctionState(tenant, namespace, functionName, key);
-
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-namespaces.md b/site2/website-next/docs/admin-api-namespaces.md
index 85d94ca1403..1a21f82cb5a 100644
--- a/site2/website-next/docs/admin-api-namespaces.md
+++ b/site2/website-next/docs/admin-api-namespaces.md
@@ -45,9 +45,7 @@ You can create new namespaces under a given [tenant](reference-terminology.md#te
Use the [`create`](/tools/pulsar-admin/) subcommand and specify the namespace by name:
```shell
-
-$ pulsar-admin namespaces create test-tenant/test-namespace
-
+pulsar-admin namespaces create test-tenant/test-namespace
```
</TabItem>
@@ -59,9 +57,7 @@ $ pulsar-admin namespaces create test-tenant/test-namespace
<TabItem value="Java">
```java
-
admin.namespaces().createNamespace(namespace);
-
```
</TabItem>
@@ -82,8 +78,7 @@ You can fetch the current policies associated with a namespace at any time.
Use the [`policies`](/tools/pulsar-admin/) subcommand and specify the namespace:
```shell
-
-$ pulsar-admin namespaces policies test-tenant/test-namespace
+pulsar-admin namespaces policies test-tenant/test-namespace
{
"auth_policies": {
"namespace_auth": {},
@@ -105,7 +100,6 @@ $ pulsar-admin namespaces policies test-tenant/test-namespace
"retention_policies": null,
"deleted": false
}
-
```
</TabItem>
@@ -117,9 +111,7 @@ $ pulsar-admin namespaces policies test-tenant/test-namespace
<TabItem value="Java">
```java
-
admin.namespaces().getPolicies(namespace);
-
```
</TabItem>
@@ -140,11 +132,9 @@ You can list all namespaces within a given Pulsar [tenant](reference-terminology
Use the [`list`](/tools/pulsar-admin/) subcommand and specify the tenant:
```shell
-
-$ pulsar-admin namespaces list test-tenant
+pulsar-admin namespaces list test-tenant
test-tenant/ns1
test-tenant/ns2
-
```
</TabItem>
@@ -156,9 +146,7 @@ test-tenant/ns2
<TabItem value="Java">
```java
-
admin.namespaces().getNamespaces(tenant);
-
```
</TabItem>
@@ -179,9 +167,7 @@ You can delete existing namespaces from a tenant.
Use the [`delete`](/tools/pulsar-admin/) subcommand and specify the namespace:
```shell
-
-$ pulsar-admin namespaces delete test-tenant/ns1
-
+pulsar-admin namespaces delete test-tenant/ns1
```
</TabItem>
@@ -193,9 +179,7 @@ $ pulsar-admin namespaces delete test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().deleteNamespace(namespace);
-
```
</TabItem>
@@ -215,11 +199,9 @@ You can set replication clusters for a namespace to enable Pulsar to internally
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-clusters test-tenant/ns1 \
- --clusters cl1
-
+```shell
+pulsar-admin namespaces set-clusters test-tenant/ns1 \
+--clusters cl1
```
</TabItem>
@@ -231,9 +213,7 @@ $ pulsar-admin namespaces set-clusters test-tenant/ns1 \
<TabItem value="Java">
```java
-
admin.namespaces().setNamespaceReplicationClusters(namespace, clusters);
-
```
</TabItem>
@@ -251,16 +231,12 @@ You can get the list of replication clusters for a given namespace.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
+```shell
+pulsar-admin namespaces get-clusters test-tenant/cl1/ns1
```
-$ pulsar-admin namespaces get-clusters test-tenant/cl1/ns1
-
-```
-
-```
-
+```shell
cl2
-
```
</TabItem>
@@ -272,9 +248,7 @@ cl2
<TabItem value="Java">
```java
-
admin.namespaces().getNamespaceReplicationClusters(namespace)
-
```
</TabItem>
@@ -286,15 +260,15 @@ admin.namespaces().getNamespaceReplicationClusters(namespace)
#### Set backlog quota policies
-Backlog quota helps the broker to restrict bandwidth/storage of a namespace once it reaches a certain threshold limit. Admin can set the limit and take corresponding action after the limit is reached.
+Backlog quota helps the broker to restrict bandwidth/storage of a namespace once it reaches a certain threshold limit. Admin can set the limit and take the corresponding action after the limit is reached.
- 1. producer_request_hold: broker holds but not persists produce request payload
+ 1. producer_request_hold: broker holds but does not persist produce request payload
2. producer_exception: broker disconnects with the client by giving an exception
3. consumer_backlog_eviction: broker starts discarding backlog messages
-Backlog quota restriction can be taken care by defining restriction of backlog-quota-type: destination_storage.
+Backlog quota restriction can be taken care of by defining the restriction of backlog-quota-type: destination_storage.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -302,10 +276,8 @@ Backlog quota restriction can be taken care by defining restriction of backlog-q
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-backlog-quota --limit 10G --limitTime 36000 --policy producer_request_hold test-tenant/ns1
-
+```shell
+pulsar-admin namespaces set-backlog-quota --limit 10G --limitTime 36000 --policy producer_request_hold test-tenant/ns1
```
</TabItem>
@@ -317,9 +289,7 @@ $ pulsar-admin namespaces set-backlog-quota --limit 10G --limitTime 36000 --poli
<TabItem value="Java">
```java
-
admin.namespaces().setBacklogQuota(namespace, new BacklogQuota(limit, limitTime, policy))
-
```
</TabItem>
@@ -337,21 +307,17 @@ You can get a configured backlog quota for a given namespace.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-backlog-quotas test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-backlog-quotas test-tenant/ns1
```
```json
-
{
"destination_storage": {
"limit": 10,
"policy": "producer_request_hold"
}
}
-
```
</TabItem>
@@ -363,9 +329,7 @@ $ pulsar-admin namespaces get-backlog-quotas test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getBacklogQuotaMap(namespace);
-
```
</TabItem>
@@ -383,10 +347,8 @@ You can remove backlog quota policies for a given namespace.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces remove-backlog-quota test-tenant/ns1
-
+```shell
+pulsar-admin namespaces remove-backlog-quota test-tenant/ns1
```
</TabItem>
@@ -398,9 +360,7 @@ $ pulsar-admin namespaces remove-backlog-quota test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().removeBacklogQuota(namespace, backlogQuotaType)
-
```
</TabItem>
@@ -428,10 +388,8 @@ Persistence policies allow users to configure persistency-level for all topic me
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 --bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 test-tenant/ns1
-
+```shell
+pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper-ensemble 3 --bookkeeper-write-quorum 2 --ml-mark-delete-max-rate 0 test-tenant/ns1
```
</TabItem>
@@ -443,9 +401,7 @@ $ pulsar-admin namespaces set-persistence --bookkeeper-ack-quorum 2 --bookkeeper
<TabItem value="Java">
```java
-
admin.namespaces().setPersistence(namespace,new PersistencePolicies(bookkeeperEnsemble, bookkeeperWriteQuorum,bookkeeperAckQuorum,managedLedgerMaxMarkDeleteRate))
-
```
</TabItem>
@@ -463,21 +419,17 @@ You can get the configured persistence policies of a given namespace.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-persistence test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-persistence test-tenant/ns1
```
```json
-
{
"bookkeeperEnsemble": 3,
"bookkeeperWriteQuorum": 2,
"bookkeeperAckQuorum": 2,
"managedLedgerMaxMarkDeleteRate": 0
}
-
```
</TabItem>
@@ -489,9 +441,7 @@ $ pulsar-admin namespaces get-persistence test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getPersistence(namespace)
-
```
</TabItem>
@@ -503,7 +453,7 @@ admin.namespaces().getPersistence(namespace)
#### Unload namespace bundles
-The namespace bundle is a virtual group of topics which belong to the same namespace. If the broker gets overloaded with the number of bundles, this command can help unload a bundle from that broker, so it can be served by some other less-loaded brokers. The namespace bundle ID ranges from 0x00000000 to 0xffffffff.
+A namespace bundle is a virtual group of topics that belong to the same namespace. If the broker gets overloaded with the number of bundles, this command can help unload a bundle from that broker, so it can be served by some other less-loaded brokers. The namespace bundle ID ranges from 0x00000000 to 0xffffffff.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -511,10 +461,8 @@ The namespace bundle is a virtual group of topics which belong to the same names
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/ns1
-
+```shell
+pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/ns1
```
</TabItem>
@@ -526,9 +474,7 @@ $ pulsar-admin namespaces unload --bundle 0x00000000_0xffffffff test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().unloadNamespaceBundle(namespace, bundle)
-
```
</TabItem>
@@ -546,25 +492,20 @@ One namespace bundle can contain multiple topics but can be served by only one b
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces split-bundle --bundle 0x00000000_0xffffffff test-tenant/ns1
-
+```shell
+pulsar-admin namespaces split-bundle --bundle 0x00000000_0xffffffff test-tenant/ns1
```
</TabItem>
<TabItem value="REST API">
-
{@inject: endpoint|PUT|/admin/v2/namespaces/:tenant/:namespace/:bundle/split|operation/splitNamespaceBundle?version=@pulsar:version_number@}
</TabItem>
<TabItem value="Java">
```java
-
admin.namespaces().splitNamespaceBundle(namespace, bundle)
-
```
</TabItem>
@@ -584,10 +525,8 @@ You can configure the time to live (in seconds) duration for messages. In the ex
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/ns1
-
+```shell
+pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/ns1
```
</TabItem>
@@ -599,9 +538,7 @@ $ pulsar-admin namespaces set-message-ttl --messageTTL 100 test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().setNamespaceMessageTTL(namespace, messageTTL)
-
```
</TabItem>
@@ -619,16 +556,12 @@ When the message-ttl for a namespace is set, you can use the command below to ge
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-message-ttl test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-message-ttl test-tenant/ns1
```
```
-
100
-
```
</TabItem>
@@ -640,15 +573,11 @@ $ pulsar-admin namespaces get-message-ttl test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getNamespaceMessageTTL(namespace)
-
```
```
-
100
-
```
</TabItem>
@@ -666,10 +595,8 @@ Remove a message TTL of the configured namespace.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces remove-message-ttl test-tenant/ns1
-
+```shell
+pulsar-admin namespaces remove-message-ttl test-tenant/ns1
```
</TabItem>
@@ -681,9 +608,7 @@ $ pulsar-admin namespaces remove-message-ttl test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().removeNamespaceMessageTTL(namespace)
-
```
</TabItem>
@@ -696,7 +621,7 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
#### Clear namespace backlog
-It clears all message backlog for all the topics that belong to a specific namespace. You can also clear backlog for a specific subscription as well.
+It clears all message backlogs for all the topics that belong to a specific namespace. You can also clear backlogs for a specific subscription as well.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -704,10 +629,8 @@ It clears all message backlog for all the topics that belong to a specific names
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/ns1
-
+```shell
+pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/ns1
```
</TabItem>
@@ -719,9 +642,7 @@ $ pulsar-admin namespaces clear-backlog --sub my-subscription test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().clearNamespaceBacklogForSubscription(namespace, subscription)
-
```
</TabItem>
@@ -731,7 +652,7 @@ admin.namespaces().clearNamespaceBacklogForSubscription(namespace, subscription)
#### Clear bundle backlog
-It clears all message backlog for all the topics that belong to a specific NamespaceBundle. You can also clear backlog for a specific subscription as well.
+It clears all message backlogs for all the topics that belong to a specific NamespaceBundle. You can also clear backlogs for a specific subscription as well.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -739,10 +660,8 @@ It clears all message backlog for all the topics that belong to a specific Names
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces clear-backlog --bundle 0x00000000_0xffffffff --sub my-subscription test-tenant/ns1
-
+```shell
+pulsar-admin namespaces clear-backlog --bundle 0x00000000_0xffffffff --sub my-subscription test-tenant/ns1
```
</TabItem>
@@ -776,10 +695,8 @@ Each namespace contains multiple topics and the retention size (storage size) of
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-retention --size 100 --time 10 test-tenant/ns1
-
+```shell
+pulsar-admin namespaces set-retention --size 100 --time 10 test-tenant/ns1
```
</TabItem>
@@ -791,9 +708,7 @@ $ pulsar-admin namespaces set-retention --size 100 --time 10 test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().setRetention(namespace, new RetentionPolicies(retentionTimeInMin, retentionSizeInMB))
-
```
</TabItem>
@@ -803,7 +718,7 @@ admin.namespaces().setRetention(namespace, new RetentionPolicies(retentionTimeIn
#### Get retention
-It shows retention information of a given namespace.
+It shows the retention information of a given namespace.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -811,19 +726,15 @@ It shows retention information of a given namespace.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-retention test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-retention test-tenant/ns1
```
```json
-
{
"retentionTimeInMinutes": 10,
"retentionSizeInMB": 100
}
-
```
</TabItem>
@@ -835,9 +746,7 @@ $ pulsar-admin namespaces get-retention test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getRetention(namespace)
-
```
</TabItem>
@@ -849,9 +758,7 @@ admin.namespaces().getRetention(namespace)
#### Set dispatch throttling for topics
-It sets message dispatch rate for all the topics under a given namespace.
-The dispatch rate can be restricted by the number of messages per X seconds (`msg-dispatch-rate`) or by the number of message-bytes per X second (`byte-dispatch-rate`).
-dispatch rate is in second and it can be configured with `dispatch-rate-period`. Default value of `msg-dispatch-rate` and `byte-dispatch-rate` is -1 which
+It sets the message dispatch rate for all the topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (`msg-dispatch-rate`) or by the number of message-bytes per X second (`byte-dispatch-rate`). The dispatch rate is in second and it can be configured with `dispatch-rate-period`. The default value of `msg-dispatch-rate` and `byte-dispatch-rate` is -1 which
disables the throttling.
:::note
@@ -868,13 +775,11 @@ disables the throttling.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-dispatch-rate test-tenant/ns1 \
- --msg-dispatch-rate 1000 \
- --byte-dispatch-rate 1048576 \
- --dispatch-rate-period 1
-
+```shell
+pulsar-admin namespaces set-dispatch-rate test-tenant/ns1 \
+--msg-dispatch-rate 1000 \
+--byte-dispatch-rate 1048576 \
+--dispatch-rate-period 1
```
</TabItem>
@@ -886,9 +791,7 @@ $ pulsar-admin namespaces set-dispatch-rate test-tenant/ns1 \
<TabItem value="Java">
```java
-
admin.namespaces().setDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
-
```
</TabItem>
@@ -898,7 +801,7 @@ admin.namespaces().setDispatchRate(namespace, new DispatchRate(1000, 1048576, 1)
#### Get configured message-rate for topics
-It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
+It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
````mdx-code-block
<Tabs groupId="api-choice"
@@ -906,20 +809,16 @@ It shows configured message-rate for the namespace (topics under this namespace
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-dispatch-rate test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-dispatch-rate test-tenant/ns1
```
```json
-
{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}
-
```
</TabItem>
@@ -931,9 +830,7 @@ $ pulsar-admin namespaces get-dispatch-rate test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getDispatchRate(namespace)
-
```
</TabItem>
@@ -945,9 +842,7 @@ admin.namespaces().getDispatchRate(namespace)
#### Set dispatch throttling for subscription
-It sets message dispatch rate for all the subscription of topics under a given namespace.
-The dispatch rate can be restricted by the number of messages per X seconds (`msg-dispatch-rate`) or by the number of message-bytes per X second (`byte-dispatch-rate`).
-dispatch rate is in second and it can be configured with `dispatch-rate-period`. Default value of `msg-dispatch-rate` and `byte-dispatch-rate` is -1 which
+It sets the message dispatch rate for all the subscriptions of topics under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (`msg-dispatch-rate`) or by the number of message-bytes per X second (`byte-dispatch-rate`). The dispatch rate is in second and it can be configured with `dispatch-rate-period`. The default value of `msg-dispatch-rate` and `byte-dispatch-rate` is -1 which
disables the throttling.
````mdx-code-block
@@ -956,13 +851,11 @@ disables the throttling.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/ns1 \
- --msg-dispatch-rate 1000 \
- --byte-dispatch-rate 1048576 \
- --dispatch-rate-period 1
-
+```shell
+pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/ns1 \
+--msg-dispatch-rate 1000 \
+--byte-dispatch-rate 1048576 \
+--dispatch-rate-period 1
```
</TabItem>
@@ -974,9 +867,7 @@ $ pulsar-admin namespaces set-subscription-dispatch-rate test-tenant/ns1 \
<TabItem value="Java">
```java
-
admin.namespaces().setSubscriptionDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
-
```
</TabItem>
@@ -986,7 +877,7 @@ admin.namespaces().setSubscriptionDispatchRate(namespace, new DispatchRate(1000,
#### Get configured message-rate for subscription
-It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
+It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second).
````mdx-code-block
<Tabs groupId="api-choice"
@@ -994,20 +885,16 @@ It shows configured message-rate for the namespace (topics under this namespace
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1
```
```json
-
{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}
-
```
</TabItem>
@@ -1019,9 +906,7 @@ $ pulsar-admin namespaces get-subscription-dispatch-rate test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getSubscriptionDispatchRate(namespace)
-
```
</TabItem>
@@ -1029,14 +914,11 @@ admin.namespaces().getSubscriptionDispatchRate(namespace)
</Tabs>
````
-### Configure dispatch throttling for replicator
+### Configure dispatch throttling for replicators
-#### Set dispatch throttling for replicator
+#### Set dispatch throttling for replicators
-It sets message dispatch rate for all the replicator between replication clusters under a given namespace.
-The dispatch rate can be restricted by the number of messages per X seconds (`msg-dispatch-rate`) or by the number of message-bytes per X second (`byte-dispatch-rate`).
-dispatch rate is in second and it can be configured with `dispatch-rate-period`. Default value of `msg-dispatch-rate` and `byte-dispatch-rate` is -1 which
-disables the throttling.
+It sets the message dispatch rate for all the replicators between replication clusters under a given namespace. The dispatch rate can be restricted by the number of messages per X seconds (`msg-dispatch-rate`) or by the number of message-bytes per X second (`byte-dispatch-rate`). The dispatch rate is in second and it can be configured with `dispatch-rate-period`. The default value of `msg-dispatch-rate` and `byte-dispatch-rate` is -1 which disables the throttling.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -1044,13 +926,11 @@ disables the throttling.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/ns1 \
- --msg-dispatch-rate 1000 \
- --byte-dispatch-rate 1048576 \
- --dispatch-rate-period 1
-
+```shell
+pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/ns1 \
+--msg-dispatch-rate 1000 \
+--byte-dispatch-rate 1048576 \
+--dispatch-rate-period 1
```
</TabItem>
@@ -1062,9 +942,7 @@ $ pulsar-admin namespaces set-replicator-dispatch-rate test-tenant/ns1 \
<TabItem value="Java">
```java
-
admin.namespaces().setReplicatorDispatchRate(namespace, new DispatchRate(1000, 1048576, 1))
-
```
</TabItem>
@@ -1072,9 +950,9 @@ admin.namespaces().setReplicatorDispatchRate(namespace, new DispatchRate(1000, 1
</Tabs>
````
-#### Get configured message-rate for replicator
+#### Get configured message-rate for replicators
-It shows configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
+It shows the configured message-rate for the namespace (topics under this namespace can dispatch this many messages per second)
````mdx-code-block
<Tabs groupId="api-choice"
@@ -1082,20 +960,16 @@ It shows configured message-rate for the namespace (topics under this namespace
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/ns1
```
```json
-
{
"dispatchThrottlingRatePerTopicInMsg" : 1000,
"dispatchThrottlingRatePerTopicInByte" : 1048576,
"ratePeriodInSecond" : 1
}
-
```
</TabItem>
@@ -1107,9 +981,7 @@ $ pulsar-admin namespaces get-replicator-dispatch-rate test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getReplicatorDispatchRate(namespace)
-
```
</TabItem>
@@ -1121,7 +993,7 @@ admin.namespaces().getReplicatorDispatchRate(namespace)
#### Get deduplication snapshot interval
-It shows configured `deduplicationSnapshotInterval` for a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)
+It shows the configured `deduplicationSnapshotInterval` for a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)
````mdx-code-block
<Tabs groupId="api-choice"
@@ -1129,10 +1001,8 @@ It shows configured `deduplicationSnapshotInterval` for a namespace (Each topic
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/ns1
-
+```shell
+pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/ns1
```
</TabItem>
@@ -1144,9 +1014,7 @@ $ pulsar-admin namespaces get-deduplication-snapshot-interval test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getDeduplicationSnapshotInterval(namespace)
-
```
</TabItem>
@@ -1156,8 +1024,7 @@ admin.namespaces().getDeduplicationSnapshotInterval(namespace)
#### Set deduplication snapshot interval
-Set configured `deduplicationSnapshotInterval` for a namespace. Each topic under the namespace will take a deduplication snapshot according to this interval.
-`brokerDeduplicationEnabled` must be set to `true` for this property to take effect.
+Set the configured `deduplicationSnapshotInterval` for a namespace. Each topic under the namespace will take a deduplication snapshot according to this interval. `brokerDeduplicationEnabled` must be set to `true` for this property to take effect.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -1165,10 +1032,8 @@ Set configured `deduplicationSnapshotInterval` for a namespace. Each topic under
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/ns1 --interval 1000
-
+```shell
+pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/ns1 --interval 1000
```
</TabItem>
@@ -1180,9 +1045,7 @@ $ pulsar-admin namespaces set-deduplication-snapshot-interval test-tenant/ns1 --
<TabItem value="Java">
```java
-
admin.namespaces().setDeduplicationSnapshotInterval(namespace, 1000)
-
```
</TabItem>
@@ -1192,7 +1055,7 @@ admin.namespaces().setDeduplicationSnapshotInterval(namespace, 1000)
#### Remove deduplication snapshot interval
-Remove configured `deduplicationSnapshotInterval` of a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval)
+Remove configured `deduplicationSnapshotInterval` of a namespace (Each topic under the namespace will take a deduplication snapshot according to this interval).
````mdx-code-block
<Tabs groupId="api-choice"
@@ -1200,10 +1063,8 @@ Remove configured `deduplicationSnapshotInterval` of a namespace (Each topic und
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
-$ pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/ns1
-
+```shell
+pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/ns1
```
</TabItem>
@@ -1215,9 +1076,7 @@ $ pulsar-admin namespaces remove-deduplication-snapshot-interval test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().removeDeduplicationSnapshotInterval(namespace)
-
```
</TabItem>
@@ -1244,9 +1103,7 @@ Use the [`unload`](/tools/pulsar-admin/) subcommand of the [`namespaces`](/tools
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin namespaces unload my-tenant/my-ns
-
+pulsar-admin namespaces unload my-tenant/my-ns
```
</TabItem>
@@ -1258,9 +1115,7 @@ $ pulsar-admin namespaces unload my-tenant/my-ns
<TabItem value="Java">
```java
-
admin.namespaces().unload(namespace)
-
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-overview.md b/site2/website-next/docs/admin-api-overview.md
index 46a45a12f75..7a5ea9e19e7 100644
--- a/site2/website-next/docs/admin-api-overview.md
+++ b/site2/website-next/docs/admin-api-overview.md
@@ -17,9 +17,7 @@ You can interact with the admin interface via:
- The `pulsar-admin` CLI tool, which is available in the `bin` folder of your Pulsar installation:
```shell
-
bin/pulsar-admin
-
```
:::tip
@@ -81,7 +79,6 @@ You can find details for the REST API exposed by Pulsar brokers in this {@inject
To use the Java admin API, instantiate a {@inject: javadoc:PulsarAdmin:/admin/org/apache/pulsar/client/admin/PulsarAdmin} object, and specify a URL for a Pulsar broker and a {@inject: javadoc:PulsarAdminBuilder:/admin/org/apache/pulsar/client/admin/PulsarAdminBuilder}. The following is a minimal example using `localhost`:
```java
-
String url = "http://localhost:8080";
// Pass auth-plugin class fully-qualified name if Pulsar-security enabled
String authPluginClassName = "com.org.MyAuthPluginClass";
@@ -96,13 +93,11 @@ PulsarAdmin admin = PulsarAdmin.builder()
.tlsTrustCertsFilePath(tlsTrustCertsFilePath)
.allowTlsInsecureConnection(tlsAllowInsecureConnection)
.build();
-
```
If you use multiple brokers, you can use multi-host like Pulsar service. For example,
```java
-
String url = "http://localhost:8080,localhost:8081,localhost:8082";
// Pass auth-plugin class fully-qualified name if Pulsar-security enabled
String authPluginClassName = "com.org.MyAuthPluginClass";
@@ -117,7 +112,6 @@ PulsarAdmin admin = PulsarAdmin.builder()
.tlsTrustCertsFilePath(tlsTrustCertsFilePath)
.allowTlsInsecureConnection(tlsAllowInsecureConnection)
.build();
-
```
</TabItem>
@@ -126,9 +120,9 @@ PulsarAdmin admin = PulsarAdmin.builder()
````
## How to define Pulsar resource names when running Pulsar in Kubernetes
-If you run Pulsar Functions or connectors on Kubernetes, you need to follow Kubernetes naming convention to define the names of your Pulsar resources, whichever admin interface you use.
+If you run Pulsar Functions or connectors on Kubernetes, you need to follow the Kubernetes naming convention to define the names of your Pulsar resources, whichever admin interface you use.
-Kubernetes requires a name that can be used as a DNS subdomain name as defined in [RFC 1123](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names). Pulsar supports more legal characters than Kubernetes naming convention. If you create a Pulsar resource name with special characters that are not supported by Kubernetes (for example, including colons in a Pulsar namespace name), Kubernetes runtime translates the Pulsar object names into Kubernetes resource labels w [...]
+Kubernetes requires a name that can be used as a DNS subdomain name as defined in [RFC 1123](https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names). Pulsar supports more legal characters than the Kubernetes naming convention. If you create a Pulsar resource name with special characters that are not supported by Kubernetes (for example, including colons in a Pulsar namespace name), Kubernetes runtime translates the Pulsar object names into Kubernetes resource labe [...]
- Truncate to 63 characters
diff --git a/site2/website-next/docs/admin-api-packages.md b/site2/website-next/docs/admin-api-packages.md
index 1aa75ec7cf6..73437f5cdf5 100644
--- a/site2/website-next/docs/admin-api-packages.md
+++ b/site2/website-next/docs/admin-api-packages.md
@@ -61,11 +61,9 @@ Packages can efficiently use the same set of functions and IO connectors. For ex
Now, you can use the elements you defined in the package by calling this package from within the package manager. The package manager locates it by the URL. For example,
```
-
sink://public/default/mysql-sink@1.0
function://my-tenant/my-ns/my-function@0.1
source://my-tenant/my-ns/mysql-cdc-source@2.3
-
```
## Package management in Pulsar
@@ -74,15 +72,17 @@ You can use the command line tools, REST API, or the Java client to manage your
To use package management service, ensure that the package management service has been enabled in your cluster by setting the following properties in `broker.conf`.
-> Note: Package management service is not enabled by default.
+:::note
-```properties
+Package management service is not enabled by default.
+:::
+
+```properties
enablePackagesManagement=true
packagesManagementStorageProvider=org.apache.pulsar.packages.management.storage.bookkeeper.BookKeeperPackagesStorageProvider
packagesReplicas=1
packagesManagementLedgerRootPath=/ledgers
-
```
### Upload a package
@@ -96,9 +96,7 @@ You can use the following commands to upload a package.
<TabItem value="pulsar-admin">
```shell
-
bin/pulsar-admin packages upload function://public/default/example@v0.1 --path package-file --description package-description
-
```
</TabItem>
@@ -112,17 +110,13 @@ bin/pulsar-admin packages upload function://public/default/example@v0.1 --path p
Upload a package to the package management service synchronously.
```java
-
- void upload(PackageMetadata metadata, String packageName, String path) throws PulsarAdminException;
-
+void upload(PackageMetadata metadata, String packageName, String path) throws PulsarAdminException;
```
Upload a package to the package management service asynchronously.
```java
-
- CompletableFuture<Void> uploadAsync(PackageMetadata metadata, String packageName, String path);
-
+CompletableFuture<Void> uploadAsync(PackageMetadata metadata, String packageName, String path);
```
</TabItem>
@@ -141,9 +135,7 @@ You can use the following commands to download a package.
<TabItem value="pulsar-admin">
```shell
-
bin/pulsar-admin packages download function://public/default/example@v0.1 --path package-file
-
```
</TabItem>
@@ -157,17 +149,13 @@ bin/pulsar-admin packages download function://public/default/example@v0.1 --path
Download a package from the package management service synchronously.
```java
-
- void download(String packageName, String path) throws PulsarAdminException;
-
+void download(String packageName, String path) throws PulsarAdminException;
```
Download a package from the package management service asynchronously.
```java
-
- CompletableFuture<Void> downloadAsync(String packageName, String path);
-
+CompletableFuture<Void> downloadAsync(String packageName, String path);
```
</TabItem>
@@ -188,9 +176,7 @@ You can use the following commands to delete a package.
The following command deletes a package of version 0.1.
```shell
-
bin/pulsar-admin packages delete functions://public/default/example@v0.1
-
```
</TabItem>
@@ -204,17 +190,13 @@ bin/pulsar-admin packages delete functions://public/default/example@v0.1
Delete a specified package synchronously.
```java
-
- void delete(String packageName) throws PulsarAdminException;
-
+void delete(String packageName) throws PulsarAdminException;
```
Delete a specified package asynchronously.
```java
-
- CompletableFuture<Void> deleteAsync(String packageName);
-
+CompletableFuture<Void> deleteAsync(String packageName);
```
</TabItem>
@@ -224,7 +206,7 @@ Delete a specified package asynchronously.
### Get the metadata of a package
-You can use the following commands to get the metadate of a package.
+You can use the following commands to get the metadata of a package.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -233,9 +215,7 @@ You can use the following commands to get the metadate of a package.
<TabItem value="pulsar-admin">
```shell
-
bin/pulsar-admin packages get-metadata function://public/default/test@v1
-
```
</TabItem>
@@ -249,17 +229,13 @@ bin/pulsar-admin packages get-metadata function://public/default/test@v1
Get the metadata of a package synchronously.
```java
-
- PackageMetadata getMetadata(String packageName) throws PulsarAdminException;
-
+PackageMetadata getMetadata(String packageName) throws PulsarAdminException;
```
Get the metadata of a package asynchronously.
```java
-
- CompletableFuture<PackageMetadata> getMetadataAsync(String packageName);
-
+CompletableFuture<PackageMetadata> getMetadataAsync(String packageName);
```
</TabItem>
@@ -278,9 +254,7 @@ You can use the following commands to update the metadata of a package.
<TabItem value="pulsar-admin">
```shell
-
bin/pulsar-admin packages update-metadata function://public/default/example@v0.1 --description update-description
-
```
</TabItem>
@@ -294,17 +268,13 @@ bin/pulsar-admin packages update-metadata function://public/default/example@v0.1
Update the metadata of a package synchronously.
```java
-
- void updateMetadata(String packageName, PackageMetadata metadata) throws PulsarAdminException;
-
+void updateMetadata(String packageName, PackageMetadata metadata) throws PulsarAdminException;
```
Update the metadata of a package asynchronously.
```java
-
- CompletableFuture<Void> updateMetadataAsync(String packageName, PackageMetadata metadata);
-
+CompletableFuture<Void> updateMetadataAsync(String packageName, PackageMetadata metadata);
```
</TabItem>
@@ -323,9 +293,7 @@ You can use the following commands to list all versions of a package.
<TabItem value="pulsar-admin">
```shell
-
bin/pulsar-admin packages list-versions type://tenant/namespace/packageName
-
```
</TabItem>
@@ -339,17 +307,13 @@ bin/pulsar-admin packages list-versions type://tenant/namespace/packageName
List all versions of a package synchronously.
```java
-
- List<String> listPackageVersions(String packageName) throws PulsarAdminException;
-
+List<String> listPackageVersions(String packageName) throws PulsarAdminException;
```
List all versions of a package asynchronously.
```java
-
- CompletableFuture<List<String>> listPackageVersionsAsync(String packageName);
-
+CompletableFuture<List<String>> listPackageVersionsAsync(String packageName);
```
</TabItem>
@@ -369,9 +333,7 @@ You can use the following commands to list all packages of a specific type under
<TabItem value="pulsar-admin">
```shell
-
bin/pulsar-admin packages list --type function public/default
-
```
</TabItem>
@@ -385,17 +347,13 @@ bin/pulsar-admin packages list --type function public/default
List all packages of a specific type under a namespace synchronously.
```java
-
- List<String> listPackages(String type, String namespace) throws PulsarAdminException;
-
+List<String> listPackages(String type, String namespace) throws PulsarAdminException;
```
List all packages of a specific type under a namespace asynchronously.
```java
-
- CompletableFuture<List<String>> listPackagesAsync(String type, String namespace);
-
+CompletableFuture<List<String>> listPackagesAsync(String type, String namespace);
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-permissions.md b/site2/website-next/docs/admin-api-permissions.md
index 734717c0dfb..d1b58ef67a0 100644
--- a/site2/website-next/docs/admin-api-permissions.md
+++ b/site2/website-next/docs/admin-api-permissions.md
@@ -24,9 +24,9 @@ import TabItem from '@theme/TabItem';
Pulsar allows you to grant namespace-level or topic-level permission to users.
-- If you grant a namespace-level permission to a user, then the user can access all the topics under the namespace.
+- If you grant namespace-level permission to a user, then the user can access all the topics under the namespace.
-- If you grant a topic-level permission to a user, then the user can access only the topic.
+- If you grant topic-level permission to a user, then the user can access only the topic.
The chapters below demonstrate how to grant namespace-level permissions to users. For how to grant topic-level permissions to users, see [manage topics](admin-api-topics.md/#grant-permission).
@@ -43,47 +43,44 @@ You can grant permissions to specific roles for lists of operations such as `pro
Use the [`grant-permission`](/tools/pulsar-admin/) subcommand and specify a namespace, actions using the `--actions` flag, and a role using the `--role` flag:
```shell
-
-$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
- --actions produce,consume \
- --role admin10
-
+pulsar-admin namespaces grant-permission test-tenant/ns1 \
+--actions produce,consume \
+--role admin10
```
Wildcard authorization can be performed when `authorizationAllowWildcardsMatching` is set to `true` in `broker.conf`.
-e.g.
+**Example**
```shell
-
-$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
- --actions produce,consume \
- --role 'my.role.*'
+pulsar-admin namespaces grant-permission test-tenant/ns1 \
+ --actions produce,consume \
+ --role 'my.role.*'
```
Then, roles `my.role.1`, `my.role.2`, `my.role.foo`, `my.role.bar`, etc. can produce and consume.
```shell
-
-$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
- --actions produce,consume \
- --role '*.role.my'
-
+pulsar-admin namespaces grant-permission test-tenant/ns1 \
+ --actions produce,consume \
+ --role '*.role.my'
```
Then, roles `1.role.my`, `2.role.my`, `foo.role.my`, `bar.role.my`, etc. can produce and consume.
-**Note**: A wildcard matching works at **the beginning or end of the role name only**.
+:::note
-e.g.
+A wildcard matching works at **the beginning or end of the role name only**.
-```shell
+:::
-$ pulsar-admin namespaces grant-permission test-tenant/ns1 \
- --actions produce,consume \
- --role 'my.*.role'
+**Example**
+```shell
+pulsar-admin namespaces grant-permission test-tenant/ns1 \
+ --actions produce,consume \
+ --role 'my.*.role'
```
In this case, only the role `my.*.role` has permissions.
@@ -98,9 +95,7 @@ Roles `my.1.role`, `my.2.role`, `my.foo.role`, `my.bar.role`, etc. **cannot** pr
<TabItem value="Java">
```java
-
admin.namespaces().grantPermissionOnNamespace(namespace, role, getAuthActions(actions));
-
```
</TabItem>
@@ -121,15 +116,13 @@ You can see which permissions have been granted to which roles in a namespace.
Use the [`permissions`](/tools/pulsar-admin/) subcommand and specify a namespace:
```shell
-
-$ pulsar-admin namespaces permissions test-tenant/ns1
+pulsar-admin namespaces permissions test-tenant/ns1
{
"admin10": [
"produce",
"consume"
]
}
-
```
</TabItem>
@@ -141,9 +134,7 @@ $ pulsar-admin namespaces permissions test-tenant/ns1
<TabItem value="Java">
```java
-
admin.namespaces().getPermissions(namespace);
-
```
</TabItem>
@@ -164,10 +155,8 @@ You can revoke permissions from specific roles, which means that those roles wil
Use the [`revoke-permission`](/tools/pulsar-admin/) subcommand and specify a namespace and a role using the `--role` flag:
```shell
-
-$ pulsar-admin namespaces revoke-permission test-tenant/ns1 \
- --role admin10
-
+pulsar-admin namespaces revoke-permission test-tenant/ns1 \
+--role admin10
```
</TabItem>
@@ -179,9 +168,7 @@ $ pulsar-admin namespaces revoke-permission test-tenant/ns1 \
<TabItem value="Java">
```java
-
admin.namespaces().revokePermissionsOnNamespace(namespace, role);
-
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-tenants.md b/site2/website-next/docs/admin-api-tenants.md
index 9b30d8b5658..e6a1b09437e 100644
--- a/site2/website-next/docs/admin-api-tenants.md
+++ b/site2/website-next/docs/admin-api-tenants.md
@@ -42,11 +42,9 @@ You can list all of the tenants associated with an [instance](reference-terminol
Use the [`list`](/tools/pulsar-admin/) subcommand.
```shell
-
-$ pulsar-admin tenants list
+pulsar-admin tenants list
my-tenant-1
my-tenant-2
-
```
</TabItem>
@@ -58,9 +56,7 @@ my-tenant-2
<TabItem value="Java">
```java
-
admin.tenants().getTenants();
-
```
</TabItem>
@@ -81,9 +77,7 @@ You can create a new tenant.
Use the [`create`](/tools/pulsar-admin/) subcommand:
```shell
-
-$ pulsar-admin tenants create my-tenant
-
+pulsar-admin tenants create my-tenant
```
When creating a tenant, you can optionally assign admin roles using the `-r`/`--admin-roles`
@@ -91,15 +85,13 @@ flag, and clusters using the `-c`/`--allowed-clusters` flag. You can specify mul
as a comma-separated list. Here are some examples:
```shell
+pulsar-admin tenants create my-tenant \
+--admin-roles role1,role2,role3 \
+--allowed-clusters cluster1
-$ pulsar-admin tenants create my-tenant \
- --admin-roles role1,role2,role3 \
- --allowed-clusters cluster1
-
-$ pulsar-admin tenants create my-tenant \
- -r role1
- -c cluster1
-
+pulsar-admin tenants create my-tenant \
+-r role1
+-c cluster1
```
</TabItem>
@@ -111,9 +103,7 @@ $ pulsar-admin tenants create my-tenant \
<TabItem value="Java">
```java
-
admin.tenants().createTenant(tenantName, tenantInfo);
-
```
</TabItem>
@@ -134,8 +124,7 @@ You can fetch the [configuration](reference-configuration.md) for an existing te
Use the [`get`](/tools/pulsar-admin/) subcommand and specify the name of the tenant. Here's an example:
```shell
-
-$ pulsar-admin tenants get my-tenant
+pulsar-admin tenants get my-tenant
{
"adminRoles": [
"admin1",
@@ -146,7 +135,6 @@ $ pulsar-admin tenants get my-tenant
"cl2"
]
}
-
```
</TabItem>
@@ -158,9 +146,7 @@ $ pulsar-admin tenants get my-tenant
<TabItem value="Java">
```java
-
admin.tenants().getTenantInfo(tenantName);
-
```
</TabItem>
@@ -181,9 +167,7 @@ Tenants can be deleted from a Pulsar [instance](reference-terminology.md#instanc
Use the [`delete`](/tools/pulsar-admin/) subcommand and specify the name of the tenant.
```shell
-
-$ pulsar-admin tenants delete my-tenant
-
+pulsar-admin tenants delete my-tenant
```
</TabItem>
@@ -195,9 +179,7 @@ $ pulsar-admin tenants delete my-tenant
<TabItem value="Java">
```java
-
admin.Tenants().deleteTenant(tenantName);
-
```
</TabItem>
@@ -218,9 +200,7 @@ You can update a tenant's configuration.
Use the [`update`](/tools/pulsar-admin/) subcommand.
```shell
-
-$ pulsar-admin tenants update my-tenant
-
+pulsar-admin tenants update my-tenant
```
</TabItem>
@@ -232,9 +212,7 @@ $ pulsar-admin tenants update my-tenant
<TabItem value="Java">
```java
-
admin.tenants().updateTenant(tenantName, tenantInfo);
-
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-topics.md b/site2/website-next/docs/admin-api-topics.md
index 5e9dc46ce24..c2d4ad14636 100644
--- a/site2/website-next/docs/admin-api-topics.md
+++ b/site2/website-next/docs/admin-api-topics.md
@@ -22,24 +22,20 @@ import TabItem from '@theme/TabItem';
:::
-Pulsar has persistent and non-persistent topics. Persistent topic is a logical endpoint for publishing and consuming messages. The topic name structure for persistent topics is:
+Pulsar has persistent and non-persistent topics. A persistent topic is a logical endpoint for publishing and consuming messages. The topic name structure for persistent topics is:
```shell
-
persistent://tenant/namespace/topic
-
```
-Non-persistent topics are used in applications that only consume real-time published messages and do not need persistent guarantee. In this way, it reduces message-publish latency by removing overhead of persisting messages. The topic name structure for non-persistent topics is:
+Non-persistent topics are used in applications that only consume real-time published messages and do not need persistent guarantees. In this way, it reduces message-publish latency by removing overhead of persisting messages. The topic name structure for non-persistent topics is:
```shell
-
non-persistent://tenant/namespace/topic
-
```
## Manage topic resources
-Whether it is persistent or non-persistent topic, you can obtain the topic resources through `pulsar-admin` tool, REST API and Java.
+Whether it is a persistent or non-persistent topic, you can obtain the topic resources through `pulsar-admin` tool, REST API and Java.
:::note
@@ -59,10 +55,8 @@ You can get the list of topics under a given namespace in the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics list \
- my-tenant/my-namespace
-
+pulsar-admin topics list \
+my-tenant/my-namespace
```
</TabItem>
@@ -74,10 +68,8 @@ $ pulsar-admin topics list \
<TabItem value="Java">
```java
-
String namespace = "my-tenant/my-namespace";
admin.topics().getList(namespace);
-
```
</TabItem>
@@ -96,11 +88,9 @@ You can grant permissions on a client role to perform specific actions on a give
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics grant-permission \
- --actions produce,consume --role application1 \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics grant-permission \
+--actions produce,consume --role application1 \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -112,12 +102,10 @@ $ pulsar-admin topics grant-permission \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
String role = "test-role";
Set<AuthAction> actions = Sets.newHashSet(AuthAction.produce, AuthAction.consume);
admin.topics().grantPermission(topic, role, actions);
-
```
</TabItem>
@@ -136,9 +124,8 @@ You can fetch permission in the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics permissions \
- persistent://test-tenant/ns1/tp1 \
+pulsar-admin topics permissions \
+persistent://test-tenant/ns1/tp1 \
{
"application1": [
@@ -146,7 +133,6 @@ $ pulsar-admin topics permissions \
"produce"
]
}
-
```
</TabItem>
@@ -158,10 +144,8 @@ $ pulsar-admin topics permissions \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().getPermissions(topic);
-
```
</TabItem>
@@ -171,7 +155,8 @@ admin.topics().getPermissions(topic);
### Revoke permission
-You can revoke a permission granted on a client role in the following ways.
+You can revoke permissions granted on a client role in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -179,10 +164,9 @@ You can revoke a permission granted on a client role in the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics revoke-permission \
- --role application1 \
- persistent://test-tenant/ns1/tp1 \
+pulsar-admin topics revoke-permission \
+--role application1 \
+persistent://test-tenant/ns1/tp1 \
{
"application1": [
@@ -190,7 +174,6 @@ $ pulsar-admin topics revoke-permission \
"produce"
]
}
-
```
</TabItem>
@@ -202,11 +185,9 @@ $ pulsar-admin topics revoke-permission \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
String role = "test-role";
admin.topics().revokePermissions(topic, role);
-
```
</TabItem>
@@ -216,7 +197,7 @@ admin.topics().revokePermissions(topic, role);
### Delete topic
-You can delete a topic in the following ways. You cannot delete a topic if any active subscription or producers is connected to the topic.
+You can delete a topic in the following ways. You cannot delete a topic if any active subscription or producer is connected to the topic.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -225,10 +206,8 @@ You can delete a topic in the following ways. You cannot delete a topic if any a
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics delete \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics delete \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -240,10 +219,8 @@ $ pulsar-admin topics delete \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().delete(topic);
-
```
</TabItem>
@@ -254,6 +231,7 @@ admin.topics().delete(topic);
### Unload topic
You can unload a topic in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -261,10 +239,8 @@ You can unload a topic in the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics unload \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics unload \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -276,10 +252,8 @@ $ pulsar-admin topics unload \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().unload(topic);
-
```
</TabItem>
@@ -317,7 +291,7 @@ You can check the following statistics of a given non-partitioned topic.
- **backlogSize**: Estimated total unconsumed or backlog size (in bytes).
- - **offloadedStorageSize**: Space used to store the offloaded messages for the topic (in bytes).
+ - **offloadedStorageSize**: Space that is used to store the offloaded messages for the topic (in bytes).
- **waitingPublishers**: The number of publishers waiting in a queue in exclusive access mode.
@@ -331,7 +305,7 @@ You can check the following statistics of a given non-partitioned topic.
- **nonContiguousDeletedMessagesRangesSerializedSize**: The serialized size of non-contiguous deleted messages ranges.
- - **publishers**: The list of all local publishers into the topic. The list ranges from zero to thousands.
+ - **publishers**: The list of all local publishers on the topic. The list ranges from zero to thousands.
- **accessMode**: The type of access to the topic that the producer requires.
@@ -373,13 +347,13 @@ You can check the following statistics of a given non-partitioned topic.
- **lastConsumedFlowTimestamp**: The timestamp of the last flow command received.
- - **lastConsumedTimestamp**: The latest timestamp of all the consumed timestamp of the consumers.
+ - **lastConsumedTimestamp**: The latest timestamp of all the consumed timestamps of the consumers.
- - **lastAckedTimestamp**: The latest timestamp of all the acked timestamp of the consumers.
+ - **lastAckedTimestamp**: The latest timestamp of all the acknowledged timestamps of the consumers.
- - **bytesOutCounter**: Total bytes delivered to consumer.
+ - **bytesOutCounter**: Total bytes delivered to a consumer.
- - **msgOutCounter**: Total messages delivered to consumer.
+ - **msgOutCounter**: Total messages delivered to a consumer.
- **msgRateRedeliver**: Total rate of messages redelivered on this subscription (msg/s).
@@ -391,11 +365,11 @@ You can check the following statistics of a given non-partitioned topic.
- **msgBacklogNoDelayed**: Number of messages in the subscription backlog that do not contain the delay messages.
- - **blockedSubscriptionOnUnackedMsgs**: Flag to verify if a subscription is blocked due to reaching threshold of unacked messages.
+ - **blockedSubscriptionOnUnackedMsgs**: Flag to verify if a subscription is blocked due to reaching the threshold of unacked messages.
- **msgDelayed**: Number of delayed messages currently being tracked.
- - **unackedMessages**: Number of unacknowledged messages for the subscription, where an unacknowledged message is one that has been sent to a consumer but not yet acknowledged. This field is only meaningful when using a subscription that tracks individual message acknowledgement.
+ - **unackedMessages**: Number of unacknowledged messages for the subscription, where an unacknowledged message is one that has been sent to a consumer but not yet acknowledged. This field is only meaningful when using a subscription that tracks individual message acknowledgment.
- **activeConsumerName**: The name of the consumer that is active for single active consumer subscriptions. For example, failover or exclusive.
@@ -407,7 +381,7 @@ You can check the following statistics of a given non-partitioned topic.
- **replicated**: Mark that the subscription state is kept in sync across different regions.
- - **allowOutOfOrderDelivery**: Whether out of order delivery is allowed on the Key_Shared subscription.
+ - **allowOutOfOrderDelivery**: Whether out-of-order delivery is allowed on the Key_Shared subscription.
- **keySharedMode**: Whether the Key_Shared subscription mode is AUTO_SPLIT or STICKY.
@@ -433,11 +407,11 @@ You can check the following statistics of a given non-partitioned topic.
- **consumerName**: The internal identifier for this consumer, generated by the client library.
- - **availablePermits**: The number of messages that the consumer has space for in the client library's listen queue. `0` means the client library's queue is full and `receive()` isn't being called. A non-zero value means this consumer is ready for dispatched messages.
+ - **availablePermits**: The number of messages that the consumer has space for in the client library's listening queue. `0` means the client library's queue is full and `receive()` isn't being called. A non-zero value means this consumer is ready for dispatched messages.
- - **unackedMessages**: The number of unacknowledged messages for the consumer, where an unacknowledged message is one that has been sent to the consumer but not yet acknowledged. This field is only meaningful when using a subscription that tracks individual message acknowledgement.
+ - **unackedMessages**: The number of unacknowledged messages for the consumer, where an unacknowledged message has been sent to the consumer but not yet acknowledged. This field is only meaningful when using a subscription that tracks individual message acknowledgment.
- - **blockedConsumerOnUnackedMsgs**: The flag used to verify if the consumer is blocked due to reaching threshold of the unacknowledged messages.
+ - **blockedConsumerOnUnackedMsgs**: The flag used to verify if the consumer is blocked due to reaching the threshold of the unacknowledged messages.
- **lastConsumedTimestamp**: The timestamp when the consumer reads a message the last time.
@@ -449,9 +423,9 @@ You can check the following statistics of a given non-partitioned topic.
- **clientVersion**: The client library version of this consumer.
- - **bytesOutCounter**: Total bytes delivered to consumer.
+ - **bytesOutCounter**: Total bytes delivered to a consumer.
- - **msgOutCounter**: Total messages delivered to consumer.
+ - **msgOutCounter**: Total messages delivered to a consumer.
- **msgRateRedeliver**: Total rate of messages redelivered by this consumer (msg/s).
@@ -459,7 +433,7 @@ You can check the following statistics of a given non-partitioned topic.
- **avgMessagesPerEntry**: Number of average messages per entry for the consumer consumed.
- - **readPositionWhenJoining**: The read position of the cursor when the consumer joining.
+ - **readPositionWhenJoining**: The read position of the cursor when the consumer joins.
- **keyHashRanges**: Hash ranges assigned to this consumer if is Key_Shared sub mode.
@@ -489,12 +463,11 @@ You can check the following statistics of a given non-partitioned topic.
- **outboundConnection**: The address of the outbound replication connection.
- - **outboundConnectedSince**: The timestamp of establishing outbound connection.
+ - **outboundConnectedSince**: The timestamp of establishing an outbound connection.
The following is an example of a topic status.
```json
-
{
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
@@ -585,7 +558,6 @@ The following is an example of a topic status.
"nonContiguousDeletedMessagesRanges" : 0,
"nonContiguousDeletedMessagesRangesSerializedSize" : 0
}
-
```
To get the status of a topic, you can use the following ways.
@@ -597,10 +569,8 @@ To get the status of a topic, you can use the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics stats \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics stats \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -612,10 +582,8 @@ $ pulsar-admin topics stats \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().getStats(topic);
-
```
</TabItem>
@@ -645,7 +613,7 @@ You can get the detailed statistics of a topic.
- **pendingAddEntriesCount**: The number of messages that complete (asynchronous) write requests.
- - **lastConfirmedEntry**: The ledgerid:entryid of the last message that is written successfully. If the entryid is `-1`, then the ledger is open, yet no entries are written.
+ - **lastConfirmedEntry**: The `ledgerid:entryid` of the last message that is written successfully. If the `entryid` is `-1`, then the ledger is open, yet no entries are written.
- **state**: The state of this ledger for writing. The state `LedgerOpened` means that a ledger is open for saving published messages.
@@ -687,17 +655,17 @@ You can get the detailed statistics of a topic.
- **markDeletePosition**: All messages before the markDeletePosition are acknowledged by the subscriber.
- - **readPosition**: The latest position of subscriber for reading message.
+ - **readPosition**: The latest position of subscriber for reading messages.
- **waitingReadOp**: This is true when the subscription has read the latest message published to the topic and is waiting for new messages to be published.
- **pendingReadOps**: The counter for how many outstanding read requests to the BookKeepers in progress.
- - **messagesConsumedCounter**: The number of messages this cursor has acked since this broker loaded this topic.
+ - **messagesConsumedCounter**: The number of messages this cursor has acknowledged since this broker loaded this topic.
- **cursorLedger**: The ledger being used to persistently store the current markDeletePosition.
- - **cursorLedgerLastEntry**: The last entryid used to persistently store the current markDeletePosition.
+ - **cursorLedgerLastEntry**: The last `entryid` used to persistently store the current markDeletePosition.
- **individuallyDeletedMessages**: If acknowledges are being done out of order, the ranges of messages acknowledged between the markDeletePosition and the read-position shows.
@@ -708,7 +676,6 @@ You can get the detailed statistics of a topic.
The following is an example of the detailed statistics of a topic.
```json
-
{
"entriesAddedCounter":0,
"numberOfEntries":0,
@@ -766,10 +733,10 @@ The following is an example of the detailed statistics of a topic.
"metadata":null
}
}
-
```
To get the internal status of a topic, you can use the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -777,10 +744,8 @@ To get the internal status of a topic, you can use the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics stats-internal \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics stats-internal \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -792,10 +757,8 @@ $ pulsar-admin topics stats-internal \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().getInternalStats(topic);
-
```
</TabItem>
@@ -806,6 +769,7 @@ admin.topics().getInternalStats(topic);
### Peek messages
You can peek a number of messages for a specific subscription of a given topic in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -813,15 +777,13 @@ You can peek a number of messages for a specific subscription of a given topic i
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics peek-messages \
- --count 10 --subscription my-subscription \
- persistent://test-tenant/ns1/tp1 \
+pulsar-admin topics peek-messages \
+--count 10 --subscription my-subscription \
+persistent://test-tenant/ns1/tp1 \
Message ID: 315674752:0
Properties: { "X-Pulsar-publish-time" : "2015-07-13 17:40:28.451" }
msg-payload
-
```
</TabItem>
@@ -833,12 +795,10 @@ msg-payload
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
String subName = "my-subscription";
int numMessages = 1;
admin.topics().peekMessages(topic, subName, numMessages);
-
```
</TabItem>
@@ -857,11 +817,9 @@ You can fetch the message with the given ledger ID and entry ID in the following
<TabItem value="pulsar-admin">
```shell
-
-$ ./bin/pulsar-admin topics get-message-by-id \
- persistent://public/default/my-topic \
- -l 10 -e 0
-
+./bin/pulsar-admin topics get-message-by-id \
+persistent://public/default/my-topic \
+-l 10 -e 0
```
</TabItem>
@@ -873,12 +831,10 @@ $ ./bin/pulsar-admin topics get-message-by-id \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
long ledgerId = 10;
long entryId = 10;
admin.topics().getMessageById(topic, ledgerId, entryId);
-
```
</TabItem>
@@ -897,11 +853,9 @@ You can examine a specific message on a topic by position relative to the earlie
<TabItem value="pulsar-admin">
```shell
-
./bin/pulsar-admin topics examine-messages \
- persistent://public/default/my-topic \
- -i latest -m 1
-
+persistent://public/default/my-topic \
+-i latest -m 1
```
</TabItem>
@@ -913,10 +867,8 @@ You can examine a specific message on a topic by position relative to the earlie
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().examineMessage(topic, "latest", 1);
-
```
</TabItem>
@@ -935,11 +887,9 @@ You can get message ID published at or just after the given datetime.
<TabItem value="pulsar-admin">
```shell
-
./bin/pulsar-admin topics get-message-id \
- persistent://public/default/my-topic \
- -d 2021-06-28T19:01:17Z
-
+persistent://public/default/my-topic \
+-d 2021-06-28T19:01:17Z
```
</TabItem>
@@ -951,11 +901,9 @@ You can get message ID published at or just after the given datetime.
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
long timestamp = System.currentTimeMillis()
admin.topics().getMessageIdByTimestamp(topic, timestamp);
-
```
</TabItem>
@@ -975,11 +923,9 @@ You can skip a number of messages for a specific subscription of a given topic i
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics skip \
- --count 10 --subscription my-subscription \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics skip \
+--count 10 --subscription my-subscription \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -991,12 +937,10 @@ $ pulsar-admin topics skip \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
String subName = "my-subscription";
int numMessages = 1;
admin.topics().skipMessages(topic, subName, numMessages);
-
```
</TabItem>
@@ -1015,11 +959,9 @@ You can skip all the old messages for a specific subscription of a given topic.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics skip-all \
- --subscription my-subscription \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics skip-all \
+--subscription my-subscription \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -1031,11 +973,9 @@ $ pulsar-admin topics skip-all \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
String subName = "my-subscription";
admin.topics().skipAllMessages(topic, subName);
-
```
</TabItem>
@@ -1045,7 +985,7 @@ admin.topics().skipAllMessages(topic, subName);
### Reset cursor
-You can reset a subscription cursor position back to the position which is recorded X minutes before. It essentially calculates time and position of cursor at X minutes before and resets it at that position. You can reset the cursor in the following ways.
+You can reset a subscription cursor position back to the position which is recorded X minutes before. It essentially calculates the time and position of the cursor at X minutes before and resets it at that position. You can reset the cursor in the following ways.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -1054,11 +994,9 @@ You can reset a subscription cursor position back to the position which is recor
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics reset-cursor \
- --subscription my-subscription --time 10 \
- persistent://test-tenant/ns1/tp1 \
-
+pulsar-admin topics reset-cursor \
+--subscription my-subscription --time 10 \
+persistent://test-tenant/ns1/tp1 \
```
</TabItem>
@@ -1094,12 +1032,10 @@ You can locate the owner broker of the given topic in the following ways.
<TabItem value="pulsar-admin">
```shell
+pulsar-admin topics lookup \
+persistent://test-tenant/ns1/tp1 \
-$ pulsar-admin topics lookup \
- persistent://test-tenant/ns1/tp1 \
-
- "pulsar://broker1.org.com:4480"
-
+"pulsar://broker1.org.com:4480"
```
</TabItem>
@@ -1111,10 +1047,8 @@ $ pulsar-admin topics lookup \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.lookup().lookupDestination(topic);
-
```
</TabItem>
@@ -1133,36 +1067,30 @@ You can locate the owner broker of the given partitioned topic in the following
<TabItem value="pulsar-admin">
```shell
+pulsar-admin topics partitioned-lookup \
+persistent://test-tenant/ns1/my-topic \
-$ pulsar-admin topics partitioned-lookup \
- persistent://test-tenant/ns1/my-topic \
-
- "persistent://test-tenant/ns1/my-topic-partition-0 pulsar://localhost:6650"
- "persistent://test-tenant/ns1/my-topic-partition-1 pulsar://localhost:6650"
- "persistent://test-tenant/ns1/my-topic-partition-2 pulsar://localhost:6650"
- "persistent://test-tenant/ns1/my-topic-partition-3 pulsar://localhost:6650"
-
+"persistent://test-tenant/ns1/my-topic-partition-0 pulsar://localhost:6650"
+"persistent://test-tenant/ns1/my-topic-partition-1 pulsar://localhost:6650"
+"persistent://test-tenant/ns1/my-topic-partition-2 pulsar://localhost:6650"
+"persistent://test-tenant/ns1/my-topic-partition-3 pulsar://localhost:6650"
```
</TabItem>
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.lookup().lookupPartitionedTopic(topic);
-
```
Lookup the partitioned topics sorted by broker URL
```shell
+pulsar-admin topics partitioned-lookup \
+persistent://test-tenant/ns1/my-topic --sort-by-broker \
-$ pulsar-admin topics partitioned-lookup \
- persistent://test-tenant/ns1/my-topic --sort-by-broker \
-
- "pulsar://localhost:6650 [persistent://test-tenant/ns1/my-topic-partition-0, persistent://test-tenant/ns1/my-topic-partition-1, persistent://test-tenant/ns1/my-topic-partition-2, persistent://test-tenant/ns1/my-topic-partition-3]"
-
+"pulsar://localhost:6650 [persistent://test-tenant/ns1/my-topic-partition-0, persistent://test-tenant/ns1/my-topic-partition-1, persistent://test-tenant/ns1/my-topic-partition-2, persistent://test-tenant/ns1/my-topic-partition-3]"
```
</TabItem>
@@ -1181,12 +1109,10 @@ You can get the range of the bundle that the given topic belongs to in the follo
<TabItem value="pulsar-admin">
```shell
+pulsar-admin topics bundle-range \
+persistent://test-tenant/ns1/tp1 \
-$ pulsar-admin topics bundle-range \
- persistent://test-tenant/ns1/tp1 \
-
- "0x00000000_0xffffffff"
-
+"0x00000000_0xffffffff"
```
</TabItem>
@@ -1198,10 +1124,8 @@ $ pulsar-admin topics bundle-range \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.lookup().getBundleRange(topic);
-
```
</TabItem>
@@ -1220,12 +1144,10 @@ You can check all subscription names for a given topic in the following ways.
<TabItem value="pulsar-admin">
```shell
+pulsar-admin topics subscriptions \
+persistent://test-tenant/ns1/tp1 \
-$ pulsar-admin topics subscriptions \
- persistent://test-tenant/ns1/tp1 \
-
- my-subscription
-
+my-subscription
```
</TabItem>
@@ -1237,10 +1159,8 @@ $ pulsar-admin topics subscriptions \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().getSubscriptions(topic);
-
```
</TabItem>
@@ -1259,9 +1179,7 @@ You can get the last committed message ID for a persistent topic. It is availabl
<TabItem value="pulsar-admin">
```shell
-
pulsar-admin topics last-message-id topic-name
-
```
</TabItem>
@@ -1273,10 +1191,8 @@ pulsar-admin topics last-message-id topic-name
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().getLastMessage(topic);
-
```
</TabItem>
@@ -1295,11 +1211,9 @@ You can get the backlog size of a single partition topic or a non-partitioned to
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics get-backlog-size \
+pulsar-admin topics get-backlog-size \
-m 1:1 \
persistent://test-tenant/ns1/tp1-partition-0 \
-
```
</TabItem>
@@ -1311,11 +1225,9 @@ $ pulsar-admin topics get-backlog-size \
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
MessageId messageId = MessageId.earliest;
admin.topics().getBacklogSizeByMessageId(topic, messageId);
-
```
</TabItem>
@@ -1336,10 +1248,8 @@ To get the topic-level deduplication snapshot interval, use one of the following
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics get-deduplication-snapshot-interval options
-
```
</TabItem>
@@ -1351,9 +1261,7 @@ pulsar-admin topics get-deduplication-snapshot-interval options
<TabItem value="Java">
```java
-
admin.topics().getDeduplicationSnapshotInterval(topic)
-
```
</TabItem>
@@ -1373,10 +1281,8 @@ To set the topic-level deduplication snapshot interval, use one of the following
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics set-deduplication-snapshot-interval options
-
```
</TabItem>
@@ -1385,20 +1291,16 @@ pulsar-admin topics set-deduplication-snapshot-interval options
{@inject: endpoint|POST|/admin/v2/topics/:tenant/:namespace/:topic/deduplicationSnapshotInterval|operation/setDeduplicationSnapshotInterval?version=@pulsar:version_number@}
```json
-
{
"interval": 1000
}
-
```
</TabItem>
<TabItem value="Java">
```java
-
admin.topics().setDeduplicationSnapshotInterval(topic, 1000)
-
```
</TabItem>
@@ -1416,10 +1318,8 @@ To remove the topic-level deduplication snapshot interval, use one of the follow
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics remove-deduplication-snapshot-interval options
-
```
</TabItem>
@@ -1431,9 +1331,7 @@ pulsar-admin topics remove-deduplication-snapshot-interval options
<TabItem value="Java">
```java
-
admin.topics().removeDeduplicationSnapshotInterval(topic)
-
```
</TabItem>
@@ -1454,10 +1352,8 @@ To get the topic-level inactive topic policies, use one of the following methods
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics get-inactive-topic-policies options
-
```
</TabItem>
@@ -1469,9 +1365,7 @@ pulsar-admin topics get-inactive-topic-policies options
<TabItem value="Java">
```java
-
admin.topics().getInactiveTopicPolicies(topic)
-
```
</TabItem>
@@ -1489,10 +1383,8 @@ To set the topic-level inactive topic policies, use one of the following methods
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics set-inactive-topic-policies options
-
```
</TabItem>
@@ -1504,9 +1396,7 @@ pulsar-admin topics set-inactive-topic-policies options
<TabItem value="Java">
```java
-
admin.topics().setInactiveTopicPolicies(topic, inactiveTopicPolicies)
-
```
</TabItem>
@@ -1524,10 +1414,8 @@ To remove the topic-level inactive topic policies, use one of the following meth
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics remove-inactive-topic-policies options
-
```
</TabItem>
@@ -1539,9 +1427,7 @@ pulsar-admin topics remove-inactive-topic-policies options
<TabItem value="Java">
```java
-
admin.topics().removeInactiveTopicPolicies(topic)
-
```
</TabItem>
@@ -1562,10 +1448,8 @@ To get the topic-level offload policies, use one of the following methods.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics get-offload-policies options
-
```
</TabItem>
@@ -1577,9 +1461,7 @@ pulsar-admin topics get-offload-policies options
<TabItem value="Java">
```java
-
admin.topics().getOffloadPolicies(topic)
-
```
</TabItem>
@@ -1597,10 +1479,8 @@ To set the topic-level offload policies, use one of the following methods.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics set-offload-policies options
-
```
</TabItem>
@@ -1612,9 +1492,7 @@ pulsar-admin topics set-offload-policies options
<TabItem value="Java">
```java
-
admin.topics().setOffloadPolicies(topic, offloadPolicies)
-
```
</TabItem>
@@ -1632,10 +1510,8 @@ To remove the topic-level offload policies, use one of the following methods.
values={[{"label":"pulsar-admin","value":"pulsar-admin"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="pulsar-admin">
-```
-
+```shell
pulsar-admin topics remove-offload-policies options
-
```
</TabItem>
@@ -1647,9 +1523,7 @@ pulsar-admin topics remove-offload-policies options
<TabItem value="Java">
```java
-
admin.topics().removeOffloadPolicies(topic)
-
```
</TabItem>
@@ -1659,7 +1533,7 @@ admin.topics().removeOffloadPolicies(topic)
## Manage non-partitioned topics
-You can use Pulsar [admin API](admin-api-overview.md) to create, delete and check status of non-partitioned topics.
+You can use Pulsar [admin API](admin-api-overview.md) to create, delete and check the status of non-partitioned topics.
### Create
Non-partitioned topics must be explicitly created. When creating a new non-partitioned topic, you need to provide a name for the topic.
@@ -1669,6 +1543,7 @@ By default, 60 seconds after creation, topics are considered inactive and delete
For more information about the two parameters, see [here](reference-configuration.md#broker).
You can create non-partitioned topics in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -1678,10 +1553,8 @@ You can create non-partitioned topics in the following ways.
When you create non-partitioned topics with the [`create`](/tools/pulsar-admin/) command, you need to specify the topic name as an argument.
```shell
-
-$ bin/pulsar-admin topics create \
- persistent://my-tenant/my-namespace/my-topic
-
+bin/pulsar-admin topics create \
+persistent://my-tenant/my-namespace/my-topic
```
:::note
@@ -1699,10 +1572,8 @@ When you create a non-partitioned topic with the suffix '-partition-' followed b
<TabItem value="Java">
```java
-
String topicName = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().createNonPartitionedTopic(topicName);
-
```
</TabItem>
@@ -1711,7 +1582,9 @@ admin.topics().createNonPartitionedTopic(topicName);
````
### Delete
+
You can delete non-partitioned topics in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -1719,10 +1592,8 @@ You can delete non-partitioned topics in the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ bin/pulsar-admin topics delete \
- persistent://my-tenant/my-namespace/my-topic
-
+bin/pulsar-admin topics delete \
+persistent://my-tenant/my-namespace/my-topic
```
</TabItem>
@@ -1734,9 +1605,7 @@ $ bin/pulsar-admin topics delete \
<TabItem value="Java">
```java
-
admin.topics().delete(topic);
-
```
</TabItem>
@@ -1746,7 +1615,8 @@ admin.topics().delete(topic);
### List
-You can get the list of topics under a given namespace in the following ways.
+You can get the list of topics under a given namespace in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -1754,11 +1624,9 @@ You can get the list of topics under a given namespace in the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics list tenant/namespace
+pulsar-admin topics list tenant/namespace
persistent://tenant/namespace/topic1
persistent://tenant/namespace/topic2
-
```
</TabItem>
@@ -1770,9 +1638,7 @@ persistent://tenant/namespace/topic2
<TabItem value="Java">
```java
-
admin.topics().getList(namespace);
-
```
</TabItem>
@@ -1782,10 +1648,9 @@ admin.topics().getList(namespace);
### Stats
-You can check the current statistics of a given topic. The following is an example. For description of each stats, refer to [get stats](#get-stats).
+You can check the current statistics of a given topic. The following is an example. For the description of each stats, refer to [get stats](#get-stats).
```json
-
{
"msgRateIn": 4641.528542257553,
"msgThroughputIn": 44663039.74947473,
@@ -1816,10 +1681,10 @@ You can check the current statistics of a given topic. The following is an examp
},
"replication": {}
}
-
```
You can check the current statistics of a given topic and its connected producers and consumers in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -1827,11 +1692,9 @@ You can check the current statistics of a given topic and its connected producer
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics stats \
- persistent://test-tenant/namespace/topic \
- --get-precise-backlog
-
+pulsar-admin topics stats \
+persistent://test-tenant/namespace/topic \
+--get-precise-backlog
```
</TabItem>
@@ -1843,9 +1706,7 @@ $ pulsar-admin topics stats \
<TabItem value="Java">
```java
-
admin.topics().getStats(topic, false /* is precise backlog */);
-
```
</TabItem>
@@ -1854,7 +1715,7 @@ admin.topics().getStats(topic, false /* is precise backlog */);
````
## Manage partitioned topics
-You can use Pulsar [admin API](admin-api-overview.md) to create, update, delete and check status of partitioned topics.
+You can use Pulsar [admin API](admin-api-overview.md) to create, update, delete and check the status of partitioned topics.
### Create
@@ -1865,6 +1726,7 @@ By default, 60 seconds after creation, topics are considered inactive and delete
For more information about the two parameters, see [here](reference-configuration.md#broker).
You can create partitioned topics in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -1875,11 +1737,9 @@ When you create partitioned topics with the [`create-partitioned-topic`](/tools/
command, you need to specify the topic name as an argument and the number of partitions using the `-p` or `--partitions` flag.
```shell
-
-$ bin/pulsar-admin topics create-partitioned-topic \
- persistent://my-tenant/my-namespace/my-topic \
- --partitions 4
-
+bin/pulsar-admin topics create-partitioned-topic \
+persistent://my-tenant/my-namespace/my-topic \
+--partitions 4
```
:::note
@@ -1897,11 +1757,9 @@ If a non-partitioned topic with the suffix '-partition-' followed by a numeric v
<TabItem value="Java">
```java
-
String topicName = "persistent://my-tenant/my-namespace/my-topic";
int numPartitions = 4;
admin.topics().createPartitionedTopic(topicName, numPartitions);
-
```
</TabItem>
@@ -1922,10 +1780,8 @@ When topic auto-creation is disabled, and you have a partitioned topic without a
You can create missed partitions with the [`create-missed-partitions`](/tools/pulsar-admin/) command and specify the topic name as an argument.
```shell
-
-$ bin/pulsar-admin topics create-missed-partitions \
- persistent://my-tenant/my-namespace/my-topic \
-
+bin/pulsar-admin topics create-missed-partitions \
+persistent://my-tenant/my-namespace/my-topic \
```
</TabItem>
@@ -1937,10 +1793,8 @@ $ bin/pulsar-admin topics create-missed-partitions \
<TabItem value="Java">
```java
-
String topicName = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().createMissedPartitions(topicName);
-
```
</TabItem>
@@ -1965,13 +1819,11 @@ Field | Description
You can check the number of partitions in a partitioned topic with the [`get-partitioned-topic-metadata`](/tools/pulsar-admin/) subcommand.
```shell
-
-$ pulsar-admin topics get-partitioned-topic-metadata \
- persistent://my-tenant/my-namespace/my-topic
+pulsar-admin topics get-partitioned-topic-metadata \
+persistent://my-tenant/my-namespace/my-topic
{
"partitions": 4
}
-
```
</TabItem>
@@ -1983,10 +1835,8 @@ $ pulsar-admin topics get-partitioned-topic-metadata \
<TabItem value="Java">
```java
-
String topicName = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().getPartitionedTopicMetadata(topicName);
-
```
</TabItem>
@@ -2009,11 +1859,9 @@ Producers and consumers can find the newly created partitions automatically.
You can update partitioned topics with the [`update-partitioned-topic`](/tools/pulsar-admin/) command.
```shell
-
-$ pulsar-admin topics update-partitioned-topic \
- persistent://my-tenant/my-namespace/my-topic \
- --partitions 8
-
+pulsar-admin topics update-partitioned-topic \
+persistent://my-tenant/my-namespace/my-topic \
+--partitions 8
```
</TabItem>
@@ -2025,9 +1873,7 @@ $ pulsar-admin topics update-partitioned-topic \
<TabItem value="Java">
```java
-
admin.topics().updatePartitionedTopic(topic, numPartitions);
-
```
</TabItem>
@@ -2045,10 +1891,8 @@ You can delete partitioned topics with the [`delete-partitioned-topic`](/tools/p
<TabItem value="pulsar-admin">
```shell
-
-$ bin/pulsar-admin topics delete-partitioned-topic \
- persistent://my-tenant/my-namespace/my-topic
-
+bin/pulsar-admin topics delete-partitioned-topic \
+persistent://my-tenant/my-namespace/my-topic
```
</TabItem>
@@ -2060,9 +1904,7 @@ $ bin/pulsar-admin topics delete-partitioned-topic \
<TabItem value="Java">
```java
-
admin.topics().delete(topic);
-
```
</TabItem>
@@ -2071,7 +1913,9 @@ admin.topics().delete(topic);
````
### List
-You can get the list of partitioned topics under a given namespace in the following ways.
+
+You can get the list of partitioned topics under a given namespace in the following ways.
+
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="pulsar-admin"
@@ -2079,11 +1923,9 @@ You can get the list of partitioned topics under a given namespace in the follow
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics list-partitioned-topics tenant/namespace
+pulsar-admin topics list-partitioned-topics tenant/namespace
persistent://tenant/namespace/topic1
persistent://tenant/namespace/topic2
-
```
</TabItem>
@@ -2095,9 +1937,7 @@ persistent://tenant/namespace/topic2
<TabItem value="Java">
```java
-
admin.topics().getPartitionedTopicList(namespace);
-
```
</TabItem>
@@ -2107,12 +1947,11 @@ admin.topics().getPartitionedTopicList(namespace);
### Stats
-You can check the current statistics of a given partitioned topic. The following is an example. For description of each stats, refer to [get stats](#get-stats).
+You can check the current statistics of a given partitioned topic. The following is an example. For the description of each stats, refer to [get stats](#get-stats).
Note that in the subscription JSON object, `chuckedMessageRate` is deprecated. Please use `chunkedMessageRate`. Both will be sent in the JSON for now.
```json
-
{
"msgRateIn" : 999.992947159793,
"msgThroughputIn" : 1070918.4635439808,
@@ -2163,7 +2002,6 @@ Note that in the subscription JSON object, `chuckedMessageRate` is deprecated. P
},
"partitions" : { }
}
-
```
You can check the current statistics of a given partitioned topic and its connected producers and consumers in the following ways.
@@ -2175,11 +2013,9 @@ You can check the current statistics of a given partitioned topic and its connec
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics partitioned-stats \
- persistent://test-tenant/namespace/topic \
- --per-partition
-
+pulsar-admin topics partitioned-stats \
+persistent://test-tenant/namespace/topic \
+--per-partition
```
</TabItem>
@@ -2191,9 +2027,7 @@ $ pulsar-admin topics partitioned-stats \
<TabItem value="Java">
```java
-
admin.topics().getPartitionedStats(topic, true /* per partition */, false /* is precise backlog */);
-
```
</TabItem>
@@ -2203,10 +2037,9 @@ admin.topics().getPartitionedStats(topic, true /* per partition */, false /* is
### Internal stats
-You can check the detailed statistics of a topic. The following is an example. For description of each stats, refer to [get internal stats](#get-internal-stats).
+You can check the detailed statistics of a topic. The following is an example. For the description of each stats, refer to [get internal stats](#get-internal-stats).
```json
-
{
"entriesAddedCounter": 20449518,
"numberOfEntries": 3233,
@@ -2241,7 +2074,6 @@ You can check the detailed statistics of a topic. The following is an example. F
}
}
}
-
```
You can get the internal stats for the partitioned topic in the following ways.
@@ -2253,10 +2085,8 @@ You can get the internal stats for the partitioned topic in the following ways.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin topics stats-internal \
- persistent://test-tenant/namespace/topic
-
+pulsar-admin topics stats-internal \
+persistent://test-tenant/namespace/topic
```
</TabItem>
@@ -2268,9 +2098,7 @@ $ pulsar-admin topics stats-internal \
<TabItem value="Java">
```java
-
admin.topics().getInternalStats(topic);
-
```
</TabItem>
@@ -2295,11 +2123,9 @@ You can create a subscription for a topic using one of the following methods.
<TabItem value="pulsar-admin">
```shell
-
pulsar-admin topics create-subscription \
--subscription my-subscription \
persistent://test-tenant/ns1/tp1
-
```
</TabItem>
@@ -2311,11 +2137,9 @@ persistent://test-tenant/ns1/tp1
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
String subscriptionName = "my-subscription";
admin.topics().createSubscription(topic, subscriptionName, MessageId.latest);
-
```
</TabItem>
@@ -2335,11 +2159,9 @@ You can check all subscription names for a given topic using one of the followin
<TabItem value="pulsar-admin">
```shell
-
pulsar-admin topics subscriptions \
persistent://test-tenant/ns1/tp1 \
my-subscription
-
```
</TabItem>
@@ -2351,10 +2173,8 @@ my-subscription
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
admin.topics().getSubscriptions(topic);
-
```
</TabItem>
@@ -2364,7 +2184,7 @@ admin.topics().getSubscriptions(topic);
### Unsubscribe subscription
-When a subscription does not process messages any more, you can unsubscribe it using one of the following methods.
+When a subscription does not process messages anymore, you can unsubscribe it using one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -2374,11 +2194,9 @@ When a subscription does not process messages any more, you can unsubscribe it u
<TabItem value="pulsar-admin">
```shell
-
pulsar-admin topics unsubscribe \
--subscription my-subscription \
persistent://test-tenant/ns1/tp1
-
```
</TabItem>
@@ -2390,11 +2208,9 @@ persistent://test-tenant/ns1/tp1
<TabItem value="Java">
```java
-
String topic = "persistent://my-tenant/my-namespace/my-topic";
String subscriptionName = "my-subscription";
admin.topics().deleteSubscription(topic, subscriptionName);
-
```
</TabItem>
diff --git a/site2/website-next/docs/admin-api-transactions.md b/site2/website-next/docs/admin-api-transactions.md
index 384e5b0fca8..0b04527afa2 100644
--- a/site2/website-next/docs/admin-api-transactions.md
+++ b/site2/website-next/docs/admin-api-transactions.md
@@ -35,10 +35,8 @@ In the production environment, there may be some long-lasting transactions that
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions slow-transactions \
- -c 1 -t 1s
-
+pulsar-admin transactions slow-transactions \
+-c 1 -t 1s
```
</TabItem>
@@ -50,11 +48,9 @@ $ pulsar-admin transactions slow-transactions \
<TabItem value="Java">
```java
-
admin.transaction().getSlowTransactionsByCoordinatorId(coordinatorId, timeout, timeUnit)
//Or get slow transactions from all coordinators
admin.transaction().getSlowTransactions(timeout, timeUnit)
-
```
</TabItem>
@@ -147,7 +143,6 @@ The following is an example of the returned values.
"ackedPartitions": {}
}
}
-
```
### ScaleTransactionCoordinators
@@ -161,10 +156,8 @@ When the performance of transactions reaches a bottleneck due to the insufficien
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions scale-transactionCoordinators \
- -r 17
-
+pulsar-admin transactions scale-transactionCoordinators \
+-r 17
```
</TabItem>
@@ -176,9 +169,7 @@ $ pulsar-admin transactions scale-transactionCoordinators \
<TabItem value="Java">
```java
-
admin.transaction().scaleTransactionCoordinators(replicas);
-
```
</TabItem>
@@ -207,10 +198,8 @@ Use one of the following ways to get your transaction metadata.
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions transaction-metadata\
- -m 1 -l 1
-
+pulsar-admin transactions transaction-metadata\
+-m 1 -l 1
```
</TabItem>
@@ -222,9 +211,7 @@ $ pulsar-admin transactions transaction-metadata\
<TabItem value="Java">
```java
-
admin.transactions().getTransactionMetadata(txnID);
-
```
</TabItem>
@@ -235,7 +222,6 @@ admin.transactions().getTransactionMetadata(txnID);
The following is an example of the returned values.
```shell
-
{
"txnId" : "(1,18)",
"status" : "ABORTING",
@@ -255,7 +241,6 @@ The following is an example of the returned values.
}
}
}
-
```
### Get transaction stats in transaction pending ack
@@ -272,10 +257,8 @@ Use one of the following ways to get transaction stats in pending ack:
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions transaction-in-pending-ack-stats \
- -m 1 -l 1 -t my-topic -s mysubname
-
+pulsar-admin transactions transaction-in-pending-ack-stats \
+-m 1 -l 1 -t my-topic -s mysubname
```
</TabItem>
@@ -287,9 +270,7 @@ $ pulsar-admin transactions transaction-in-pending-ack-stats \
<TabItem value="Java">
```java
-
admin.transaction().getTransactionInPendingAckStats(txnID, topic, subname);
-
```
</TabItem>
@@ -300,11 +281,9 @@ admin.transaction().getTransactionInPendingAckStats(txnID, topic, subname);
The following is an example of the returned value.
```shell
-
{
"cumulativeAckPosition" : "137:49959"
}
-
```
### Get transaction stats in transaction buffer
@@ -322,10 +301,8 @@ Use one of the following ways to get transaction stats in transaction buffer:
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions transaction-in-buffer-stats \
- -m 1 -l 1 -t my-topic
-
+pulsar-admin transactions transaction-in-buffer-stats \
+-m 1 -l 1 -t my-topic
```
</TabItem>
@@ -337,9 +314,7 @@ $ pulsar-admin transactions transaction-in-buffer-stats \
<TabItem value="Java">
```java
-
admin.transaction().getTransactionInBufferStatsAsync(txnID, topic);
-
```
</TabItem>
@@ -350,12 +325,10 @@ admin.transaction().getTransactionInBufferStatsAsync(txnID, topic);
The following is an example of the returned values.
```shell
-
{
"startPosition" : "137:49759",
"aborted" : false
}
-
```
## Transaction coordinator stats
@@ -380,10 +353,8 @@ Use one of the following ways to get transaction coordinator stats:
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions coordinator-stats \
- -c 1
-
+pulsar-admin transactions coordinator-stats \
+-c 1
```
</TabItem>
@@ -395,11 +366,9 @@ $ pulsar-admin transactions coordinator-stats \
<TabItem value="Java">
```java
-
admin.transaction().getCoordinatorStatsById(coordinatorId);
//Or get all coordinator stats.
admin.transaction().getCoordinatorStats();
-
```
</TabItem>
@@ -410,7 +379,6 @@ admin.transaction().getCoordinatorStats();
The following is an example of the returned values.
```shell
-
{
"state" : "Ready",
"leastSigBits" : 1,
@@ -419,7 +387,6 @@ The following is an example of the returned values.
"recoverStartTime" : 1657021892377,
"recoverEndTime" : 1657021892378
}
-
```
### Get coordinator internal stats
@@ -437,10 +404,8 @@ Use one of the following ways to get coordinator’s internal stats:
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions coordinator-internal-stats \
- -c 1 -m
-
+pulsar-admin transactions coordinator-internal-stats \
+-c 1 -m
```
</TabItem>
@@ -452,9 +417,7 @@ $ pulsar-admin transactions coordinator-internal-stats \
<TabItem value="Java">
```java
-
admin.transaction().getCoordinatorInternalStats(coordinatorId, metadata);
-
```
</TabItem>
@@ -465,7 +428,6 @@ admin.transaction().getCoordinatorInternalStats(coordinatorId, metadata);
The following is an example of the returned values.
```shell
-
{
"transactionLogStats" : {
"managedLedgerName" : "pulsar/system/persistent/__transaction_log_1",
@@ -510,7 +472,6 @@ The following is an example of the returned values.
}
}
}
-
```
## Transaction pending ack stats
@@ -535,10 +496,8 @@ Use one of the following ways to get transaction pending ack stats:
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions pending-ack-stats \
- -t my-topic -s mysubName -l
-
+pulsar-admin transactions pending-ack-stats \
+-t my-topic -s mysubName -l
```
</TabItem>
@@ -550,9 +509,7 @@ $ pulsar-admin transactions pending-ack-stats \
<TabItem value="Java">
```java
-
admin.transaction().getPendingAckStats(topic, subName, lowWaterMarks)
-
```
</TabItem>
@@ -563,7 +520,6 @@ admin.transaction().getPendingAckStats(topic, subName, lowWaterMarks)
The following is an example of the returned values.
```shell
-
{
"state" : "Ready",
"lowWaterMarks" : {
@@ -573,7 +529,6 @@ The following is an example of the returned values.
"recoverStartTime" : 1657021899202,
"recoverEndTime" : 1657021899203
}
-
```
### Get transaction pending ack internal stats
@@ -592,10 +547,8 @@ Use one of the following ways to get transaction pending ack internal stats:
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions pending-ack-internal-stats \
- -t my-topic -s mysubName -m
-
+pulsar-admin transactions pending-ack-internal-stats \
+-t my-topic -s mysubName -m
```
</TabItem>
@@ -607,9 +560,7 @@ $ pulsar-admin transactions pending-ack-internal-stats \
<TabItem value="Java">
```java
-
admin.transaction().getPendingAckInternalStats(topic, subName, boolean metadata);
-
```
</TabItem>
@@ -620,7 +571,6 @@ admin.transaction().getPendingAckInternalStats(topic, subName, boolean metadata)
The following is an example of the returned values.
```shell
-
{
"pendingAckLogStats" : {
"managedLedgerName" : "public/default/persistent/my-topic-mysubName__transaction_pending_ack",
@@ -672,7 +622,6 @@ The following is an example of the returned values.
}
}
}
-
```
### Get position stats in pending ack
@@ -682,7 +631,7 @@ The position stats in pending ack include:
* **MarkDelete** The position is already acknowledged.
* **NotInPendingAck** The position is not acknowledged within a transaction.
* **PendingAckNotReady** The pending ack has not been initialized.
-* **InvalidPosition** The position is an invalid position, for example, batch index > batch size.
+* **InvalidPosition** The position is invalid, for example, batch index > batch size.
If you want to know whether the position has been acknowledged, you can use one of the following ways to get position stats pending ack:
@@ -693,11 +642,8 @@ If you want to know whether the position has been acknowledged, you can use one
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions position-stats-in-pending-ack \
- -t my-topic -s mysubName -l 15 -e 6
-
-
+pulsar-admin transactions position-stats-in-pending-ack \
+-t my-topic -s mysubName -l 15 -e 6
```
</TabItem>
@@ -710,9 +656,7 @@ $ pulsar-admin transactions position-stats-in-pending-ack \
<TabItem value="Java">
```java
-
admin.transaction().getPositionStatsInPendingAckAsync(topic, subName, ledgerId, entryId, lowWaterMarks);
-
```
</TabItem>
@@ -723,11 +667,9 @@ admin.transaction().getPositionStatsInPendingAckAsync(topic, subName, ledgerId,
The following is an example of the returned values.
```shell
-
{
-“State” : “MarkDelete”
+"State" : "MarkDelete"
}
-
```
## Transaction buffer stats
@@ -755,10 +697,8 @@ Use one of the following ways to get transaction buffer stats:
<TabItem value="pulsar-admin">
```shell
-
-$ pulsar-admin transactions transaction-buffer-stats \
- -t my-topic -l
-
+pulsar-admin transactions transaction-buffer-stats \
+-t my-topic -l
```
</TabItem>
@@ -770,9 +710,7 @@ $ pulsar-admin transactions transaction-buffer-stats \
<TabItem value="Java">
```java
-
admin.transaction().getTransactionBufferStats(topic, lowWaterMarks);
-
```
</TabItem>
@@ -783,7 +721,6 @@ admin.transaction().getTransactionBufferStats(topic, lowWaterMarks);
The following is an example of the returned values.
```shell
-
{
"state" : "Ready",
"maxReadPosition" : "38:101",
@@ -796,5 +733,4 @@ The following is an example of the returned values.
"recoverStartTime" : 1657021892850,
"recoverEndTime" : 1657021893372
}
-
```
\ No newline at end of file
diff --git a/site2/website-next/docs/administration-dashboard.md b/site2/website-next/docs/administration-dashboard.md
index 18969a0f4df..c722097cbf3 100644
--- a/site2/website-next/docs/administration-dashboard.md
+++ b/site2/website-next/docs/administration-dashboard.md
@@ -21,40 +21,32 @@ You can use the [Django](https://www.djangoproject.com) web app to render the co
The easiest way to use the dashboard is to run it inside a [Docker](https://www.docker.com/products/docker) container.
```shell
-
-$ SERVICE_URL=http://broker.example.com:8080/
-$ docker run -p 80:80 \
- -e SERVICE_URL=$SERVICE_URL \
- apachepulsar/pulsar-dashboard:@pulsar:version@
-
+SERVICE_URL=http://broker.example.com:8080/
+docker run -p 80:80 \
+-e SERVICE_URL=$SERVICE_URL \
+apachepulsar/pulsar-dashboard:@pulsar:version@
```
You can find the {@inject: github:Dockerfile:/dashboard/Dockerfile} in the `dashboard` directory and build an image from scratch as well:
```shell
-
-$ docker build -t apachepulsar/pulsar-dashboard dashboard
-
+docker build -t apachepulsar/pulsar-dashboard dashboard
```
-If token authentication is enabled:
-> Provided token should have super-user access.
+If token authentication is enabled, the provided token should have super-user access.
```shell
-
-$ SERVICE_URL=http://broker.example.com:8080/
-$ JWT_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c
-$ docker run -p 80:80 \
- -e SERVICE_URL=$SERVICE_URL \
- -e JWT_TOKEN=$JWT_TOKEN \
- apachepulsar/pulsar-dashboard
-
+SERVICE_URL=http://broker.example.com:8080/
+JWT_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c
+docker run -p 80:80 \
+-e SERVICE_URL=$SERVICE_URL \
+-e JWT_TOKEN=$JWT_TOKEN \
+apachepulsar/pulsar-dashboard
```
-
You need to specify only one service URL for a Pulsar cluster. Internally, the collector figures out all the existing clusters and the brokers from where it needs to pull the metrics. If you connect the dashboard to Pulsar running in standalone mode, the URL is `http://<broker-ip>:8080` by default. `<broker-ip>` is the IP address or hostname of the machine that runs Pulsar standalone. The IP address or hostname should be accessible from the running dashboard in the docker instance.
-Once the Docker container starts, the web dashboard is accessible via `localhost` or whichever host that Docker uses.
+Once the Docker container starts, the web dashboard is accessible via `localhost` or whichever host that the Docker uses.
> The `SERVICE_URL` that the dashboard uses needs to be reachable from inside the Docker container.
@@ -65,11 +57,11 @@ Similarly, given the Pulsar standalone advertises itself with localhost by defau
explicitly set the advertise address to the host IP address. For example:
```shell
-
-$ bin/pulsar standalone --advertised-address 1.2.3.4
-
+bin/pulsar standalone --advertised-address 1.2.3.4
```
-### Known issues
+:::note
Currently, only Pulsar Token [authentication](security-overview.md#authentication-providers) is supported.
+
+:::
\ No newline at end of file
diff --git a/site2/website-next/docs/administration-geo.md b/site2/website-next/docs/administration-geo.md
index 778b5f61137..e6ce7c70350 100644
--- a/site2/website-next/docs/administration-geo.md
+++ b/site2/website-next/docs/administration-geo.md
@@ -31,9 +31,9 @@ In normal cases, when connectivity issues are none, messages are replicated imme
Applications can create producers and consumers in any of the clusters, even when the remote clusters are not reachable (like during a network partition).
-Producers and consumers can publish messages to and consume messages from any cluster in a Pulsar instance. However, subscriptions cannot only be local to the cluster where the subscriptions are created but also can be transferred between clusters after replicated subscription is enabled. Once replicated subscription is enabled, you can keep subscription state in synchronization. Therefore, a topic can be asynchronously replicated across multiple geographical regions. In case of failover [...]
+Producers and consumers can publish messages to and consume messages from any cluster in a Pulsar instance. However, subscriptions cannot only be local to the cluster where the subscriptions are created but also can be transferred between clusters after the replicated subscription is enabled. Once the replicated subscription is enabled, you can keep the subscription state in synchronization. Therefore, a topic can be asynchronously replicated across multiple geographical regions. In case [...]
-
+
In the aforementioned example, the **T1** topic is replicated among three clusters, **Cluster-A**, **Cluster-B**, and **Cluster-C**.
@@ -60,12 +60,10 @@ Suppose that you have 3 replication clusters: `us-west`, `us-cent`, and `us-east
Run the following command on `us-west`.
```shell
-
-$ bin/pulsar-admin clusters create \
- --broker-url pulsar://<DNS-OF-US-EAST>:<PORT> \
- --url http://<DNS-OF-US-EAST>:<PORT> \
- us-east
-
+bin/pulsar-admin clusters create \
+--broker-url pulsar://<DNS-OF-US-EAST>:<PORT> \
+--url http://<DNS-OF-US-EAST>:<PORT> \
+us-east
```
:::tip
@@ -79,29 +77,25 @@ $ bin/pulsar-admin clusters create \
Run the following command on `us-west`.
-```shell
-
-$ bin/pulsar-admin clusters create \
- --broker-url pulsar://<DNS-OF-US-CENT>:<PORT> \
- --url http://<DNS-OF-US-CENT>:<PORT> \
- us-cent
-
-```
+ ```shell
+ bin/pulsar-admin clusters create \
+ --broker-url pulsar://<DNS-OF-US-CENT>:<PORT> \
+ --url http://<DNS-OF-US-CENT>:<PORT> \
+ us-cent
+ ```
3. Run similar commands on `us-east` and `us-cent` to create connections among clusters.
### Grant permissions to properties
-To replicate to a cluster, the tenant needs permission to use that cluster. You can grant permission to the tenant when you create the tenant or grant later.
+To replicate to a cluster, the tenant needs permission to use that cluster. You can grant permission to the tenant when you create the tenant or grant it later.
Specify all the intended clusters when you create a tenant:
```shell
-
-$ bin/pulsar-admin tenants create my-tenant \
- --admin-roles my-admin-role \
- --allowed-clusters us-west,us-east,us-cent
-
+bin/pulsar-admin tenants create my-tenant \
+--admin-roles my-admin-role \
+--allowed-clusters us-west,us-east,us-cent
```
To update permissions of an existing tenant, use `update` instead of `create`.
@@ -115,18 +109,14 @@ You can enable geo-replication at **namespace** or **topic** level.
You can create a namespace with the following command sample.
```shell
-
-$ bin/pulsar-admin namespaces create my-tenant/my-namespace
-
+bin/pulsar-admin namespaces create my-tenant/my-namespace
```
Initially, the namespace is not assigned to any cluster. You can assign the namespace to clusters using the `set-clusters` subcommand:
```shell
-
-$ bin/pulsar-admin namespaces set-clusters my-tenant/my-namespace \
- --clusters us-west,us-east,us-cent
-
+bin/pulsar-admin namespaces set-clusters my-tenant/my-namespace \
+--clusters us-west,us-east,us-cent
```
#### Enable geo-replication at topic level
@@ -134,9 +124,7 @@ $ bin/pulsar-admin namespaces set-clusters my-tenant/my-namespace \
You can set geo-replication at topic level using the command `pulsar-admin topics set-replication-clusters`. For the latest and complete information about `Pulsar admin`, including commands, flags, descriptions, and more information, see [Pulsar admin doc](/tools/pulsar-admin/).
```shell
-
-$ bin/pulsar-admin topics set-replication-clusters --clusters us-west,us-east,us-cent my-tenant/my-namespace/my-topic
-
+bin/pulsar-admin topics set-replication-clusters --clusters us-west,us-east,us-cent my-tenant/my-namespace/my-topic
```
:::tip
@@ -144,10 +132,12 @@ $ bin/pulsar-admin topics set-replication-clusters --clusters us-west,us-east,us
- You can change the replication clusters for a namespace at any time, without disruption to ongoing traffic. Replication channels are immediately set up or stopped in all clusters as soon as the configuration changes.
- Once you create a geo-replication namespace, any topics that producers or consumers create within that namespace are replicated across clusters. Typically, each application uses the `serviceUrl` for the local cluster.
- If you are using Pulsar version `2.10.x`, to enable geo-replication at topic level, you need to change the following configurations in the `conf/broker.conf` or `conf/standalone.conf` file to enable topic policies service.
-```shell
+
+```conf
systemTopicEnabled=true
topicLevelPoliciesEnabled=true
```
+
:::
### Use topics with geo-replication
@@ -156,10 +146,9 @@ topicLevelPoliciesEnabled=true
By default, messages are replicated to all clusters configured for the namespace. You can restrict replication selectively by specifying a replication list for a message, and then that message is replicated only to the subset in the replication list.
-The following is an example for the [Java API](client-libraries-java.md). Note the use of the `setReplicationClusters` method when you construct the {@inject: javadoc:Message:/client/org/apache/pulsar/client/api/Message} object:
+The following is an example of the [Java](client-libraries-java.md) API](client-libraries-java.md). Note the use of the `setReplicationClusters` method when you construct the {@inject: javadoc:Message:/client/org/apache/pulsar/client/api/Message} object:
```java
-
List<String> restrictReplicationTo = Arrays.asList(
"us-west",
"us-east"
@@ -173,7 +162,6 @@ producer.newMessage()
.value("my-payload".getBytes())
.setReplicationClusters(restrictReplicationTo)
.send();
-
```
#### Topic stats
@@ -189,9 +177,7 @@ You can check topic-specific statistics for geo-replication topics using one of
Use the [`pulsar-admin topics stats`](/tools/pulsar-admin/) command.
```shell
-
-$ bin/pulsar-admin topics stats persistent://my-tenant/my-namespace/my-topic
-
+bin/pulsar-admin topics stats persistent://my-tenant/my-namespace/my-topic
```
</TabItem>
@@ -222,22 +208,20 @@ To delete a geo-replication topic, close all producers and consumers on the topi
## Replicated subscriptions
-Pulsar supports replicated subscriptions, so you can keep subscription state in sync, within a sub-second timeframe, in the context of a topic that is being asynchronously replicated across multiple geographical regions.
+Pulsar supports replicated subscriptions, so you can keep the subscription state in sync, within a sub-second timeframe, in the context of a topic that is being asynchronously replicated across multiple geographical regions.
In case of failover, a consumer can restart consuming from the failure point in a different cluster.
### Enable replicated subscription
-Replicated subscription is disabled by default. You can enable replicated subscription when creating a consumer.
+Replicated subscription is disabled by default. You can enable replicated subscriptions when creating a consumer.
```java
-
Consumer<String> consumer = client.newConsumer(Schema.STRING)
.topic("my-topic")
.subscriptionName("my-subscription")
.replicateSubscriptionState(true)
.subscribe();
-
```
### Advantages
@@ -249,7 +233,7 @@ Consumer<String> consumer = client.newConsumer(Schema.STRING)
### Limitations
-* When you enable replicated subscription, you're creating a consistent distributed snapshot to establish an association between message ids from different clusters. The snapshots are taken periodically. The default value is `1 second`. It means that a consumer failing over to a different cluster can potentially receive 1 second of duplicates. You can also configure the frequency of the snapshot in the `broker.conf` file.
+* When you enable replicated subscriptions, you're creating a consistent distributed snapshot to establish an association between message ids from different clusters. The snapshots are taken periodically. The default value is `1 second`. It means that a consumer failing over to a different cluster can potentially receive 1 second of duplicates. You can also configure the frequency of the snapshot in the `broker.conf` file.
* Only the base line cursor position is synced in replicated subscriptions while the individual acknowledgments are not synced. This means the messages acknowledged out-of-order could end up getting delivered again, in the case of a cluster failover.
## Migrate data between clusters using geo-replication
@@ -259,36 +243,27 @@ Using geo-replication to migrate data between clusters is a special use case of
1. Create your new cluster.
2. Add the new cluster to your old cluster.
-```shell
-
- bin/pulsar-admin cluster create new-cluster
-
-```
+ ```shell
+ bin/pulsar-admin cluster create new-cluster
+ ```
3. Add the new cluster to your tenant.
-```shell
-
- bin/pulsar-admin tenants update my-tenant --cluster old-cluster,new-cluster
-
-```
+ ```shell
+ bin/pulsar-admin tenants update my-tenant --cluster old-cluster,new-cluster
+ ```
4. Set the clusters on your namespace.
-```shell
-
- bin/pulsar-admin namespaces set-clusters my-tenant/my-ns --cluster old-cluster,new-cluster
-
-```
+ ```shell
+ bin/pulsar-admin namespaces set-clusters my-tenant/my-ns --cluster old-cluster,new-cluster
+ ```
5. Update your applications using [replicated subscriptions](#replicated-subscriptions).
6. Validate subscription replication is active.
-
-```shell
-
- bin/pulsar-admin topics stats-internal public/default/t1
-
-```
+ ```shell
+ bin/pulsar-admin topics stats-internal public/default/t1
+ ```
7. Move your consumers and producers to the new cluster by modifying the values of `serviceURL`.
diff --git a/site2/website-next/docs/administration-isolation-bookie.md b/site2/website-next/docs/administration-isolation-bookie.md
index 579ea95e68e..4ebbe576921 100644
--- a/site2/website-next/docs/administration-isolation-bookie.md
+++ b/site2/website-next/docs/administration-isolation-bookie.md
@@ -26,7 +26,7 @@ To isolate bookies, you need to complete the following tasks.
## Understand bookie data isolation policy
-Bookie data isolation policy is built on top of the existing BookKeeper rack-aware placement policy. The “rack” concept can be anything, for example, racks, regions, availability zones. It writes the configured isolation policy into the metadata store. Both BookKeeper clients on the broker and bookie auto-recovery side read the configured isolation policy from the metadata store and apply it when choosing bookies to store messages.
+Bookie data isolation policy is built on top of the existing BookKeeper rack-aware placement policy. The "rack" concept can be anything, for example, racks, regions, availability zones. It writes the configured isolation policy into the metadata store. Both BookKeeper clients on the broker and bookie auto-recovery side read the configured isolation policy from the metadata store and apply it when choosing bookies to store messages.
BookKeeper provides three kinds of data isolation policies for disaster tolerance.
* Rack-aware placement policy (default)
@@ -36,7 +36,7 @@ BookKeeper provides three kinds of data isolation policies for disaster toleranc
:::tip
* Both [rack-aware placement policy](#rack-aware-placement-policy) and [region-aware placement policy](#region-aware-placement-policy) can be used in all kinds of deployments where racks are a subset of a region. The major difference between the two policies is:
- * With `RackawareEnsemblePlacementPolicy` configured, the BookKeeper client chooses bookies from different **racks** to reduce the single-point-of-failure. If there is only one rack available, the policy falls back on choosing a random bookie across available ones.
+ * With `RackawareEnsemblePlacementPolicy` configured, the BookKeeper client chooses bookies from different **racks** to reduce the single point of failure. If there is only one rack available, the policy falls back on choosing a random bookie across available ones.
* With `RegionAwareEnsemblePlacementPolicy` configured, the BookKeeper client chooses bookies from different **regions**; for the selected region, it chooses bookies from different racks if more than one ensemble falls into the same region.
* Zone-aware placement policy (`ZoneAwareEnsemblePlacementPolicy`) can be used in a public cloud infrastructure where Availability Zones (AZs) are isolated locations within the data center regions that public cloud services originate from and operate in.
@@ -70,13 +70,13 @@ For example, you have the same BookKeeper cluster with the same topic requiremen
-* If you have configured `EnforceMinNumRacksPerWriteQuorum=false`, the BookKeeper client tries its best-effort to apply the placement policy depending on the available number of racks and bookies. It may still work as the above diagram or the following diagram.
+* If you have configured `EnforceMinNumRacksPerWriteQuorum=false`, the BookKeeper client tries its best effort to apply the placement policy depending on the available number of racks and bookies. It may still work as the above diagram or the following diagram.
### Region-aware placement policy
-Region-aware placement policy enforces different data replicas to be placed in different regions and racks to guarantee the region-level disaster tolerance. To achieve datacenter level disaster tolerance, you need to write data replicas into different data centers. You can use `RegionAwareEnsemblePlacementPolicy` to configure region and rack information for each bookie node to ensure region-level disaster tolerance.
+Region-aware placement policy enforces different data replicas to be placed in different regions and racks to guarantee region-level disaster tolerance. To achieve datacenter level disaster tolerance, you need to write data replicas into different data centers. You can use `RegionAwareEnsemblePlacementPolicy` to configure region and rack information for each bookie node to ensure region-level disaster tolerance.
For example, the BookKeeper cluster has 4 regions, and each region has several racks with their bookie instances, as shown in the following diagram. If a topic is configured with `EnsembleSize=3, WriteQuorum=3, and AckQuorum=2`, the BookKeeper client chooses three different regions, such as Region A, Region C and Region D. For each region, it chooses one bookie on a single rack, such as Bookie5 on Rack2, Bookie17 on Rack6, and Bookie21 on Rack8.
@@ -164,7 +164,6 @@ Usage: set-bookie-rack [options]
Bookie rack name
```
-
:::tip
In addition, you can also group bookies across racks or regions to serve broker-level isolation by specifying a group name for each bookie and assigning the group name to a specific namespace. See [configure bookie affinity groups](#configure-bookie-affinity-groups) for more details.
@@ -226,10 +225,8 @@ To configure bookie affinity groups, you can use one of the following methods.
<TabItem value="Pulsar-admin CLI">
-```
-
+```shell
pulsar-admin namespaces set-bookie-affinity-group options
-
```
For more information about the command `pulsar-admin namespaces set-bookie-affinity-group options`, see [Pulsar admin docs](/tools/pulsar-admin/).
@@ -237,7 +234,6 @@ For more information about the command `pulsar-admin namespaces set-bookie-affin
**Example**
```shell
-
bin/pulsar-admin bookies set-bookie-rack \
--bookie 127.0.0.1:3181 \
--hostname 127.0.0.1:3181 \
@@ -246,7 +242,6 @@ bin/pulsar-admin bookies set-bookie-rack \
bin/pulsar-admin namespaces set-bookie-affinity-group public/default \
--primary-group group-bookie1
-
```
:::note
diff --git a/site2/website-next/docs/administration-isolation-broker.md b/site2/website-next/docs/administration-isolation-broker.md
index 0b1f0c11d13..b23e2f9754a 100644
--- a/site2/website-next/docs/administration-isolation-broker.md
+++ b/site2/website-next/docs/administration-isolation-broker.md
@@ -21,7 +21,7 @@ To set a namespace isolation policy for a broker cluster, you can use one of the
<TabItem value="Pulsar-admin CLI">
-```
+```shell
pulsar-admin ns-isolation-policy set options
```
@@ -40,7 +40,7 @@ bin/pulsar-admin ns-isolation-policy set \
</TabItem>
<TabItem value="REST API">
-[PUT /admin/v2/namespaces/{tenant}/{namespace}](https://pulsar.apache.org/admin-rest-api/?version=master&apiversion=v2#operation/createNamespace)
+{@inject: endpoint|PUT|/admin/v2/:namespace/:tenant/:namespace|operation/createNamespace?version=@pulsar:version_number@}
</TabItem>
<TabItem value="Java admin API">
diff --git a/site2/website-next/docs/administration-isolation.md b/site2/website-next/docs/administration-isolation.md
index c35f6cd5cbf..cb4ecd19986 100644
--- a/site2/website-next/docs/administration-isolation.md
+++ b/site2/website-next/docs/administration-isolation.md
@@ -5,7 +5,7 @@ sidebar_label: "Pulsar isolation"
---
-In an organization, a Pulsar instance provides services to multiple teams. When organizing the resources across multiple teams, you want to make a suitable isolation plan to avoid the resource competition between different teams and applications and provide high-quality messaging service. In this case, you need to take resource isolation into consideration and weigh your intended actions against expected and unexpected consequences.
+In an organization, a Pulsar instance provides services to multiple teams. When organizing the resources across multiple teams, you want to make a suitable isolation plan to avoid resource competition between different teams and applications and provide high-quality messaging service. In this case, you need to consider resource isolation and weigh your intended actions against expected and unexpected consequences.
The multi-layer and segment-centric architecture and hierarchical resource management of Pulsar provide a solid foundation for isolation, which allows you to isolate resources in your desired manner, prevent resource competition, and attain stability.
@@ -13,7 +13,7 @@ The multi-layer and segment-centric architecture and hierarchical resource manag
## Isolation levels
Pulsar supports isolation at either of the following two levels or both.
-* [Broker-level isolation](administration-isolation-broker.md) divides brokers into different groups and assigns broker groups to different namespaces. In this way, you can bind topics in a namespace to a set of brokers that belong to the specific groups.
+* [Broker-level isolation](administration-isolation-broker.md) divides brokers into different groups and assigns broker groups to different namespaces. In this way, you can bind topics in a namespace to a set of brokers that belong to specific groups.
* [Bookie-level isolation](administration-isolation-bookie.md) divides bookies into different racks/regions and assigns data replicas to bookies based on a specified data placement policy for disaster tolerance.
@@ -57,7 +57,7 @@ Here are some key points for understanding how it works:
- Each Pulsar cluster has one or multiple brokers.
- Each Pulsar cluster has one metadata store.
-As illustrated below, all bookie groups use a shared BookKeeper cluster and a metadata store, and each [bookie affinity group](administration-isolation-bookie.md#configure-bookie-affinity-groups) has one or several bookies. You can specify one or multiple primary/secondary groups for a namespace. Topics under the namespace are created on the bookies in the primary group firstly and then created on the bookies in the secondary group.
+As illustrated below, all bookie groups use a shared BookKeeper cluster and a metadata store, and each [bookie affinity group](administration-isolation-bookie.md#configure-bookie-affinity-groups) has one or several bookies. You can specify one or multiple primary/secondary groups for a namespace. Topics under the namespace are created on the bookies in the primary group first and then created on the bookies in the secondary group.
@@ -69,5 +69,5 @@ The following illustration demonstrates how to achieve isolation inside a single
Here are some key points for understanding how it works:
- Each cluster exposes its service through a DNS entry point and makes sure a client can access the cluster through the DNS entry point. Clients can use one or multiple Pulsar URLs that the Pulsar cluster exposes as the service URL.
-- Broker isolation is achieved by setting [namespace isolation policy](administration-isolation-broker.md).
+- Broker isolation is achieved by setting a [namespace isolation policy](administration-isolation-broker.md).
diff --git a/site2/website-next/docs/administration-load-balance.md b/site2/website-next/docs/administration-load-balance.md
index 49598193fff..c3d887335bc 100644
--- a/site2/website-next/docs/administration-load-balance.md
+++ b/site2/website-next/docs/administration-load-balance.md
@@ -9,11 +9,11 @@ Pulsar is a horizontally scalable messaging system, so the traffic in a logical
You can use multiple settings and tools to control the traffic distribution which requires a bit of context to understand how the traffic is managed in Pulsar. Though in most cases, the core requirement mentioned above is true out of the box and you should not worry about it.
-The following sections introduce how the load-balanced assignments work across Pulsar brokers and how you can leverage the framework to adjust.
+The following sections introduce how load-balanced assignments work across Pulsar brokers and how you can leverage the framework to adjust.
## Dynamic assignments
-Topics are dynamically assigned to brokers based on the load conditions of all brokers in the cluster. The assignment of topics to brokers is not done at the topic level but at the **bundle** level (a higher level). Instead of individual topic assignments, each broker takes ownership of a subset of the topics for a namespace. This subset is called a bundle and effectively this subset is a sharding mechanism.
+Topics are dynamically assigned to brokers based on the load conditions of all brokers in the cluster. The assignment of topics to brokers is not done at the topic level but the **bundle** level (a higher level). Instead of individual topic assignments, each broker takes ownership of a subset of the topics for a namespace. This subset is called a bundle and effectively this subset is a sharding mechanism.
In other words, each namespace is an "administrative" unit and sharded into a list of bundles, with each bundle comprising a portion of the overall hash range of the namespace. Topics are assigned to a particular bundle by taking the hash of the topic name and checking in which bundle the hash falls. Each bundle is independent of the others and thus is independently assigned to different brokers.
@@ -32,20 +32,16 @@ For partitioned topics, different partitions are assigned to different brokers.
When you create a new namespace, a number of bundles are assigned to the namespace. You can set this number in the `conf/broker.conf` file:
-```properties
-
+```conf
# When a namespace is created without specifying the number of bundles, this
# value will be used as the default
defaultNumberOfNamespaceBundles=4
-
```
Alternatively, you can override the value when you create a new namespace using [Pulsar admin](/tools/pulsar-admin/):
```shell
-
bin/pulsar-admin namespaces create my-tenant/my-namespace --clusters us-west --bundles 16
-
```
With the above command, you create a namespace with 16 initial bundles. Therefore the topics for this namespace can immediately be spread across up to 16 brokers.
@@ -65,18 +61,15 @@ Pulsar supports the following two bundle split algorithms:
To enable bundle split, you need to configure the following settings in the `broker.conf` file, and set `defaultNamespaceBundleSplitAlgorithm` based on your needs.
-```properties
-
+```conf
loadBalancerAutoBundleSplitEnabled=true
loadBalancerAutoUnloadSplitBundlesEnabled=true
defaultNamespaceBundleSplitAlgorithm=range_equally_divide
-
```
You can configure more parameters for splitting thresholds. Any existing bundle that exceeds any of the thresholds is a candidate to be split. By default, the newly split bundles are immediately reassigned to other brokers, to facilitate the traffic distribution.
-```properties
-
+```conf
# maximum topics in a bundle, otherwise bundle split will be triggered
loadBalancerNamespaceBundleMaxTopics=1000
@@ -91,7 +84,6 @@ loadBalancerNamespaceBundleMaxBandwidthMbytes=100
# maximum number of bundles in a namespace (for auto-split)
loadBalancerNamespaceMaximumBundles=128
-
```
## Shed load automatically
@@ -111,15 +103,13 @@ For example, the default threshold is 85% and if a broker is over quota at 95% C
Additional settings that apply to shedding:
-```properties
-
+```conf
# Load shedding interval. Broker periodically checks whether some traffic should be offload from
# some over-loaded broker to other under-loaded brokers
loadBalancerSheddingIntervalMinutes=1
# Prevent the same topics to be shed and moved to other brokers more than once within this timeframe
loadBalancerSheddingGracePeriodMinutes=30
-
```
Pulsar supports the following types of automatic load shedding strategies.
@@ -135,7 +125,8 @@ Pulsar supports the following types of automatic load shedding strategies.
:::
### ThresholdShedder
-This strategy tends to shed the bundles if any broker's usage is above the configured threshold. It does this by first computing the average resource usage per broker for the whole cluster. The resource usage for each broker is calculated using the following method `LocalBrokerData#getMaxResourceUsageWithWeight`. Historical observations are included in the running average based on the broker's setting for `loadBalancerHistoryResourcePercentage`. Once the average resource usage is calcula [...]
+
+This strategy tends to shed the bundles if any broker's usage is above the configured threshold. It does this by first computing the average resource usage per broker for the whole cluster. The resource usage for each broker is calculated using the following method `LocalBrokerData#getMaxResourceUsageWithWeight`. Historical observations are included in the running average based on the broker's setting for `loadBalancerHistoryResourcePercentage`. Once the average resource usage is calcula [...]
@@ -146,8 +137,7 @@ To use the `ThresholdShedder` strategy, configure brokers with this value.
You can configure the weights for each resource per broker in the `conf/broker.conf` file.
-```properties
-
+```conf
# The BandWithIn usage weight when calculating new resource usage. The range is between 0 and 1.0.
loadBalancerBandwithInResourceWeight=1.0
@@ -162,7 +152,6 @@ loadBalancerMemoryResourceWeight=1.0
# The direct memory usage weight when calculating new resource usage. The range is between 0 and 1.0.
loadBalancerDirectMemoryResourceWeight=1.0
-
```
### OverloadShedder
@@ -208,17 +197,13 @@ Unloading is the mechanism that the load manager uses to perform the load sheddi
Unloading a topic has no effect on the assignment, but just closes and reopens the particular topic:
```shell
-
pulsar-admin topics unload persistent://tenant/namespace/topic
-
```
To unload all topics for a namespace and trigger reassignments:
```shell
-
pulsar-admin namespaces unload tenant/namespace
-
```
## Distribute anti-affinity namespaces across failure domains
@@ -259,9 +244,7 @@ One broker can only be registered to a single failure domain.
To create a domain under a specific cluster and register brokers, run the following command:
```bash
-
pulsar-admin clusters create-failure-domain <cluster-name> --domain-name <domain-name> --broker-list <broker-list-comma-separated>
-
```
You can also view, update, and delete domains under a specific cluster. For more information, refer to [Pulsar admin doc](/tools/pulsar-admin/).
@@ -271,9 +254,7 @@ You can also view, update, and delete domains under a specific cluster. For more
An anti-affinity group is created automatically when the first namespace is assigned to the group. To assign a namespace to an anti-affinity group, run the following command. It sets an anti-affinity group name for a namespace.
```bash
-
pulsar-admin namespaces set-anti-affinity-group <namespace> --group <group-name>
-
```
For more information about `anti-affinity-group` related commands, refer to [Pulsar admin doc](/tools/pulsar-admin/).
diff --git a/site2/website-next/docs/administration-metadata-store.md b/site2/website-next/docs/administration-metadata-store.md
index 9734e02682f..eef1603854e 100644
--- a/site2/website-next/docs/administration-metadata-store.md
+++ b/site2/website-next/docs/administration-metadata-store.md
@@ -34,23 +34,19 @@ Pulsar metadata store can be deployed on a separate ZooKeeper cluster or deploye
To use ZooKeeper as the metadata store, add the following parameters to the `conf/broker.conf` or `conf/standalone.conf` file.
-```properties
-
+```conf
metadataStoreUrl=zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181
configurationMetadataStoreUrl=zk:my-global-zk-1:2181,my-global-zk-2:2181,my-global-zk-3:2181
-
```
## Use etcd as metadata store
To use etcd as the metadata store, add the following parameters to the `conf/broker.conf` or `conf/standalone.conf` file.
-```properties
-
+```conf
metadataStoreUrl=etcd:http://my-etcd-1:2379,http://my-etcd-2:2379,http://my-etcd-3:2379
configurationMetadataStoreUrl=etcd:my-global-etcd-1:2379,my-global-etcd-2:2379,my-global-etcd-3:2379
# metadataStoreConfigPath=/path/to/file
-
```
:::tip
@@ -72,11 +68,9 @@ authority=
To use RocksDB as the metadata store, add the following parameters to the `conf/broker.conf` or `conf/standalone.conf` file.
-```properties
-
+```conf
metadataStoreUrl=rocksdb://data/metadata
# metadataStoreConfigPath=/path/to/file
-
```
:::tip
@@ -89,10 +83,8 @@ The `metadataStoreConfigPath` parameter is required when you want to use advance
To use local memory as the metadata store, add the following parameters to the `conf/broker.conf` or `conf/standalone.conf` file.
-```properties
-
+```conf
metadataStoreUrl=memory://local
-
```
@@ -102,8 +94,7 @@ Pulsar metadata store supports batch operations and caching to meet low latency
To enable batch operations on the metadata store, you can configure the following parameters in the `conf/broker.conf` or `conf/standalone.conf` file.
-```properties
-
+```conf
# Whether we should enable metadata operations batching
metadataStoreBatchingEnabled=true
@@ -115,6 +106,5 @@ metadataStoreBatchingMaxOperations=1000
# Maximum size of a batch
metadataStoreBatchingMaxSizeKb=128
-
```
diff --git a/site2/website-next/docs/administration-proxy.md b/site2/website-next/docs/administration-proxy.md
index 5228b9a65dd..9f0915d7cdd 100644
--- a/site2/website-next/docs/administration-proxy.md
+++ b/site2/website-next/docs/administration-proxy.md
@@ -8,7 +8,7 @@ Pulsar proxy is an optional gateway. Pulsar proxy is used when direct connection
## Configure the proxy
-Before using the proxy, you need to configure it with the brokers addresses in the cluster. You can configure the broker URL in the proxy configuration, or the proxy to connect directly using service discovery.
+Before using a proxy, you need to configure it with a broker's address in the cluster. You can configure the broker URL in the proxy configuration, or the proxy to connect directly using service discovery.
> In a production environment service discovery is not recommended.
@@ -21,24 +21,20 @@ Proxy authorization requires access to ZooKeeper, so if you use these broker URL
You can configure the broker URLs in `conf/proxy.conf` as follows.
```properties
-
brokerServiceURL=pulsar://brokers.example.com:6650
brokerWebServiceURL=http://brokers.example.com:8080
functionWorkerWebServiceURL=http://function-workers.example.com:8080
-
```
If you use TLS, configure the broker URLs in the following way:
```properties
-
brokerServiceURLTLS=pulsar+ssl://brokers.example.com:6651
brokerWebServiceURLTLS=https://brokers.example.com:8443
functionWorkerWebServiceURL=https://function-workers.example.com:8443
-
```
-The hostname in the URLs provided should be a DNS entry which points to multiple brokers or a virtual IP address, which is backed by multiple broker IP addresses, so that the proxy does not lose connectivity to Pulsar cluster if a single broker becomes unavailable.
+The hostname in the URLs provided should be a DNS entry that points to multiple brokers or a virtual IP address, which is backed by multiple broker IP addresses, so that the proxy does not lose connectivity to Pulsar cluster if a single broker becomes unavailable.
The ports to connect to the brokers (6650 and 8080, or in the case of TLS, 6651 and 8443) should be open in the network ACLs.
@@ -49,10 +45,8 @@ Note that if you do not use functions, you do not need to configure `functionWor
Pulsar uses [ZooKeeper](https://zookeeper.apache.org) for service discovery. To connect the proxy to ZooKeeper, specify the following in `conf/proxy.conf`.
```properties
-
metadataStoreUrl=my-zk-0:2181,my-zk-1:2181,my-zk-2:2181
configurationMetadataStoreUrl=my-zk-0:2184,my-zk-remote:2184
-
```
> To use service discovery, you need to open the network ACLs, so the proxy can connects to the ZooKeeper nodes through the ZooKeeper client port (port `2181`) and the configuration store client port (port `2184`).
@@ -64,12 +58,10 @@ configurationMetadataStoreUrl=my-zk-0:2184,my-zk-remote:2184
To start the proxy:
```bash
-
-$ cd /path/to/pulsar/directory
-$ bin/pulsar proxy \
- --metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181 \
- --configuration-metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181
-
+cd /path/to/pulsar/directory
+bin/pulsar proxy \
+--metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181 \
+--configuration-metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181
```
> You can run multiple instances of the Pulsar proxy in a cluster.
diff --git a/site2/website-next/docs/administration-pulsar-manager.md b/site2/website-next/docs/administration-pulsar-manager.md
index a3c3498db2f..bc717f8afdf 100644
--- a/site2/website-next/docs/administration-pulsar-manager.md
+++ b/site2/website-next/docs/administration-pulsar-manager.md
@@ -18,13 +18,11 @@ If you are monitoring your current stats with Pulsar dashboard, we recommend you
The easiest way to use the Pulsar Manager is to run it inside a [Docker](https://www.docker.com/products/docker) container.
```shell
-
docker pull apachepulsar/pulsar-manager:v0.2.0
docker run -it \
- -p 9527:9527 -p 7750:7750 \
- -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
- apachepulsar/pulsar-manager:v0.2.0
-
+ -p 9527:9527 -p 7750:7750 \
+ -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
+ apachepulsar/pulsar-manager:v0.2.0
```
* Pulsar Manager is divided into front-end and back-end, the front-end service port is `9527` and the back-end service port is `7750`.
@@ -41,18 +39,15 @@ The following is an example of PostgreSQL.
2. Download and modify the [configuration file](https://github.com/apache/pulsar-manager/blob/master/src/main/resources/application.properties), then add the PostgreSQL configuration.
```properties
-
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.url=jdbc:postgresql://127.0.0.1:5432/pulsar_manager
spring.datasource.username=postgres
spring.datasource.password=postgres
-
```
3. Add a configuration mount and start with a docker image.
```bash
-
docker pull apachepulsar/pulsar-manager:v0.2.0
docker run -it \
-p 9527:9527 -p 7750:7750 \
@@ -60,7 +55,6 @@ docker run -it \
manager/application.properties
-e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
apachepulsar/pulsar-manager:v0.2.0
-
```
#### Enable JWT authentication (optional)
@@ -68,7 +62,6 @@ manager/application.properties
If you want to turn on JWT authentication, configure the `application.properties` file.
```properties
-
backend.jwt.token=token
jwt.broker.token.mode=PRIVATE
@@ -78,35 +71,31 @@ jwt.broker.private.key=file:///path/broker-private.key
or
jwt.broker.token.mode=SECRET
jwt.broker.secret.key=file:///path/broker-secret.key
-
```
-• `backend.jwt.token`: token for the superuser. You need to configure this parameter during cluster initialization.
-• `jwt.broker.token.mode`: multiple modes of generating token, including PUBLIC, PRIVATE, and SECRET.
-• `jwt.broker.public.key`: configure this option if you use the PUBLIC mode.
-• `jwt.broker.private.key`: configure this option if you use the PRIVATE mode.
-• `jwt.broker.secret.key`: configure this option if you use the SECRET mode.
+* `backend.jwt.token`: token for the superuser. You need to configure this parameter during cluster initialization.
+* `jwt.broker.token.mode`: multiple modes of generating tokens, including PUBLIC, PRIVATE, and SECRET.
+* `jwt.broker.public.key`: configure this option if you use the PUBLIC mode.
+* `jwt.broker.private.key`: configure this option if you use the PRIVATE mode.
+* `jwt.broker.secret.key`: configure this option if you use the SECRET mode.
For more information, see [Token Authentication Admin of Pulsar](security-token-admin.md).
Docker command to add profile and key files mount.
```bash
-
docker pull apachepulsar/pulsar-manager:v0.2.0
docker run -it \
- -p 9527:9527 -p 7750:7750 \
- -v /your-path/application.properties:/pulsar-manager/pulsar-
+ -p 9527:9527 -p 7750:7750 \
+ -v /your-path/application.properties:/pulsar-manager/pulsar-
manager/application.properties
- -v /your-path/private.key:/pulsar-manager/private.key
- -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
- apachepulsar/pulsar-manager:v0.2.0
-
+ -v /your-path/private.key:/pulsar-manager/private.key
+ -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
+ apachepulsar/pulsar-manager:v0.2.0
```
### Set the administrator account and password
```bash
-
CSRF_TOKEN=$(curl http://localhost:7750/pulsar-manager/csrf-token)
curl \
-H 'X-XSRF-TOKEN: $CSRF_TOKEN' \
@@ -114,30 +103,27 @@ curl \
-H "Content-Type: application/json" \
-X PUT http://localhost:7750/pulsar-manager/users/superuser \
-d '{"name": "admin", "password": "apachepulsar", "description": "test", "email": "username@test.org"}'
-
```
The request parameter in curl command:
```json
-
{"name": "admin", "password": "apachepulsar", "description": "test", "email": "username@test.org"}
-
```
- `name` is the Pulsar Manager login username, currently `admin`.
- `password` is the password of the current user of Pulsar Manager, currently `apachepulsar`. The password should be more than or equal to 6 digits.
-
### Configure the environment
-1. Login to the system, Visit http://localhost:9527 to login. The current default account is `admin/apachepulsar`
+
+1. Login to the system, Visit http://localhost:9527 to login. The current default account is `admin/apachepulsar`
2. Click "New Environment" button to add an environment.
3. Input the "Environment Name". The environment name is used for identifying an environment.
-4. Input the "Service URL". The Service URL is the admin service url of your Pulsar cluster.
+4. Input the "Service URL". The Service URL is the admin service URL of your Pulsar cluster.
## Other Installation
@@ -148,21 +134,17 @@ When using binary packages for direct deployment, you can follow these steps.
- Download and unzip the binary package, which is available on the [Pulsar Download](/download/) page.
```bash
-
wget https://dist.apache.org/repos/dist/release/pulsar/pulsar-manager/pulsar-manager-0.2.0/apache-pulsar-manager-0.2.0-bin.tar.gz
tar -zxvf apache-pulsar-manager-0.2.0-bin.tar.gz
-
```
- Extract the back-end service binary package and place the front-end resources in the back-end service directory.
```bash
-
cd pulsar-manager
tar -zxvf pulsar-manager.tar
cd pulsar-manager
cp -r ../dist ui
-
```
- Modify `application.properties` configuration on demand.
@@ -172,9 +154,7 @@ When using binary packages for direct deployment, you can follow these steps.
- Start Pulsar Manager
```bash
-
./bin/pulsar-manager
-
```
### Custom docker image installation
@@ -182,7 +162,6 @@ When using binary packages for direct deployment, you can follow these steps.
You can find the docker image in the [Docker Hub](https://github.com/apache/pulsar-manager/tree/master/docker) directory and build an image from the source code as well:
```bash
-
git clone https://github.com/apache/pulsar-manager
cd pulsar-manager/front-end
npm install --save
@@ -191,13 +170,10 @@ You can find the docker image in the [Docker Hub](https://github.com/apache/puls
./gradlew build -x test
cd ..
docker build -f docker/Dockerfile --build-arg BUILD_DATE=`date -u +"%Y-%m-%dT%H:%M:%SZ"` --build-arg VCS_REF=`latest` --build-arg VERSION=`latest` -t apachepulsar/pulsar-manager .
-
```
## Configuration
-
-
| application.properties | System env on Docker Image | Desc | Example |
| ----------------------------------- | -------------------------- | ------------------------------------------------------------ | ------------------------------------------------- |
| backend.jwt.token | JWT_TOKEN | token for the superuser. You need to configure this parameter during cluster initialization. | `token` |
diff --git a/site2/website-next/docs/administration-pulsar-shell.md b/site2/website-next/docs/administration-pulsar-shell.md
index 9f0bc202dd6..cbbb548457e 100644
--- a/site2/website-next/docs/administration-pulsar-shell.md
+++ b/site2/website-next/docs/administration-pulsar-shell.md
@@ -31,7 +31,7 @@ cd apache-pulsar-shell-@pulsar:version@-bin.tar.gz
Now you can enter Pulsar shell's interactive mode:
```shell
-$ ./bin/pulsar-shell
+./bin/pulsar-shell
Welcome to Pulsar shell!
Service URL: pulsar://localhost:6650/
Admin URL: http://localhost:8080/
@@ -48,7 +48,11 @@ default(localhost)>
By default, the shell tries to connect to a local Pulsar instance.
To connect to a different cluster, you have to register the cluster with Pulsar shell. You can do this in a few different ways depending on where your config file is located:
-> The configuration file must be a valid `client.conf` file, the same one you use for `pulsar-admin`, `pulsar-client` and other client tools.
+:::note
+
+The configuration file must be a valid `client.conf` file, the same one you use for `pulsar-admin`, `pulsar-client` and other client tools.
+
+:::
````mdx-code-block
<Tabs groupId="shell-config-modes"
@@ -59,7 +63,6 @@ To connect to a different cluster, you have to register the cluster with Pulsar
The `--url` value must point to a valid remote file.
-
```
default(localhost)> config create --url https://<url_to_my_client.conf> mycluster
```
@@ -163,8 +166,6 @@ admin tenants create new-tenant
admin namespaces create new-tenant/new-namespace
" > ./bin/pulsar-shell --fail-on-error -
-
-
```
</TabItem>
diff --git a/site2/website-next/docs/administration-upgrade.md b/site2/website-next/docs/administration-upgrade.md
index e2d216c2067..5ad06f04124 100644
--- a/site2/website-next/docs/administration-upgrade.md
+++ b/site2/website-next/docs/administration-upgrade.md
@@ -42,19 +42,14 @@ To upgrade an Apache Pulsar cluster, follow the upgrade sequence.
- a. Disable `autorecovery` with the following command.
```shell
-
bin/bookkeeper shell autorecovery -disable
-
```
-
- b. Roll out the upgraded version to all bookies in the cluster after you determine that a version is safe after canary.
- c. After you upgrade all bookies, re-enable `autorecovery` with the following command.
```shell
-
bin/bookkeeper shell autorecovery -enable
-
```
3. Upgrade brokers.
@@ -107,17 +102,13 @@ To upgrade a bookie to a new version, complete the following steps:
3. Start the bookie in `ReadOnly` mode to verify if the bookie of this new version runs well for reading workload.
```shell
-
bin/pulsar bookie --readOnly
-
```
4. When the bookie runs successfully in `ReadOnly` mode, stop the bookie and restart it in `Write/Read` mode.
```shell
-
bin/pulsar bookie
-
```
5. Observe and make sure the cluster serves both write and read traffic.
@@ -196,9 +187,7 @@ To check the health of the broker, you can use the following command or API.
<TabItem value="Admin CLI">
```bash
-
-$ pulsar-admin brokers healthcheck
-
+pulsar-admin brokers healthcheck
```
</TabItem>
diff --git a/site2/website-next/docs/administration-zk-bk.md b/site2/website-next/docs/administration-zk-bk.md
index d63271423c4..e3466528700 100644
--- a/site2/website-next/docs/administration-zk-bk.md
+++ b/site2/website-next/docs/administration-zk-bk.md
@@ -11,8 +11,11 @@ Pulsar relies on two external systems for essential tasks:
ZooKeeper and BookKeeper are both open-source [Apache](https://www.apache.org/) projects.
-> Skip to the [How Pulsar uses ZooKeeper and BookKeeper](#how-pulsar-uses-zookeeper-and-bookkeeper) section below for a more schematic explanation of the role of these two systems in Pulsar.
+:::tip
+Skip to the [How Pulsar uses ZooKeeper and BookKeeper](#how-pulsar-uses-zookeeper-and-bookkeeper) section below for a more schematic explanation of the role of these two systems in Pulsar.
+
+:::
## ZooKeeper
@@ -27,28 +30,24 @@ ZooKeeper manages a variety of essential coordination-related and configuration-
To deploy a Pulsar instance, you need to stand up one local ZooKeeper cluster *per Pulsar cluster*.
-To begin, add all ZooKeeper servers to the quorum configuration specified in the [`conf/zookeeper.conf`](reference-configuration.md#zookeeper) file. Add a `server.N` line for each node in the cluster to the configuration, where `N` is the number of the ZooKeeper node. The following is an example for a three-node cluster:
+To begin, add all ZooKeeper servers to the quorum configuration specified in the [`conf/zookeeper.conf`](reference-configuration.md#zookeeper) file. Add a `server.N` line for each node in the cluster to the configuration, where `N` is the number of the ZooKeeper node. The following is an example of a three-node cluster:
```properties
-
server.1=zk1.us-west.example.com:2888:3888
server.2=zk2.us-west.example.com:2888:3888
server.3=zk3.us-west.example.com:2888:3888
-
```
On each host, you need to specify the node ID in `myid` file of each node, which is in `data/zookeeper` folder of each server by default (you can change the file location via the [`dataDir`](reference-configuration.md#zookeeper-dataDir) parameter).
-> See the [Multi-server setup guide](https://zookeeper.apache.org/doc/r3.4.10/zookeeperAdmin.html#sc_zkMulitServerSetup) in the ZooKeeper documentation for detailed information on `myid` and more.
+For detailed information on `myid` and more, see the [Multi-server setup guide](https://zookeeper.apache.org/doc/r3.4.10/zookeeperAdmin.html#sc_zkMulitServerSetup) in the ZooKeeper documentation.
On a ZooKeeper server at `zk1.us-west.example.com`, for example, you can set the `myid` value like this:
```shell
-
-$ mkdir -p data/zookeeper
-$ echo 1 > data/zookeeper/myid
-
+mkdir -p data/zookeeper
+echo 1 > data/zookeeper/myid
```
On `zk2.us-west.example.com` the command is `echo 2 > data/zookeeper/myid` and so on.
@@ -56,9 +55,7 @@ On `zk2.us-west.example.com` the command is `echo 2 > data/zookeeper/myid` and s
Once you add each server to the `zookeeper.conf` configuration and each server has the appropriate `myid` entry, you can start ZooKeeper on all hosts (in the background, using nohup) with the [`pulsar-daemon`](reference-cli-tools.md#pulsar-daemon) CLI tool:
```shell
-
-$ bin/pulsar-daemon start zookeeper
-
+bin/pulsar-daemon start zookeeper
```
### Deploy configuration store
@@ -74,12 +71,10 @@ If your Pulsar instance consists of just one cluster, then you can deploy a conf
To deploy a ZooKeeper configuration store in a single-cluster instance, add the same ZooKeeper servers that the local quorum uses to the configuration file in [`conf/global_zookeeper.conf`](reference-configuration.md#configuration-store) using the same method for [local ZooKeeper](#local-zookeeper), but make sure to use a different port (2181 is the default for ZooKeeper). The following is an example that uses port 2184 for a three-node ZooKeeper cluster:
```properties
-
clientPort=2184
server.1=zk1.us-west.example.com:2185:2186
server.2=zk2.us-west.example.com:2185:2186
server.3=zk3.us-west.example.com:2185:2186
-
```
As before, create the `myid` files for each server on `data/global-zookeeper/myid`.
@@ -92,22 +87,15 @@ The key here is to make sure the ZK quorum members are spread across at least 3
Again, given the very low expected load on the configuration store servers, you can share the same hosts used for the local ZooKeeper quorum.
-For example, you can assume a Pulsar instance with the following clusters `us-west`, `us-east`, `us-central`, `eu-central`, `ap-south`. Also you can assume, each cluster has its own local ZK servers named such as
+For example, you can assume a Pulsar instance with the following clusters `us-west`, `us-east`, `us-central`, `eu-central`, `ap-south`. Also you can assume, each cluster has its own local ZK servers named such as `zk[1-3].${CLUSTER}.example.com`.
-```
-
-zk[1-3].${CLUSTER}.example.com
-
-```
-
-In this scenario you want to pick the quorum participants from few clusters and let all the others be ZK observers. For example, to form a 7 servers quorum, you can pick 3 servers from `us-west`, 2 from `us-central` and 2 from `us-east`.
+In this scenario, you want to pick the quorum participants from a few clusters and let all the others be ZK observers. For example, to form a 7 servers quorum, you can pick 3 servers from `us-west`, 2 from `us-central` and 2 from `us-east`.
-This guarantees that writes to configuration store is possible even if one of these regions is unreachable.
+This guarantees writing to the configuration store is possible even if one of these regions is unreachable.
-The ZK configuration in all the servers looks like:
+The ZK configuration in all the servers looks like below:
```properties
-
clientPort=2184
server.1=zk1.us-west.example.com:2185:2186
server.2=zk2.us-west.example.com:2185:2186
@@ -124,15 +112,12 @@ server.12=zk3.eu-central.example.com:2185:2186:observer
server.13=zk1.ap-south.example.com:2185:2186:observer
server.14=zk2.ap-south.example.com:2185:2186:observer
server.15=zk3.ap-south.example.com:2185:2186:observer
-
```
Additionally, ZK observers need to have:
```properties
-
peerType=observer
-
```
##### Start the service
@@ -140,20 +125,18 @@ peerType=observer
Once your configuration store configuration is in place, you can start up the service using [`pulsar-daemon`](reference-cli-tools.md#pulsar-daemon)
```shell
-
-$ bin/pulsar-daemon start configuration-store
-
+bin/pulsar-daemon start configuration-store
```
### ZooKeeper configuration
In Pulsar, ZooKeeper configuration is handled by two separate configuration files in the `conf` directory of your Pulsar installation:
* The `conf/zookeeper.conf` file handles the configuration for local ZooKeeper.
-* The `conf/global-zookeeper.conf` file handles the configuration for configuration store.
+* The `conf/global-zookeeper.conf` file handles the configuration for the configuration store.
See [parameters](reference-configuration.md#zookeeper) for more details.
#### Configure batching operations
-Using the batching operations reduces the remote procedure call (RPC) traffic between ZooKeeper client and servers. It also reduces the number of write transactions, because each batching operation corresponds to a single ZooKeeper transaction, containing multiple read and write operations.
+Using the batching operations reduces the remote procedure call (RPC) traffic between the ZooKeeper client and servers. It also reduces the number of write transactions, because each batching operation corresponds to a single ZooKeeper transaction, containing multiple read and write operations.
The following figure demonstrates a basic benchmark of batching read/write operations that can be requested to ZooKeeper in one second:
@@ -175,10 +158,10 @@ Bookie hosts store message data on disk. To provide optimal performance, ensure
- Disk I/O capacity read/write
- Storage capacity
-Message entries written to bookies are always synced to disk before returning an acknowledgement to the Pulsar broker by default. To ensure low write latency, BookKeeper is designed to use multiple devices:
+Message entries written to bookies are always synced to disk before returning an acknowledgment to the Pulsar broker by default. To ensure low write latency, BookKeeper is designed to use multiple devices:
- A **journal** to ensure durability. For sequential writes, it is critical to have fast [fsync](https://linux.die.net/man/2/fsync) operations on bookie hosts. Typically, small and fast [solid-state drives](https://en.wikipedia.org/wiki/Solid-state_drive) (SSDs) should suffice, or [hard disk drives](https://en.wikipedia.org/wiki/Hard_disk_drive) (HDDs) with a [RAID](https://en.wikipedia.org/wiki/RAID) controller and a battery-backed write cache. Both solutions can reach fsync latency of [...]
-- A **ledger storage device** stores data. Writes happen in the background, so write I/O is not a big concern. Reads happen sequentially most of the time and the backlog is drained only in case of consumer drain. To store large amounts of data, a typical configuration involves multiple HDDs with a RAID controller.
+- A **ledger storage device** stores data. Writes happen in the background, so writing I/O is not a big concern. Reads happen sequentially most of the time and the backlog is drained only in case of consumer drain. To store large amounts of data, a typical configuration involves multiple HDDs with a RAID controller.
### Configure BookKeeper
@@ -188,12 +171,11 @@ The minimum configuration changes required in `conf/bookkeeper.conf` are as foll
:::note
-Set `journalDirectory` and `ledgerDirectories` carefully. It is difficilt to change them later.
+Set `journalDirectory` and `ledgerDirectories` carefully. It is difficult to change them later.
:::
```properties
-
# Change to point to journal disk mount point
journalDirectory=data/bookkeeper/journal
@@ -205,12 +187,11 @@ zkServers=zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181
#It is recommended to set this parameter. Otherwise, BookKeeper can't start normally in certain environments (for example, Huawei Cloud).
advertisedAddress=
-
```
To change the ZooKeeper root path that BookKeeper uses, use `zkLedgersRootPath=/MY-PREFIX/ledgers` instead of `zkServers=localhost:2181/MY-PREFIX`.
-> For more information about BookKeeper, refer to the official [BookKeeper docs](http://bookkeeper.apache.org).
+For more information about BookKeeper, refer to the official [BookKeeper docs](http://bookkeeper.apache.org).
### Deploy BookKeeper
@@ -223,25 +204,19 @@ You can start a bookie in the foreground or as a background daemon.
To start a bookie in the foreground, use the [`bookkeeper`](reference-cli-tools.md#bookkeeper) CLI tool:
```bash
-
-$ bin/bookkeeper bookie
-
+bin/bookkeeper bookie
```
To start a bookie in the background, use the [`pulsar-daemon`](reference-cli-tools.md#pulsar-daemon) CLI tool:
```bash
-
-$ bin/pulsar-daemon start bookie
-
+bin/pulsar-daemon start bookie
```
You can verify whether the bookie works properly with the `bookiesanity` command for the [BookKeeper shell](reference-cli-tools.md#bookkeeper-shell):
```shell
-
-$ bin/bookkeeper shell bookiesanity
-
+bin/bookkeeper shell bookiesanity
```
When you use this command, you create a new ledger on the local bookie, write a few entries, read them back and finally delete the ledger.
@@ -258,28 +233,26 @@ Before you decommission a bookie, you need to check your environment and meet th
And then you can decommission bookies safely. To decommission bookies, complete the following steps.
-1. Log in to the bookie node, check if there are underreplicated ledgers. The decommission command force to replicate the underreplicated ledgers.
-`$ bin/bookkeeper shell listunderreplicated`
+1. Log in to the bookie node, and check if there are under-replicated ledgers. The decommission command force to replicate the underreplicated ledgers.
+`bin/bookkeeper shell listunderreplicated`
2. Stop the bookie by killing the bookie process. Make sure that no liveness/readiness probes setup for the bookies to spin them back up if you deploy it in a Kubernetes environment.
3. Run the decommission command.
- If you have logged in to the node to be decommissioned, you do not need to provide `-bookieid`.
- If you are running the decommission command for the target bookie node from another bookie node, you should mention the target bookie ID in the arguments for `-bookieid`
- `$ bin/bookkeeper shell decommissionbookie`
+ `bin/bookkeeper shell decommissionbookie`
or
- `$ bin/bookkeeper shell decommissionbookie -bookieid <target bookieid>`
+ `bin/bookkeeper shell decommissionbookie -bookieid <target bookieid>`
4. Validate that no ledgers are on the decommissioned bookie.
-`$ bin/bookkeeper shell listledgers -bookieid <target bookieid>`
+`bin/bookkeeper shell listledgers -bookieid <target bookieid>`
-You can run the following command to check if the bookie you have decommissioned is listed in the bookies list:
+You can run the following command to check if the bookie you have decommissioned is listed:
```bash
-
./bookkeeper shell listbookies -rw -h
./bookkeeper shell listbookies -ro -h
-
```
## BookKeeper persistence policies
@@ -309,11 +282,9 @@ Flag | Description | Default
The following is an example:
```shell
-
-$ pulsar-admin namespaces set-persistence my-tenant/my-ns \
- --bookkeeper-ack-quorum 3 \
- --bookkeeper-ensemble 2
-
+pulsar-admin namespaces set-persistence my-tenant/my-ns \
+--bookkeeper-ack-quorum 3 \
+--bookkeeper-ensemble 2
```
#### REST API
@@ -323,7 +294,6 @@ $ pulsar-admin namespaces set-persistence my-tenant/my-ns \
#### Java
```java
-
int bkEnsemble = 2;
int bkQuorum = 3;
int bkAckQuorum = 2;
@@ -331,7 +301,6 @@ double markDeleteRate = 0.7;
PersistencePolicies policies =
new PersistencePolicies(ensemble, quorum, ackQuorum, markDeleteRate);
admin.namespaces().setPersistence(namespace, policies);
-
```
### List persistence policies
@@ -345,15 +314,13 @@ Use the [`get-persistence`](/tools/pulsar-admin/) subcommand and specify the nam
The following is an example:
```shell
-
-$ pulsar-admin namespaces get-persistence my-tenant/my-ns
+pulsar-admin namespaces get-persistence my-tenant/my-ns
{
"bookkeeperEnsemble": 1,
"bookkeeperWriteQuorum": 1,
"bookkeeperAckQuorum", 1,
"managedLedgerMaxMarkDeleteRate": 0
}
-
```
#### REST API
@@ -363,9 +330,7 @@ $ pulsar-admin namespaces get-persistence my-tenant/my-ns
#### Java
```java
-
PersistencePolicies policies = admin.namespaces().getPersistence(namespace);
-
```
## How Pulsar uses ZooKeeper and BookKeeper
diff --git a/site2/website-next/docs/client-libraries-cpp.md b/site2/website-next/docs/client-libraries-cpp.md
index 9988989d5a9..27c679ab06f 100644
--- a/site2/website-next/docs/client-libraries-cpp.md
+++ b/site2/website-next/docs/client-libraries-cpp.md
@@ -10,7 +10,7 @@ All the methods in producer, consumer, and reader of a C++ client are thread-saf
## Supported platforms
-Pulsar C++ client is supported on **Linux** ,**MacOS** and **Windows** platforms.
+Pulsar C++ client is supported on **Linux**, **macOS** and **Windows** platforms.
Doxygen-generated API docs for the C++ client are available [here](/api/cpp).
@@ -37,51 +37,43 @@ You need to install the following components before using the C++ client:
1. Clone the Pulsar repository.
```shell
-
-$ git clone https://github.com/apache/pulsar
-
+git clone https://github.com/apache/pulsar
```
2. Install all necessary dependencies.
```shell
-
-$ apt-get install cmake libssl-dev libcurl4-openssl-dev liblog4cxx-dev \
- libprotobuf-dev protobuf-compiler libboost-all-dev google-mock libgtest-dev libjsoncpp-dev
-
+apt-get install cmake libssl-dev libcurl4-openssl-dev liblog4cxx-dev \
+libprotobuf-dev protobuf-compiler libboost-all-dev google-mock libgtest-dev libjsoncpp-dev
```
3. Compile and install [Google Test](https://github.com/google/googletest).
```shell
-
# libgtest-dev version is 1.18.0 or above
-$ cd /usr/src/googletest
-$ sudo cmake .
-$ sudo make
-$ sudo cp ./googlemock/libgmock.a ./googlemock/gtest/libgtest.a /usr/lib/
+cd /usr/src/googletest
+sudo cmake .
+sudo make
+sudo cp ./googlemock/libgmock.a ./googlemock/gtest/libgtest.a /usr/lib/
# less than 1.18.0
-$ cd /usr/src/gtest
-$ sudo cmake .
-$ sudo make
-$ sudo cp libgtest.a /usr/lib
-
-$ cd /usr/src/gmock
-$ sudo cmake .
-$ sudo make
-$ sudo cp libgmock.a /usr/lib
+cd /usr/src/gtest
+sudo cmake .
+sudo make
+sudo cp libgtest.a /usr/lib
+cd /usr/src/gmock
+sudo cmake .
+sudo make
+sudo cp libgmock.a /usr/lib
```
4. Compile the Pulsar client library for C++ inside the Pulsar repository.
```shell
-
-$ cd pulsar-client-cpp
-$ cmake .
-$ make
-
+cd pulsar-client-cpp
+cmake .
+make
```
After you install the components successfully, the files `libpulsar.so` and `libpulsar.a` are in the `lib` folder of the repository. The tools `perfProducer` and `perfConsumer` are in the `perf` directory.
@@ -92,49 +84,43 @@ After you install the components successfully, the files `libpulsar.so` and `lib
After you download and install RPM or DEB, the `libpulsar.so`, `libpulsarnossl.so`, `libpulsar.a`, and `libpulsarwithdeps.a` libraries are in your `/usr/lib` directory.
-By default, they are built in code path `${PULSAR_HOME}/pulsar-client-cpp`. You can build with the command below.
+By default, they are built-in code path `${PULSAR_HOME}/pulsar-client-cpp`. You can build with the command below.
- `cmake . -DBUILD_TESTS=OFF -DLINK_STATIC=ON && make pulsarShared pulsarSharedNossl pulsarStatic pulsarStaticWithDeps -j 3`.
+ ```bash
+ cmake . -DBUILD_TESTS=OFF -DLINK_STATIC=ON && make pulsarShared pulsarSharedNossl pulsarStatic pulsarStaticWithDeps -j 3
+ ```
-These libraries rely on some other libraries. If you want to get detailed version of dependencies, see [RPM](https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/pkg/rpm/Dockerfile) or [DEB](https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/pkg/deb/Dockerfile) files.
+These libraries rely on some other libraries. If you want to get a detailed version of dependencies, see [RPM](https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/pkg/rpm/Dockerfile) or [DEB](https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/pkg/deb/Dockerfile) files.
1. `libpulsar.so` is a shared library, containing statically linked `boost` and `openssl`. It also dynamically links all other necessary libraries. You can use this Pulsar library with the command below.
```bash
-
g++ --std=c++11 PulsarTest.cpp -o test /usr/lib/libpulsar.so -I/usr/local/ssl/include
-
```
2. `libpulsarnossl.so` is a shared library, similar to `libpulsar.so` except that the libraries `openssl` and `crypto` are dynamically linked. You can use this Pulsar library with the command below.
```bash
-
g++ --std=c++11 PulsarTest.cpp -o test /usr/lib/libpulsarnossl.so -lssl -lcrypto -I/usr/local/ssl/include -L/usr/local/ssl/lib
-
```
3. `libpulsar.a` is a static library. You need to load dependencies before using this library. You can use this Pulsar library with the command below.
```bash
-
g++ --std=c++11 PulsarTest.cpp -o test /usr/lib/libpulsar.a -lssl -lcrypto -ldl -lpthread -I/usr/local/ssl/include -L/usr/local/ssl/lib -lboost_system -lboost_regex -lcurl -lprotobuf -lzstd -lz
-
```
4. `libpulsarwithdeps.a` is a static library, based on `libpulsar.a`. It is archived in the dependencies of `libboost_regex`, `libboost_system`, `libcurl`, `libprotobuf`, `libzstd` and `libz`. You can use this Pulsar library with the command below.
```bash
-
g++ --std=c++11 PulsarTest.cpp -o test /usr/lib/libpulsarwithdeps.a -lssl -lcrypto -ldl -lpthread -I/usr/local/ssl/include -L/usr/local/ssl/lib
-
```
The `libpulsarwithdeps.a` does not include library openssl related libraries `libssl` and `libcrypto`, because these two libraries are related to security. It is more reasonable and easier to use the versions provided by the local system to handle security issues and upgrade libraries.
### Install RPM
-1. Download a RPM package from the links in the table.
+1. Download an RPM package from the links in the table.
| Link | Crypto files |
|------|--------------|
@@ -145,18 +131,14 @@ The `libpulsarwithdeps.a` does not include library openssl related libraries `li
2. Install the package using the following command.
```bash
-
-$ rpm -ivh apache-pulsar-client*.rpm
-
+rpm -ivh apache-pulsar-client*.rpm
```
After you install RPM successfully, Pulsar libraries are in the `/usr/lib` directory, for example：
```bash
-
lrwxrwxrwx 1 root root 18 Dec 30 22:21 libpulsar.so -> libpulsar.so.2.9.1
lrwxrwxrwx 1 root root 23 Dec 30 22:21 libpulsarnossl.so -> libpulsarnossl.so.2.9.1
-
```
:::note
@@ -168,10 +150,8 @@ If you get the error that `libpulsar.so: cannot open shared object file: No such
2. Install the GCC and g++ using the following command, otherwise errors would occur in installing Node.js.
```bash
-
-$ sudo yum -y install gcc automake autoconf libtool make
-$ sudo yum -y install gcc-c++
-
+sudo yum -y install gcc automake autoconf libtool make
+sudo yum -y install gcc-c++
```
### Install Debian
@@ -186,16 +166,14 @@ $ sudo yum -y install gcc-c++
2. Install the package using the following command.
```bash
-
-$ apt install ./apache-pulsar-client*.deb
-
+apt install ./apache-pulsar-client*.deb
```
After you install DEB successfully, Pulsar libraries are in the `/usr/lib` directory.
### Build
-> If you want to build RPM and Debian packages from the latest master, follow the instructions below. You should run all the instructions at the root directory of your cloned Pulsar repository.
+If you want to build RPM and Debian packages from the latest master, follow the instructions below. You must run all the instructions at the root directory of your cloned Pulsar repository.
There are recipes that build RPM and Debian packages containing a
statically linked `libpulsar.so` / `libpulsarnossl.so` / `libpulsar.a` / `libpulsarwithdeps.a` with all required dependencies.
@@ -203,9 +181,7 @@ statically linked `libpulsar.so` / `libpulsarnossl.so` / `libpulsar.a` / `libpul
To build the C++ library packages, you need to build the Java packages first.
```shell
-
mvn install -DskipTests
-
```
#### RPM
@@ -213,9 +189,7 @@ mvn install -DskipTests
To build the RPM inside a Docker container, use the command below. The RPMs are in the `pulsar-client-cpp/pkg/rpm/RPMS/x86_64/` path.
```shell
-
pulsar-client-cpp/pkg/rpm/docker-build-rpm.sh
-
```
| Package name | Content |
@@ -229,9 +203,7 @@ pulsar-client-cpp/pkg/rpm/docker-build-rpm.sh
To build Debian packages, enter the following command.
```shell
-
pulsar-client-cpp/pkg/deb/docker-build-deb.sh
-
```
Debian packages are created in the `pulsar-client-cpp/pkg/deb/BUILD/DEB/` path.
@@ -248,41 +220,35 @@ Debian packages are created in the `pulsar-client-cpp/pkg/deb/BUILD/DEB/` path.
1. Clone the Pulsar repository.
```shell
-
-$ git clone https://github.com/apache/pulsar
-
+git clone https://github.com/apache/pulsar
```
2. Install all necessary dependencies.
```shell
-
# OpenSSL installation
-$ brew install openssl
-$ export OPENSSL_INCLUDE_DIR=/usr/local/opt/openssl/include/
-$ export OPENSSL_ROOT_DIR=/usr/local/opt/openssl/
+brew install openssl
+export OPENSSL_INCLUDE_DIR=/usr/local/opt/openssl/include/
+export OPENSSL_ROOT_DIR=/usr/local/opt/openssl/
# Protocol Buffers installation
-$ brew install protobuf boost boost-python log4cxx
-# If you are using python3, you need to install boost-python3
+brew install protobuf boost boost-python log4cxx
+# If you are using python3, you need to install boost-python3
# Google Test installation
-$ git clone https://github.com/google/googletest.git
-$ cd googletest
-$ git checkout release-1.12.1
-$ cmake .
-$ make install
-
+git clone https://github.com/google/googletest.git
+cd googletest
+git checkout release-1.12.1
+cmake .
+make install
```
3. Compile the Pulsar client library in the repository that you cloned.
```shell
-
-$ cd pulsar-client-cpp
-$ cmake .
-$ make
-
+cd pulsar-client-cpp
+cmake .
+make
```
### Install `libpulsar`
@@ -290,9 +256,7 @@ $ make
Pulsar releases are available in the [Homebrew](https://brew.sh/) core repository. You can install the C++ client library with the following command. The package is installed with the library and headers.
```shell
-
brew install libpulsar
-
```
## Windows (64-bit)
@@ -302,41 +266,35 @@ brew install libpulsar
1. Clone the Pulsar repository.
```shell
-
-$ git clone https://github.com/apache/pulsar
-
+git clone https://github.com/apache/pulsar
```
2. Install all necessary dependencies.
```shell
-
cd ${PULSAR_HOME}/pulsar-client-cpp
vcpkg install --feature-flags=manifests --triplet x64-windows
-
```
3. Build C++ libraries.
```shell
-
cmake -B ./build -A x64 -DBUILD_PYTHON_WRAPPER=OFF -DBUILD_TESTS=OFF -DVCPKG_TRIPLET=x64-windows -DCMAKE_BUILD_TYPE=Release -S .
cmake --build ./build --config Release
-
```
-> **NOTE**
->
-> 1. For Windows 32-bit, you need to use `-A Win32` and `-DVCPKG_TRIPLET=x86-windows`.
-> 2. For MSVC Debug mode, you need to replace `Release` with `Debug` for both `CMAKE_BUILD_TYPE` variable and `--config` option.
+:::note
+
+* For Windows 32-bit, you need to use `-A Win32` and `-DVCPKG_TRIPLET=x86-windows`.
+* For MSVC Debug mode, you need to replace `Release` with `Debug` for both `CMAKE_BUILD_TYPE` variable and `--config` option.
+
+:::
4. Client libraries are available in the following places.
```
-
${PULSAR_HOME}/pulsar-client-cpp/build/lib/Release/pulsar.lib
${PULSAR_HOME}/pulsar-client-cpp/build/lib/Release/pulsar.dll
-
```
## Connection URLs
@@ -372,7 +330,6 @@ To use Pulsar as a producer, you need to create a producer on the C++ client. Th
This example sends 100 messages using the blocking style. While simple, it does not produce high throughput as it waits for each ack to come back before sending the next message.
```cpp
-
#include <pulsar/Client.h>
#include <thread>
@@ -410,19 +367,17 @@ int main() {
client.close();
return 0;
}
-
```
### Non-blocking example
-This example sends 100 messages using the non-blocking style calling `sendAsync` instead of `send`. This allows the producer to have multiple messages inflight at a time which increases throughput.
+This example sends 100 messages using the non-blocking style calling `sendAsync` instead of `send`. This allows the producer to have multiple messages in-flight at a time which increases throughput.
The producer configuration `blockIfQueueFull` is useful here to avoid `ResultProducerQueueIsFull` errors when the internal queue for outgoing send requests becomes full. Once the internal queue is full, `sendAsync` becomes blocking which can make your code simpler.
Without this configuration, the result code `ResultProducerQueueIsFull` is passed to the callback. You must decide how to deal with that (retry, discard etc).
```cpp
-
#include <pulsar/Client.h>
#include <thread>
#include <atomic>
@@ -473,7 +428,6 @@ int main() {
client.close();
return 0;
}
-
```
### Partitioned topics and lazy producers
@@ -490,11 +444,9 @@ With our example above, that reduces the number of internal producers spread out
Note that there can be extra latency for the first message sent. If you set a low send timeout, this timeout could be reached if the initial connection handshake is slow to complete.
```cpp
-
ProducerConfiguration producerConf;
producerConf.setPartitionsRoutingMode(ProducerConfiguration::UseSinglePartition);
producerConf.setLazyStartPartitionedProducers(true);
-
```
### Enable chunking
@@ -504,22 +456,24 @@ Message [chunking](concepts-messaging.md#chunking) enables Pulsar to process lar
The message chunking feature is OFF by default. The following is an example about how to enable message chunking when creating a producer.
```cpp
-
ProducerConfiguration conf;
conf.setBatchingEnabled(false);
conf.setChunkingEnabled(true);
Producer producer;
client.createProducer("my-topic", conf, producer);
-
```
-> **Note:** To enable chunking, you need to disable batching (`setBatchingEnabled`=`false`) concurrently.
+:::note
+
+To enable chunking, you need to disable batching (`setBatchingEnabled`=`false`) concurrently.
+
+:::
## Create a consumer
To use Pulsar as a consumer, you need to create a consumer on the C++ client. There are two main ways of using the consumer:
- [Blocking style](#blocking-example): synchronously calling `receive(msg)`.
-- [Non-blocking](#consumer-with-a-message-listener) (event based) style: using a message listener.
+- [Non-blocking](#consumer-with-a-message-listener) (event-based) style: using a message listener.
### Blocking example
@@ -528,7 +482,6 @@ The benefit of this approach is that it is the simplest code. Simply keeps calli
This example starts a subscription at the earliest offset and consumes 100 messages.
```cpp
-
#include <pulsar/Client.h>
using namespace pulsar;
@@ -562,17 +515,15 @@ int main() {
client.close();
return 0;
}
-
```
### Consumer with a message listener
-You can avoid running a loop with blocking calls with an event based style by using a message listener which is invoked for each message that is received.
+You can avoid running a loop by blocking calls with an event-based style by using a message listener which is invoked for each message that is received.
This example starts a subscription at the earliest offset and consumes 100 messages.
```cpp
-
#include <pulsar/Client.h>
#include <atomic>
#include <thread>
@@ -614,7 +565,6 @@ int main() {
client.close();
return 0;
}
-
```
### Configure chunking
@@ -624,20 +574,17 @@ You can limit the maximum number of chunked messages a consumer maintains concur
The following is an example of how to configure message chunking.
```cpp
-
ConsumerConfiguration conf;
conf.setAutoAckOldestChunkedMessageOnQueueFull(true);
conf.setMaxPendingChunkedMessage(100);
Consumer consumer;
client.subscribe("my-topic", "my-sub", conf, consumer);
-
```
## Enable authentication in connection URLs
If you use TLS authentication when connecting to Pulsar, you need to add `ssl` in the connection URLs, and the default port is `6651`. The following is an example.
```cpp
-
ClientConfiguration config = ClientConfiguration();
config.setUseTls(true);
config.setTlsTrustCertsFilePath("/path/to/cacert.pem");
@@ -646,7 +593,6 @@ config.setAuth(pulsar::AuthTls::create(
"/path/to/client-cert.pem", "/path/to/client-key.pem"););
Client client("pulsar+ssl://my-broker.com:6651", config);
-
```
For complete examples, refer to [C++ client examples](https://github.com/apache/pulsar/tree/master/pulsar-client-cpp/examples).
@@ -661,7 +607,6 @@ schema, see [Pulsar schema](schema-get-started.md).
- The following example shows how to create a producer with an Avro schema.
```cpp
-
static const std::string exampleSchema =
"{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\","
"\"fields\":[{\"name\":\"a\",\"type\":\"int\"},{\"name\":\"b\",\"type\":\"int\"}]}";
@@ -669,13 +614,11 @@ schema, see [Pulsar schema](schema-get-started.md).
ProducerConfiguration producerConf;
producerConf.setSchema(SchemaInfo(AVRO, "Avro", exampleSchema));
client.createProducer("topic-avro", producerConf, producer);
-
```
- The following example shows how to create a consumer with an Avro schema.
```cpp
-
static const std::string exampleSchema =
"{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\","
"\"fields\":[{\"name\":\"a\",\"type\":\"int\"},{\"name\":\"b\",\"type\":\"int\"}]}";
@@ -683,50 +626,32 @@ schema, see [Pulsar schema](schema-get-started.md).
Consumer consumer;
consumerConf.setSchema(SchemaInfo(AVRO, "Avro", exampleSchema));
client.subscribe("topic-avro", "sub-2", consumerConf, consumer)
-
```
### ProtobufNative schema
The following example shows how to create a producer and a consumer with a ProtobufNative schema.
-
-1. Generate the `User` class using Protobuf3.
-
- :::note
- You need to use Protobuf3 or later versions.
-
- :::
-
-
+1. Generate the `User` class using Protobuf3 or later versions.
```protobuf
-
syntax = "proto3";
message User {
string name = 1;
int32 age = 2;
}
-
```
-
2. Include the `ProtobufNativeSchema.h` in your source code. Ensure the Protobuf dependency has been added to your project.
-
```cpp
-
#include <pulsar/ProtobufNativeSchema.h>
-
```
-
3. Create a producer to send a `User` instance.
-
```cpp
-
ProducerConfiguration producerConf;
producerConf.setSchema(createProtobufNativeSchema(User::GetDescriptor()));
Producer producer;
@@ -737,15 +662,11 @@ The following example shows how to create a producer and a consumer with a Proto
std::string content;
user.SerializeToString(&content);
producer.send(MessageBuilder().setContent(content).build());
-
```
-
4. Create a consumer to receive a `User` instance.
-
```cpp
-
ConsumerConfiguration consumerConf;
consumerConf.setSchema(createProtobufNativeSchema(User::GetDescriptor()));
consumerConf.setSubscriptionInitialPosition(InitialPositionEarliest);
@@ -755,6 +676,5 @@ The following example shows how to create a producer and a consumer with a Proto
consumer.receive(msg);
User user2;
user2.ParseFromArray(msg.getData(), msg.getLength());
-
```
diff --git a/site2/website-next/docs/client-libraries-dotnet.md b/site2/website-next/docs/client-libraries-dotnet.md
index 3bd5367d20f..0401b578fe4 100644
--- a/site2/website-next/docs/client-libraries-dotnet.md
+++ b/site2/website-next/docs/client-libraries-dotnet.md
@@ -8,7 +8,7 @@ You can use the Pulsar C# client (DotPulsar) to create Pulsar producers and cons
## Installation
-You can install the Pulsar C# client library either through the dotnet CLI or through the Visual Studio. This section describes how to install the Pulsar C# client library through the dotnet CLI. For information about how to install the Pulsar C# client library through the Visual Studio, see [here](https://docs.microsoft.com/en-us/visualstudio/mac/nuget-walkthrough?view=vsmac-2019).
+You can install the Pulsar C# client library either through the dotnet CLI or through Visual Studio. This section describes how to install the Pulsar C# client library through the dotnet CLI. For information about how to install the Pulsar C# client library through Visual Studio, see [here](https://docs.microsoft.com/en-us/visualstudio/mac/nuget-walkthrough?view=vsmac-2019).
### Prerequisites
@@ -16,7 +16,7 @@ Install the [.NET Core SDK](https://dotnet.microsoft.com/download/), which provi
### Procedures
-To install the Pulsar C# client library, following these steps:
+To install the Pulsar C# client library, follow these steps:
1. Create a project.
@@ -27,9 +27,7 @@ To install the Pulsar C# client library, following these steps:
3. Create the project using the following command.
```
-
dotnet new console
-
```
4. Use `dotnet run` to test that the app has been created properly.
@@ -39,19 +37,15 @@ To install the Pulsar C# client library, following these steps:
1. Use the following command to install the `DotPulsar` package.
```
-
dotnet add package DotPulsar
-
```
2. After the command completes, open the `.csproj` file to see the added reference.
```xml
-
<ItemGroup>
<PackageReference Include="DotPulsar" Version="2.0.1" />
</ItemGroup>
-
```
## Connection URLs
@@ -85,11 +79,9 @@ This section describes some configuration examples for the Pulsar C# client.
This example shows how to create a Pulsar C# client connected to localhost.
```csharp
-
using DotPulsar;
var client = PulsarClient.Builder().Build();
-
```
To create a Pulsar C# client by using the builder, you can specify the following options.
@@ -106,25 +98,21 @@ This section describes how to create a producer.
- Create a producer by using the builder.
```csharp
-
using DotPulsar;
using DotPulsar.Extensions;
var producer = client.NewProducer()
.Topic("persistent://public/default/mytopic")
.Create();
-
```
- Create a producer without using the builder.
```csharp
-
using DotPulsar;
var options = new ProducerOptions<byte[]>("persistent://public/default/mytopic", Schema.ByteArray);
var producer = client.CreateProducer(options);
-
```
### Create consumer
@@ -134,7 +122,6 @@ This section describes how to create a consumer.
- Create a consumer by using the builder.
```csharp
-
using DotPulsar;
using DotPulsar.Extensions;
@@ -142,18 +129,15 @@ This section describes how to create a consumer.
.SubscriptionName("MySubscription")
.Topic("persistent://public/default/mytopic")
.Create();
-
```
- Create a consumer without using the builder.
```csharp
-
using DotPulsar;
var options = new ConsumerOptions<byte[]>("MySubscription", "persistent://public/default/mytopic", Schema.ByteArray);
var consumer = client.CreateConsumer(options);
-
```
### Create reader
@@ -163,7 +147,6 @@ This section describes how to create a reader.
- Create a reader by using the builder.
```csharp
-
using DotPulsar;
using DotPulsar.Extensions;
@@ -171,18 +154,15 @@ This section describes how to create a reader.
.StartMessageId(MessageId.Earliest)
.Topic("persistent://public/default/mytopic")
.Create();
-
```
- Create a reader without using the builder.
```csharp
-
using DotPulsar;
var options = new ReaderOptions<byte[]>(MessageId.Earliest, "persistent://public/default/mytopic", Schema.ByteArray);
var reader = client.CreateReader(options);
-
```
### Configure encryption policies
@@ -197,13 +177,11 @@ The Pulsar C# client supports four kinds of encryption policies:
This example shows how to set the `EnforceUnencrypted` encryption policy.
```csharp
-
using DotPulsar;
var client = PulsarClient.Builder()
.ConnectionSecurity(EncryptionPolicy.EnforceEncrypted)
.Build();
-
```
### Configure authentication
@@ -215,15 +193,12 @@ If you have followed [Authentication using TLS](security-tls-authentication.md),
1. Create an unencrypted and password-less pfx file.
```csharp
-
openssl pkcs12 -export -keypbe NONE -certpbe NONE -out admin.pfx -inkey admin.key.pem -in admin.cert.pem -passout pass:
-
```
2. Use the admin.pfx file to create an X509Certificate2 and pass it to the Pulsar C# client.
```csharp
-
using System.Security.Cryptography.X509Certificates;
using DotPulsar;
@@ -231,22 +206,19 @@ If you have followed [Authentication using TLS](security-tls-authentication.md),
var client = PulsarClient.Builder()
.AuthenticateUsingClientCertificate(clientCertificate)
.Build();
-
```
## Producer
-A producer is a process that attaches to a topic and publishes messages to a Pulsar broker for processing. This section describes some configuration examples about the producer.
+A producer is a process that attaches to a topic and publishes messages to a Pulsar broker for processing. This section describes some configuration examples of the producer.
## Send data
This example shows how to send data.
```csharp
-
var data = Encoding.UTF8.GetBytes("Hello World");
await producer.Send(data);
-
```
### Send messages with customized metadata
@@ -254,22 +226,18 @@ await producer.Send(data);
- Send messages with customized metadata by using the builder.
```csharp
-
var messageId = await producer.NewMessage()
.Property("SomeKey", "SomeValue")
.Send(data);
-
```
- Send messages with customized metadata without using the builder.
```csharp
-
var data = Encoding.UTF8.GetBytes("Hello World");
var metadata = new MessageMetadata();
metadata["SomeKey"] = "SomeValue";
var messageId = await producer.Send(metadata, data));
-
```
## Consumer
@@ -281,32 +249,26 @@ A consumer is a process that attaches to a topic through a subscription and then
This example shows how a consumer receives messages from a topic.
```csharp
-
await foreach (var message in consumer.Messages())
{
Console.WriteLine("Received: " + Encoding.UTF8.GetString(message.Data.ToArray()));
}
-
```
### Acknowledge messages
-Messages can be acknowledged individually or cumulatively. For details about message acknowledgement, see [acknowledgement](concepts-messaging.md#acknowledgement).
+Messages can be acknowledged individually or cumulatively. For details about message acknowledgment, see [acknowledgment](concepts-messaging.md#acknowledgment).
- Acknowledge messages individually.
```csharp
-
await consumer.Acknowledge(message);
-
```
- Acknowledge messages cumulatively.
```csharp
-
await consumer.AcknowledgeCumulative(message);
-
```
### Unsubscribe from topics
@@ -314,28 +276,26 @@ Messages can be acknowledged individually or cumulatively. For details about mes
This example shows how a consumer unsubscribes from a topic.
```csharp
-
await consumer.Unsubscribe();
-
```
-#### Note
+:::note
-> A consumer cannot be used and is disposed once the consumer unsubscribes from a topic.
+A consumer cannot be used and is disposed once the consumer unsubscribes from a topic.
+
+:::
## Reader
-A reader is actually just a consumer without a cursor. This means that Pulsar does not keep track of your progress and there is no need to acknowledge messages.
+A reader is just a consumer without a cursor. This means that Pulsar does not keep track of your progress and there is no need to acknowledge messages.
This example shows how a reader receives messages.
```csharp
-
await foreach (var message in reader.Messages())
{
Console.WriteLine("Received: " + Encoding.UTF8.GetString(message.Data.ToArray()));
}
-
```
## Monitoring
@@ -357,7 +317,6 @@ The following table lists states available for the producer.
This example shows how to monitor the producer state.
```csharp
-
private static async ValueTask Monitor(IProducer producer, CancellationToken cancellationToken)
{
var state = ProducerState.Disconnected;
@@ -382,7 +341,6 @@ private static async ValueTask Monitor(IProducer producer, CancellationToken can
return;
}
}
-
```
### Monitor consumer state
@@ -402,7 +360,6 @@ The following table lists states available for the consumer.
This example shows how to monitor the consumer state.
```csharp
-
private static async ValueTask Monitor(IConsumer consumer, CancellationToken cancellationToken)
{
var state = ConsumerState.Disconnected;
@@ -429,7 +386,6 @@ private static async ValueTask Monitor(IConsumer consumer, CancellationToken can
return;
}
}
-
```
### Monitor reader state
@@ -444,10 +400,9 @@ The following table lists states available for the reader.
| Faulted | An unrecoverable error has occurred. |
| ReachedEndOfTopic | No more messages are delivered. |
-This example shows how to monitor the reader state.
+This example shows how to monitor the reader's state.
```csharp
-
private static async ValueTask Monitor(IReader reader, CancellationToken cancellationToken)
{
var state = ReaderState.Disconnected;
@@ -472,6 +427,5 @@ private static async ValueTask Monitor(IReader reader, CancellationToken cancell
return;
}
}
-
```
diff --git a/site2/website-next/docs/client-libraries-go.md b/site2/website-next/docs/client-libraries-go.md
index f4d7cbdac56..a7a72cdf2f2 100644
--- a/site2/website-next/docs/client-libraries-go.md
+++ b/site2/website-next/docs/client-libraries-go.md
@@ -14,39 +14,31 @@ API docs are available on the [Godoc](https://godoc.org/github.com/apache/pulsar
You can get the `pulsar` library by using `go get` or use it with `go module`.
-Download the library of Go client to local environment:
+Download the library of Go client to your local environment:
```bash
-
-$ go get -u "github.com/apache/pulsar-client-go/pulsar"
-
+go get -u "github.com/apache/pulsar-client-go/pulsar"
```
Once installed locally, you can import it into your project:
```go
-
import "github.com/apache/pulsar-client-go/pulsar"
-
```
Use with go module:
```bash
-
-$ mkdir test_dir && cd test_dir
-
+mkdir test_dir && cd test_dir
```
Write a sample script in the `test_dir` directory (such as `test_example.go`) and write `package main` at the beginning of the file.
```bash
-
-$ go mod init test_dir
-$ go mod tidy && go mod download
-$ go build test_example.go
-$ ./test_example
-
+go mod init test_dir
+go mod tidy && go mod download
+go build test_example.go
+./test_example
```
## Connection URLs
@@ -73,10 +65,9 @@ pulsar+ssl://pulsar.us-west.example.com:6651
## Create a client
-In order to interact with Pulsar, you'll first need a `Client` object. You can create a client object using the `NewClient` function, passing in a `ClientOptions` object (more on configuration [below](#client-configuration)). Here's an example:
+To interact with Pulsar, you need a `Client` object first. You can create a client object using the `NewClient` function, passing in a `ClientOptions` object (more on configuration [below](#client-configuration)). Here's an example:
```go
-
import (
"log"
"time"
@@ -96,13 +87,11 @@ func main() {
defer client.Close()
}
-
```
If you have multiple brokers, you can initiate a client object as below.
```go
-
import (
"log"
"time"
@@ -121,7 +110,6 @@ func main() {
defer client.Close()
}
-
```
The following configurable parameters are available for Pulsar clients:
@@ -145,7 +133,6 @@ The following configurable parameters are available for Pulsar clients:
Pulsar producers publish messages to Pulsar topics. You can [configure](#producer-configuration) Go producers using a `ProducerOptions` object. Here's an example:
```go
-
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "my-topic",
})
@@ -164,7 +151,6 @@ if err != nil {
fmt.Println("Failed to publish message", err)
}
fmt.Println("Published message")
-
```
### Producer operations
@@ -186,7 +172,6 @@ Method | Description | Return type
#### How to use message router in producer
```go
-
client, err := NewClient(pulsar.ClientOptions{
URL: serviceURL,
})
@@ -219,18 +204,15 @@ if err != nil {
log.Fatal(err)
}
defer producer.Close()
-
```
#### How to use schema interface in producer
```go
-
type testJSON struct {
ID int `json:"id"`
Name string `json:"name"`
}
-
```
```go
@@ -239,11 +221,9 @@ var (
exampleSchemaDef = "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," +
"\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":\"string\"}]}"
)
-
```
```go
-
client, err := NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
@@ -271,13 +251,11 @@ if err != nil {
log.Fatal(err)
}
producer.Close()
-
```
#### How to use delay relative in producer
```go
-
client, err := NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
@@ -330,7 +308,6 @@ if err != nil {
}
fmt.Println(msg.Payload())
canc()
-
```
#### How to use Prometheus metrics in producer
@@ -340,7 +317,6 @@ Pulsar Go client registers client metrics using Prometheus. This section demonst
1. Write a simple producer application.
```go
-
// Create a Pulsar client
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
@@ -394,20 +370,17 @@ err = http.ListenAndServe(":"+strconv.Itoa(webPort), nil)
if err != nil {
log.Fatal(err)
}
-
```
2. To scrape metrics from applications, configure a local running Prometheus instance using a configuration file (`prometheus.yml`).
```yaml
-
scrape_configs:
- job_name: pulsar-client-go-metrics
scrape_interval: 10s
static_configs:
- targets:
- localhost:2112
-
```
Now you can query Pulsar client metrics on Prometheus.
@@ -440,7 +413,6 @@ Now you can query Pulsar client metrics on Prometheus.
Pulsar consumers subscribe to one or more Pulsar topics and listen for incoming messages produced on that topic/those topics. You can [configure](#consumer-configuration) Go consumers using a `ConsumerOptions` object. Here's a basic example that uses channels:
```go
-
consumer, err := client.Subscribe(pulsar.ConsumerOptions{
Topic: "topic-1",
SubscriptionName: "my-sub",
@@ -466,7 +438,6 @@ for i := 0; i < 10; i++ {
if err := consumer.Unsubscribe(); err != nil {
log.Fatal(err)
}
-
```
### Consumer operations
@@ -494,7 +465,6 @@ Method | Description | Return type
#### How to use regex consumer
```go
-
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
@@ -520,13 +490,11 @@ if err != nil {
log.Fatal(err)
}
defer consumer.Close()
-
```
#### How to use multi topics Consumer
```go
-
func newTopicName() string {
return fmt.Sprintf("my-topic-%v", time.Now().Nanosecond())
}
@@ -550,13 +518,11 @@ if err != nil {
log.Fatal(err)
}
defer consumer.Close()
-
```
#### How to use consumer listener
```go
-
import (
"fmt"
"log"
@@ -600,13 +566,11 @@ func main() {
consumer.Ack(msg)
}
}
-
```
#### How to use consumer receive timeout
```go
-
client, err := NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
@@ -635,31 +599,25 @@ fmt.Println(msg.Payload())
if err != nil {
log.Fatal(err)
}
-
```
#### How to use schema in consumer
```go
-
type testJSON struct {
ID int `json:"id"`
Name string `json:"name"`
}
-
```
```go
-
var (
exampleSchemaDef = "{\"type\":\"record\",\"name\":\"Example\",\"namespace\":\"test\"," +
"\"fields\":[{\"name\":\"ID\",\"type\":\"int\"},{\"name\":\"Name\",\"type\":\"string\"}]}"
)
-
```
```go
-
client, err := NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
@@ -686,7 +644,6 @@ if err != nil {
}
defer consumer.Close()
-
```
#### How to use Prometheus metrics in consumer
@@ -695,7 +652,6 @@ In this guide, This section demonstrates how to create a simple Pulsar consumer
1. Write a simple consumer application.
```go
-
// Create a Pulsar client
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
@@ -750,20 +706,17 @@ err = http.ListenAndServe(":"+strconv.Itoa(webPort), nil)
if err != nil {
log.Fatal(err)
}
-
```
2. To scrape metrics from applications, configure a local running Prometheus instance using a configuration file (`prometheus.yml`).
```yaml
-
scrape_configs:
- job_name: pulsar-client-go-metrics
scrape_interval: 10s
static_configs:
- targets:
- localhost:2112
-
```
Now you can query Pulsar client metrics on Prometheus.
@@ -798,7 +751,6 @@ Now you can query Pulsar client metrics on Prometheus.
Pulsar readers process messages from Pulsar topics. Readers are different from consumers because with readers you need to explicitly specify which message in the stream you want to begin with (consumers, on the other hand, automatically begin with the most recent unacked message). You can [configure](#reader-configuration) Go readers using a `ReaderOptions` object. Here's an example:
```go
-
reader, err := client.CreateReader(pulsar.ReaderOptions{
Topic: "topic-1",
StartMessageID: pulsar.EarliestMessageID(),
@@ -807,7 +759,6 @@ if err != nil {
log.Fatal(err)
}
defer reader.Close()
-
```
### Reader operations
@@ -830,7 +781,6 @@ Method | Description | Return type
Here's an example usage of a Go reader that uses the `Next()` method to process incoming messages:
```go
-
import (
"context"
"fmt"
@@ -866,26 +816,22 @@ func main() {
msg.ID(), string(msg.Payload()))
}
}
-
```
In the example above, the reader begins reading from the earliest available message (specified by `pulsar.EarliestMessage`). The reader can also begin reading from the latest message (`pulsar.LatestMessage`) or some other message ID specified by bytes using the `DeserializeMessageID` function, which takes a byte array and returns a `MessageID` object. Here's an example:
```go
-
lastSavedId := // Read last saved message id from external store as byte[]
reader, err := client.CreateReader(pulsar.ReaderOptions{
Topic: "my-golang-topic",
StartMessageID: pulsar.DeserializeMessageID(lastSavedId),
})
-
```
#### How to use reader to read specific message
```go
-
client, err := NewClient(pulsar.ClientOptions{
URL: lookupURL,
})
@@ -948,7 +894,6 @@ if err != nil {
log.Fatal(err)
}
defer readerInclusive.Close()
-
```
### Reader configuration
@@ -962,15 +907,14 @@ defer readerInclusive.Close()
| StartMessageIDInclusive | If true, the reader will start at the `StartMessageID`, included. Default is `false` and the reader will start from the "next" message | false |
| MessageChannel | MessageChannel sets a `MessageChannel` for the consumer When a message is received, it will be pushed to the channel for consumption| |
| ReceiverQueueSize | ReceiverQueueSize sets the size of the consumer receive queue. | 1000 |
-| SubscriptionRolePrefix| SubscriptionRolePrefix set the subscription role prefix. | “reader” |
+| SubscriptionRolePrefix| SubscriptionRolePrefix set the subscription role prefix. | "reader" |
| ReadCompacted | If enabled, the reader will read messages from the compacted topic rather than reading the full message backlog of the topic. ReadCompacted can only be enabled when reading from a persistent topic. | false|
## Messages
-The Pulsar Go client provides a `ProducerMessage` interface that you can use to construct messages to producer on Pulsar topics. Here's an example message:
+The Pulsar Go client provides a `ProducerMessage` interface that you can use to construct messages to producers on Pulsar topics. Here's an example message:
```go
-
msg := pulsar.ProducerMessage{
Payload: []byte("Here is some message data"),
Key: "message-key",
@@ -984,7 +928,6 @@ msg := pulsar.ProducerMessage{
if _, err := producer.send(msg); err != nil {
log.Fatalf("Could not publish message due to: %v", err)
}
-
```
The following methods parameters are available for `ProducerMessage` objects:
@@ -1004,7 +947,7 @@ Parameter | Description
## TLS encryption and authentication
-In order to use [TLS encryption](security-tls-transport.md), you'll need to configure your client to do so:
+To use [TLS encryption](security-tls-transport.md), you need to configure your client to do so:
* Use `pulsar+ssl` URL type
* Set `TLSTrustCertsFilePath` to the path to the TLS certs used by your client and the Pulsar broker
@@ -1013,22 +956,20 @@ In order to use [TLS encryption](security-tls-transport.md), you'll need to conf
Here's an example:
```go
-
opts := pulsar.ClientOptions{
URL: "pulsar+ssl://my-cluster.com:6651",
TLSTrustCertsFilePath: "/path/to/certs/my-cert.csr",
Authentication: NewAuthenticationTLS("my-cert.pem", "my-key.pem"),
}
-
```
## OAuth2 authentication
-To use [OAuth2 authentication](security-oauth2.md), you'll need to configure your client to perform the following operations.
+To use [OAuth2 authentication](security-oauth2.md), you need to configure your client to perform the following operations.
+
This example shows how to configure OAuth2 authentication.
```go
-
oauth := pulsar.NewAuthenticationOAuth2(map[string]string{
"type": "client_credentials",
"issuerUrl": "https://dev-kt-aa9ne.us.auth0.com",
@@ -1040,6 +981,5 @@ client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://my-cluster:6650",
Authentication: oauth,
})
-
```
diff --git a/site2/website-next/docs/client-libraries-java.md b/site2/website-next/docs/client-libraries-java.md
index 3c02e4edcf8..30f88956079 100644
--- a/site2/website-next/docs/client-libraries-java.md
+++ b/site2/website-next/docs/client-libraries-java.md
@@ -40,7 +40,6 @@ The latest version of the Pulsar Java client library is available via [Maven Cen
If you use Maven, add the following information to the `pom.xml` file.
```xml
-
<!-- in your <properties> block -->
<pulsar.version>@pulsar:version@</pulsar.version>
@@ -50,7 +49,6 @@ If you use Maven, add the following information to the `pom.xml` file.
<artifactId>pulsar-client</artifactId>
<version>${pulsar.version}</version>
</dependency>
-
```
### Gradle
@@ -58,13 +56,11 @@ If you use Maven, add the following information to the `pom.xml` file.
If you use Gradle, add the following information to the `build.gradle` file.
```groovy
-
def pulsarVersion = '@pulsar:version@'
dependencies {
compile group: 'org.apache.pulsar', name: 'pulsar-client', version: pulsarVersion
}
-
```
## Connection URLs
@@ -94,21 +90,17 @@ pulsar+ssl://pulsar.us-west.example.com:6651
You can instantiate a {@inject: javadoc:PulsarClient:/client/org/apache/pulsar/client/api/PulsarClient} object using just a URL for the target Pulsar [cluster](reference-terminology.md#cluster) like this:
```java
-
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://localhost:6650")
.build();
-
```
If you have multiple brokers, you can initiate a PulsarClient like this:
```java
-
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://localhost:6650,localhost:6651,localhost:6652")
.build();
-
```
> ### Default broker URLs for standalone clusters
@@ -157,15 +149,13 @@ You can set the client memory allocator configurations through Java properties.<
`pulsar.allocator.leak_detection` | String | The leak detection policy for Pulsar bytebuf allocator. <li> **Disabled**: No leak detection and no overhead. </li> <li> **Simple**: Instruments 1% of the allocated buffer to track for leaks. </li> <li> **Advanced**: Instruments 1% of the allocated buffer to track for leaks, reporting stack traces of places where the buffer is used. </li> <li> **Paranoid**: Instruments 100% of the allocated buffer to track for leaks, reporting stack traces of [...]
`pulsar.allocator.out_of_memory_policy` | String | When an OOM occurs, the client throws an exception or fallbacks to heap | FallbackToHeap | <li> ThrowException </li> <li> FallbackToHeap </li>
-**Example**:
-
-```
-
--Dpulsar.allocator.pooled=true
--Dpulsar.allocator.exit_on_oom=false
--Dpulsar.allocator.leak_detection=Disabled
--Dpulsar.allocator.out_of_memory_policy=ThrowException
+**Example**
+```conf
+Dpulsar.allocator.pooled=true
+Dpulsar.allocator.exit_on_oom=false
+Dpulsar.allocator.leak_detection=Disabled
+Dpulsar.allocator.out_of_memory_policy=ThrowException
```
### Cluster-level failover
@@ -192,10 +182,11 @@ This chapter describes the concept, benefits, use cases, constraints, usage, wor
- [How does cluster-level failover work?](#how-does-cluster-level-failover-work)
-> #### What is cluster-level failover
+#### What is cluster-level failover
This chapter helps you better understand the concept of cluster-level failover.
-> ##### Concept of cluster-level failover
+
+##### Concept of cluster-level failover
````mdx-code-block
<Tabs groupId="failover-choice"
@@ -221,13 +212,13 @@ Controlled cluster-level failover supports Pulsar clients switching from a prima
Once the primary cluster functions again, Pulsar clients can switch back to the primary cluster. Most of the time users won’t even notice a thing. Users can keep using applications and services without interruptions or timeouts.
-> ##### Why use cluster-level failover?
+##### Why use cluster-level failover?
The cluster-level failover provides fault tolerance, continuous availability, and high availability together. It brings a number of benefits, including but not limited to:
* Reduced cost: services can be switched and recovered automatically with no data loss.
-* Simplified management: businesses can operate on an “always-on” basis since no immediate user intervention is required.
+* Simplified management: businesses can operate on an "always-on" basis since no immediate user intervention is required.
* Improved stability and robustness: it ensures continuous performance and minimizes service downtime.
@@ -239,7 +230,7 @@ The cluster-level failover protects your environment in a number of ways, includ
* Planned migration: if you want to migrate production workloads from an old cluster to a new cluster, you can improve the migration efficiency with cluster-level failover. For example, you can test whether the data migration goes smoothly in case of a failover event, identify possible issues and risks before the migration.
-> ##### When cluster-level failover is triggered?
+##### When cluster-level failover is triggered?
````mdx-code-block
<Tabs groupId="failover-choice"
@@ -267,7 +258,7 @@ Controlled cluster-level failover is triggered when administrators set the switc
</Tabs>
````
-> ##### Why does cluster-level failover fail?
+##### Why does cluster-level failover fail?
Obviously, the cluster-level failover does not succeed if the backup cluster is unreachable by active Pulsar clients. This can happen for many reasons, including but not limited to:
@@ -281,11 +272,11 @@ Obviously, the cluster-level failover does not succeed if the backup cluster is
* Fail to authenticate or authorize between 1) primary and backup clusters, or 2) between two backup clusters.
-> ##### What are the limitations of cluster-level failover?
+##### What are the limitations of cluster-level failover?
Currently, cluster-level failover can perform probes to prevent data loss, but it can not check the status of backup clusters. If backup clusters are not healthy, you cannot produce or consume data.
-> #### What are the relationships between cluster-level failover and geo-replication?
+#### What are the relationships between cluster-level failover and geo-replication?
The cluster-level failover is an extension of [geo-replication](concepts-replication.md) to improve stability and robustness. The cluster-level failover depends on geo-replication, and they have some **differences** as below.
@@ -295,7 +286,7 @@ Do administrators have heavy workloads?|No or maybe.<br /><br />- For the **auto
Result in data loss?|No.<br /><br />For both **automatic** and **controlled** cluster-level failover, if the failed primary cluster doesn't replicate messages immediately to the backup cluster, the Pulsar client can't consume the non-replicated messages. After the primary cluster is restored and the Pulsar client switches back, the non-replicated data can still be consumed by the Pulsar client. Consequently, the data is not lost.<br /><br />- For the **automatic** cluster-level failover, [...]
Result in Pulsar client failure? |No or maybe.<br /><br />- For **automatic** cluster-level failover, services can be switched and recovered automatically and the Pulsar client does not fail. <br /><br />- For **controlled** cluster-level failover, services can be switched and recovered manually, but the Pulsar client fails before administrators can take action. |Same as above.
-> #### How to use cluster-level failover
+#### How to use cluster-level failover
This section guides you through every step on how to configure cluster-level failover.
@@ -327,29 +318,26 @@ This section guides you through every step on how to configure cluster-level fai
This is an example of how to construct a Java Pulsar client to use automatic cluster-level failover. The switchover is triggered automatically.
-```
-
- private PulsarClient getAutoFailoverClient() throws PulsarClientException {
-
- ServiceUrlProvider failover = AutoClusterFailover.builder()
- .primary("pulsar://localhost:6650")
- .secondary(Collections.singletonList("pulsar://other1:6650","pulsar://other2:6650"))
- .failoverDelay(30, TimeUnit.SECONDS)
- .switchBackDelay(60, TimeUnit.SECONDS)
- .checkInterval(1000, TimeUnit.MILLISECONDS)
- .secondaryTlsTrustCertsFilePath("/path/to/ca.cert.pem")
- .secondaryAuthentication("org.apache.pulsar.client.impl.auth.AuthenticationTls",
-"tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem")
-
- .build();
-
- PulsarClient pulsarClient = PulsarClient.builder()
- .build();
-
- failover.initialize(pulsarClient);
- return pulsarClient;
- }
-
+```java
+private PulsarClient getAutoFailoverClient() throws PulsarClientException {
+ ServiceUrlProvider failover = AutoClusterFailover.builder()
+ .primary("pulsar://localhost:6650")
+ .secondary(Collections.singletonList("pulsar://other1:6650", "pulsar://other2:6650"))
+ .failoverDelay(30, TimeUnit.SECONDS)
+ .switchBackDelay(60, TimeUnit.SECONDS)
+ .checkInterval(1000, TimeUnit.MILLISECONDS)
+ .secondaryTlsTrustCertsFilePath("/path/to/ca.cert.pem")
+ .secondaryAuthentication("org.apache.pulsar.client.impl.auth.AuthenticationTls",
+ "tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem")
+
+ .build();
+
+ PulsarClient pulsarClient = PulsarClient.builder()
+ .build();
+
+ failover.initialize(pulsarClient);
+ return pulsarClient;
+}
```
Configure the following parameters:
@@ -371,31 +359,29 @@ This is an example of how to construct a Java Pulsar client to use controlled cl
**Note**: you can have one or several backup clusters but can only specify one.
-```
-
- public PulsarClient getControlledFailoverClient() throws IOException {
-Map<String, String> header = new HashMap();
- header.put(“service_user_id”, “my-user”);
- header.put(“service_password”, “tiger”);
- header.put(“clusterA”, “tokenA”);
- header.put(“clusterB”, “tokenB”);
-
- ServiceUrlProvider provider =
- ControlledClusterFailover.builder()
- .defaultServiceUrl("pulsar://localhost:6650")
- .checkInterval(1, TimeUnit.MINUTES)
- .urlProvider("http://localhost:8080/test")
- .urlProviderHeader(header)
- .build();
-
- PulsarClient pulsarClient =
- PulsarClient.builder()
- .build();
-
- provider.initialize(pulsarClient);
- return pulsarClient;
+```java
+public PulsarClient getControlledFailoverClient() throws IOException {
+ Map<String, String> header = new HashMap();
+ header.put("service_user_id", "my-user");
+ header.put("service_password", "tiger");
+ header.put("clusterA", "tokenA");
+ header.put("clusterB", "tokenB");
+
+ ServiceUrlProvider provider =
+ ControlledClusterFailover.builder()
+ .defaultServiceUrl("pulsar://localhost:6650")
+ .checkInterval(1, TimeUnit.MINUTES)
+ .urlProvider("http://localhost:8080/test")
+ .urlProviderHeader(header)
+ .build();
+
+ PulsarClient pulsarClient =
+ PulsarClient.builder()
+ .build();
+
+ provider.initialize(pulsarClient);
+ return pulsarClient;
}
-
```
Parameter|Default value|Required?|Description
@@ -419,8 +405,7 @@ Assume that you want to connect Pulsar client 1 to cluster A.
**Note**: **the credential must be in a JSON file and contain parameters as shown**.
- ```
-
+ ```java
{
"serviceUrl": "pulsar+ssl://target:6651",
"tlsTrustCertsFilePath": "/security/ca.cert.pem",
@@ -428,7 +413,6 @@ Assume that you want to connect Pulsar client 1 to cluster A.
"authParamsString": " \"tlsCertFile\": \"/security/client.cert.pem\"
\"tlsKeyFile\": \"/security/client-pk8.pem\" "
}
-
```
3. Pulsar client 1 connects to cluster A using credential *c1*.
@@ -438,7 +422,7 @@ Assume that you want to connect Pulsar client 1 to cluster A.
</Tabs>
````
->#### How does cluster-level failover work?
+#### How does cluster-level failover work?
This chapter explains the working process of cluster-level failover. For more implementation details, see [PIP-121](https://github.com/apache/pulsar/issues/13315).
@@ -495,51 +479,41 @@ In an automatic failover cluster, the primary cluster and backup cluster are awa
In Pulsar, producers write messages to topics. Once you've instantiated a {@inject: javadoc:PulsarClient:/client/org/apache/pulsar/client/api/PulsarClient} object (as in the section [above](#client-configuration)), you can create a {@inject: javadoc:Producer:/client/org/apache/pulsar/client/api/Producer} for a specific Pulsar [topic](reference-terminology.md#topic).
```java
-
Producer<byte[]> producer = client.newProducer()
.topic("my-topic")
.create();
// You can then send messages to the broker and topic you specified:
producer.send("My message".getBytes());
-
```
By default, producers produce messages that consist of byte arrays. You can produce different types by specifying a message [schema](#schema).
```java
-
Producer<String> stringProducer = client.newProducer(Schema.STRING)
.topic("my-topic")
.create();
stringProducer.send("My message");
-
```
> Make sure that you close your producers, consumers, and clients when you do not need them.
> ```java
->
> producer.close();
> consumer.close();
> client.close();
->
->
> ```
>
> Close operations can also be asynchronous:
> ```java
->
> producer.closeAsync()
> .thenRun(() -> System.out.println("Producer closed"))
> .exceptionally((ex) -> {
> System.err.println("Failed to close producer: " + ex);
> return null;
> });
->
->
> ```
@@ -572,14 +546,12 @@ You can configure parameters if you do not want to use the default configuration
For a full list, see the Javadoc for the {@inject: javadoc:ProducerBuilder:/client/org/apache/pulsar/client/api/ProducerBuilder} class. The following is an example.
```java
-
Producer<byte[]> producer = client.newProducer()
.topic("my-topic")
.batchingMaxPublishDelay(10, TimeUnit.MILLISECONDS)
.sendTimeout(10, TimeUnit.SECONDS)
.blockIfQueueFull(true)
.create();
-
```
### Publish to partitioned topics
@@ -682,11 +654,9 @@ You can publish messages [asynchronously](concepts-messaging.md#send-modes) usin
The following is an example.
```java
-
producer.sendAsync("my-async-message".getBytes()).thenAccept(msgId -> {
System.out.println("Message with ID " + msgId + " successfully sent");
});
-
```
As you can see from the example above, async send operations return a {@inject: javadoc:MessageId:/client/org/apache/pulsar/client/api/MessageId} wrapped in a [`CompletableFuture`](http://www.baeldung.com/java-completablefuture).
@@ -696,14 +666,12 @@ As you can see from the example above, async send operations return a {@inject:
In addition to a value, you can set additional items on a given message:
```java
-
producer.newMessage()
.key("my-message-key")
.value("my-async-message".getBytes())
.property("my-key", "my-value")
.property("my-other-key", "my-other-value")
.send();
-
```
You can terminate the builder chain with `sendAsync()` and get a future return.
@@ -715,17 +683,20 @@ Message [chunking](concepts-messaging.md#chunking) enables Pulsar to process lar
The message chunking feature is OFF by default. The following is an example of how to enable message chunking when creating a producer.
```java
-
Producer<byte[]> producer = client.newProducer()
.topic(topic)
.enableChunking(true)
.enableBatching(false)
.create();
-
```
By default, producer chunks the large message based on max message size (`maxMessageSize`) configured at broker (eg: 5MB). However, client can also configure max chunked size using producer configuration `chunkMaxMessageSize`.
-> **Note:** To enable chunking, you need to disable batching (`enableBatching`=`false`) concurrently.
+
+:::note
+
+To enable chunking, you need to disable batching (`enableBatching`=`false`) concurrently.
+
+:::
### Intercept messages
@@ -775,18 +746,15 @@ In Pulsar, consumers subscribe to topics and handle messages that producers publ
Once you've instantiated a {@inject: javadoc:PulsarClient:/client/org/apache/pulsar/client/api/PulsarClient} object, you can create a {@inject: javadoc:Consumer:/client/org/apache/pulsar/client/api/Consumer} by specifying a [topic](reference-terminology.md#topic) and a [subscription](concepts-messaging.md#subscription-types).
```java
-
Consumer consumer = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
.subscribe();
-
```
-The `subscribe` method will auto-subscribe the consumer to the specified topic and subscription. One way to make the consumer listen on the topic is to set up a `while` loop. In this example loop, the consumer listens for messages, prints the contents of any received message, and then [acknowledges](reference-terminology.md#acknowledgment-ack) that the message has been processed. If the processing logic fails, you can use [negative acknowledgement](reference-terminology.md#acknowledgment [...]
+The `subscribe` method will auto-subscribe the consumer to the specified topic and subscription. One way to make the consumer listen to the topic is to set up a `while` loop. In this example loop, the consumer listens for messages, prints the contents of any received message, and then [acknowledges](reference-terminology.md#acknowledgment-ack) that the message has been processed. If the processing logic fails, you can use [negative acknowledgment](reference-terminology.md#acknowledgment- [...]
```java
-
while (true) {
// Wait for a message
Message msg = consumer.receive();
@@ -802,13 +770,11 @@ while (true) {
consumer.negativeAcknowledge(msg);
}
}
-
```
If you don't want to block your main thread and rather listen constantly for new messages, consider using a `MessageListener`.
```java
-
MessageListener myMessageListener = (consumer, msg) -> {
try {
System.out.println("Message received: " + new String(msg.getData()));
@@ -823,7 +789,6 @@ Consumer consumer = client.newConsumer()
.subscriptionName("my-subscription")
.messageListener(myMessageListener)
.subscribe();
-
```
### Configure consumer
@@ -854,7 +819,7 @@ When you create a consumer, you can use the `loadConf` configuration. The follow
`regexSubscriptionMode`|RegexSubscriptionMode|When subscribing to a topic using a regular expression, you can pick a certain type of topics.<br /><br /><li>**PersistentOnly**: only subscribe to persistent topics.</li><li>**NonPersistentOnly**: only subscribe to non-persistent topics.</li><li>**AllTopics**: subscribe to both persistent and non-persistent topics.</li>|RegexSubscriptionMode.PersistentOnly
`deadLetterPolicy`|DeadLetterPolicy|Dead letter policy for consumers.<br /><br />By default, some messages are probably redelivered many times, even to the extent that it never stops.<br /><br />By using the dead letter mechanism, messages have the max redelivery count. **When exceeding the maximum number of redeliveries, messages are sent to the Dead Letter Topic and acknowledged automatically**.<br /><br />You can enable the dead letter mechanism by setting `deadLetterPolicy`.<br /><br [...]
`autoUpdatePartitions`|boolean|If `autoUpdatePartitions` is enabled, a consumer subscribes to partition increasement automatically.<br /><br />**Note**: this is only for partitioned consumers.|true
-`replicateSubscriptionState`|boolean|If `replicateSubscriptionState` is enabled, a subscription state is replicated to geo-replicated clusters.|false
+`replicateSubscriptionState`|boolean|If `replicateSubscriptionState` is enabled, a subscription state is replicated to geo-replicated clusters.|false
`negativeAckRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is negativeAcked policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
`ackTimeoutRedeliveryBackoff`|RedeliveryBackoff|Interface for custom message is ackTimeout policy. You can specify `RedeliveryBackoff` for a consumer.| `MultiplierRedeliveryBackoff`
`autoAckOldestChunkedMessageOnQueueFull`|boolean|Whether to automatically acknowledge pending chunked messages when the threashold of `maxPendingChunkedMessage` is reached. If set to `false`, these messages will be redelivered by their broker. |true
@@ -867,14 +832,12 @@ You can configure parameters if you do not want to use the default configuration
The following is an example.
```java
-
Consumer consumer = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
.ackTimeout(10, TimeUnit.SECONDS)
.subscriptionType(SubscriptionType.Exclusive)
.subscribe();
-
```
### Async receive
@@ -884,9 +847,7 @@ The `receive` method receives messages synchronously (the consumer process is bl
The following is an example.
```java
-
CompletableFuture<Message> asyncMessage = consumer.receiveAsync();
-
```
Async receive operations return a {@inject: javadoc:Message:/client/org/apache/pulsar/client/api/Message} wrapped inside of a [`CompletableFuture`](http://www.baeldung.com/java-completablefuture).
@@ -898,22 +859,19 @@ Use `batchReceive` to receive multiple messages for each call.
The following is an example.
```java
-
Messages messages = consumer.batchReceive();
for (Object message : messages) {
// do something
}
consumer.acknowledge(messages)
-
```
:::note
Batch receive policy limits the number and bytes of messages in a single batch. You can specify a timeout to wait for enough messages.
-The batch receive is completed if any of the following conditions is met: enough number of messages, bytes of messages, wait timeout.
+The batch receive is completed if any of the following conditions are met: enough number of messages, bytes of messages, wait timeout.
```java
-
Consumer consumer = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
@@ -923,19 +881,16 @@ Consumer consumer = client.newConsumer()
.timeout(200, TimeUnit.MILLISECONDS)
.build())
.subscribe();
-
```
The default batch receive policy is:
```java
-
BatchReceivePolicy.builder()
.maxNumMessage(-1)
.maxNumBytes(10 * 1024 * 1024)
.timeout(100, TimeUnit.MILLISECONDS)
.build();
-
```
:::
@@ -947,7 +902,6 @@ You can limit the maximum number of chunked messages a consumer maintains concur
The following is an example of how to configure message chunking.
```java
-
Consumer<byte[]> consumer = client.newConsumer()
.topic(topic)
.subscriptionName("test")
@@ -955,7 +909,6 @@ Consumer<byte[]> consumer = client.newConsumer()
.maxPendingChunkedMessage(100)
.expireTimeOfIncompleteChunkedMessage(10, TimeUnit.MINUTES)
.subscribe();
-
```
### Negative acknowledgment redelivery backoff
@@ -963,7 +916,6 @@ Consumer<byte[]> consumer = client.newConsumer()
The `RedeliveryBackoff` introduces a redelivery backoff mechanism. You can achieve redelivery with different delays by setting `redeliveryCount ` of messages.
```java
-
Consumer consumer = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
@@ -972,16 +924,13 @@ Consumer consumer = client.newConsumer()
.maxDelayMs(60 * 1000)
.build())
.subscribe();
-
```
-### Acknowledgement timeout redelivery backoff
+### Acknowledgment timeout redelivery backoff
-The `RedeliveryBackoff` introduces a redelivery backoff mechanism. You can redeliver messages with different delays by setting the number
-of times the messages is retried.
+The `RedeliveryBackoff` introduces a redelivery backoff mechanism. You can redeliver messages with different delays by setting the number of times the messages are retried.
```java
-
Consumer consumer = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
@@ -992,7 +941,6 @@ Consumer consumer = client.newConsumer()
.multiplier(2)
.build())
.subscribe();
-
```
The message redelivery behavior should be as follows.
@@ -1022,7 +970,6 @@ In addition to subscribing a consumer to a single Pulsar topic, you can also sub
The followings are some examples.
```java
-
import org.apache.pulsar.client.api.Consumer;
import org.apache.pulsar.client.api.PulsarClient;
@@ -1044,20 +991,17 @@ Pattern someTopicsInNamespace = Pattern.compile("public/default/foo.*");
Consumer allTopicsConsumer = consumerBuilder
.topicsPattern(someTopicsInNamespace)
.subscribe();
-
```
In the above example, the consumer subscribes to the `persistent` topics that can match the topic name pattern. If you want the consumer subscribes to all `persistent` and `non-persistent` topics that can match the topic name pattern, set `subscriptionTopicsMode` to `RegexSubscriptionMode.AllTopics`.
```java
-
Pattern pattern = Pattern.compile("public/default/.*");
pulsarClient.newConsumer()
.subscriptionName("my-sub")
.topicsPattern(pattern)
.subscriptionTopicsMode(RegexSubscriptionMode.AllTopics)
.subscribe();
-
```
:::note
@@ -1069,7 +1013,6 @@ By default, the `subscriptionTopicsMode` of the consumer is `PersistentOnly`. Av
You can also subscribe to an explicit list of topics (across namespaces if you wish):
```java
-
List<String> topics = Arrays.asList(
"topic-1",
"topic-2",
@@ -1088,13 +1031,11 @@ Consumer multiTopicConsumer = consumerBuilder
"topic-3"
)
.subscribe();
-
```
You can also subscribe to multiple topics asynchronously using the `subscribeAsync` method rather than the synchronous `subscribe` method. The following is an example.
```java
-
Pattern allTopicsInNamespace = Pattern.compile("persistent://public/default.*");
consumerBuilder
.topics(topics)
@@ -1107,21 +1048,19 @@ private void receiveMessageFromConsumer(Object consumer) {
receiveMessageFromConsumer(consumer);
});
}
-
```
### Subscription types
Pulsar has various [subscription types](concepts-messaging#subscription-types) to match different scenarios. A topic can have multiple subscriptions with different subscription types. However, a subscription can only have one subscription type at a time.
-A subscription is identical with the subscription name; a subscription name can specify only one subscription type at a time. To change the subscription type, you should first stop all consumers of this subscription.
+A subscription is identical to the subscription name; a subscription name can specify only one subscription type at a time. To change the subscription type, you should first stop all consumers of this subscription.
Different subscription types have different message distribution types. This section describes the differences between subscription types and how to use them.
-In order to better describe their differences, assuming you have a topic named "my-topic", and the producer has published 10 messages.
+To better describe their differences, assume you have a topic named "my-topic", and the producer has published 10 messages.
```java
-
Producer<String> producer = client.newProducer(Schema.STRING)
.topic("my-topic")
.enableBatching(false)
@@ -1137,7 +1076,6 @@ producer.newMessage().key("key-3").value("message-3-1").send();
producer.newMessage().key("key-3").value("message-3-2").send();
producer.newMessage().key("key-4").value("message-4-1").send();
producer.newMessage().key("key-4").value("message-4-2").send();
-
```
#### Exclusive
@@ -1145,13 +1083,11 @@ producer.newMessage().key("key-4").value("message-4-2").send();
Create a new consumer and subscribe with the `Exclusive` subscription type.
```java
-
Consumer consumer = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
.subscriptionType(SubscriptionType.Exclusive)
.subscribe()
-
```
Only the first consumer is allowed to the subscription, other consumers receive an error. The first consumer receives all 10 messages, and the consuming order is the same as the producing order.
@@ -1167,7 +1103,6 @@ If topic is a partitioned topic, the first consumer subscribes to all partitione
Create new consumers and subscribe with the`Failover` subscription type.
```java
-
Consumer consumer1 = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
@@ -1180,33 +1115,28 @@ Consumer consumer2 = client.newConsumer()
.subscribe()
//conumser1 is the active consumer, consumer2 is the standby consumer.
//consumer1 receives 5 messages and then crashes, consumer2 takes over as an active consumer.
-
```
-Multiple consumers can attach to the same subscription, yet only the first consumer is active, and others are standby. When the active consumer is disconnected, messages will be dispatched to one of standby consumers, and the standby consumer then becomes active consumer.
+Multiple consumers can attach to the same subscription, yet only the first consumer is active, and others are standby. When the active consumer is disconnected, messages will be dispatched to one of standby consumers, and the standby consumer then becomes the active consumer.
If the first active consumer is disconnected after receiving 5 messages, the standby consumer becomes active consumer. Consumer1 will receive:
```
-
("key-1", "message-1-1")
("key-1", "message-1-2")
("key-1", "message-1-3")
("key-2", "message-2-1")
("key-2", "message-2-2")
-
```
consumer2 will receive:
```
-
("key-2", "message-2-3")
("key-3", "message-3-1")
("key-3", "message-3-2")
("key-4", "message-4-1")
("key-4", "message-4-2")
-
```
:::note
@@ -1220,7 +1150,6 @@ If a topic is a partitioned topic, each partition has only one active consumer,
Create new consumers and subscribe with `Shared` subscription type.
```java
-
Consumer consumer1 = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
@@ -1233,7 +1162,6 @@ Consumer consumer2 = client.newConsumer()
.subscriptionType(SubscriptionType.Shared)
.subscribe()
//Both consumer1 and consumer2 are active consumers.
-
```
In Shared subscription type, multiple consumers can attach to the same subscription and messages are delivered in a round-robin distribution across consumers.
@@ -1241,35 +1169,30 @@ In Shared subscription type, multiple consumers can attach to the same subscript
If a broker dispatches only one message at a time, consumer1 receives the following information.
```
-
("key-1", "message-1-1")
("key-1", "message-1-3")
("key-2", "message-2-2")
("key-3", "message-3-1")
("key-4", "message-4-1")
-
```
consumer2 receives the following information.
```
-
("key-1", "message-1-2")
("key-2", "message-2-1")
("key-2", "message-2-3")
("key-3", "message-3-2")
("key-4", "message-4-2")
-
```
-`Shared` subscription is different from `Exclusive` and `Failover` subscription types. `Shared` subscription has better flexibility, but cannot provide order guarantee.
+The `Shared` subscription is different from the `Exclusive` and `Failover` subscription types. `Shared` subscription has better flexibility, but cannot provide an ordering guarantee.
#### Key_shared
This is a new subscription type since 2.4.0 release. Create new consumers and subscribe with `Key_Shared` subscription type.
```java
-
Consumer consumer1 = client.newConsumer()
.topic("my-topic")
.subscriptionName("my-subscription")
@@ -1282,60 +1205,51 @@ Consumer consumer2 = client.newConsumer()
.subscriptionType(SubscriptionType.Key_Shared)
.subscribe()
//Both consumer1 and consumer2 are active consumers.
-
```
-Just like in `Shared` subscription, all consumers in `Key_Shared` subscription type can attach to the same subscription. But `Key_Shared` subscription type is different from the `Shared` subscription. In `Key_Shared` subscription type, messages with the same key are delivered to only one consumer in order. The possible distribution of messages between different consumers (by default we do not know in advance which keys will be assigned to a consumer, but a key will only be assigned to a [...]
+Just like in the `Shared` subscription, all consumers in the `Key_Shared` subscription type can attach to the same subscription. But the `Key_Shared` subscription type is different from the `Shared` subscription. In the `Key_Shared` subscription type, messages with the same key are delivered to only one consumer in order. The possible distribution of messages between different consumers (by default we do not know in advance which keys will be assigned to a consumer, but a key will only b [...]
consumer1 receives the following information.
```
-
("key-1", "message-1-1")
("key-1", "message-1-2")
("key-1", "message-1-3")
("key-3", "message-3-1")
("key-3", "message-3-2")
-
```
consumer2 receives the following information.
```
-
("key-2", "message-2-1")
("key-2", "message-2-2")
("key-2", "message-2-3")
("key-4", "message-4-1")
("key-4", "message-4-2")
-
```
If batching is enabled at the producer side, messages with different keys are added to a batch by default. The broker will dispatch the batch to the consumer, so the default batch mechanism may break the Key_Shared subscription guaranteed message distribution semantics. The producer needs to use the `KeyBasedBatcher`.
```java
-
Producer producer = client.newProducer()
.topic("my-topic")
.batcherBuilder(BatcherBuilder.KEY_BASED)
.create();
-
```
Or the producer can disable batching.
```java
-
Producer producer = client.newProducer()
.topic("my-topic")
.enableBatching(false)
.create();
-
```
:::note
-If the message key is not specified, messages without key are dispatched to one consumer in order by default.
+If the message key is not specified, messages without keys are dispatched to one consumer in order by default.
:::
@@ -1401,12 +1315,11 @@ If you are using multiple interceptors, they apply in the order they are passed
## Reader
-With the [reader interface](concepts-clients.md#reader-interface), Pulsar clients can "manually position" themselves within a topic and reading all messages from a specified message onward. The Pulsar API for Java enables you to create {@inject: javadoc:Reader:/client/org/apache/pulsar/client/api/Reader} objects by specifying a topic and a {@inject: javadoc:MessageId:/client/org/apache/pulsar/client/api/MessageId}.
+With the [reader interface](concepts-clients.md#reader-interface), Pulsar clients can "manually position" themselves within a topic and read all messages from a specified message onward. The Pulsar API for Java enables you to create {@inject: javadoc:Reader:/client/org/apache/pulsar/client/api/Reader} objects by specifying a topic and a {@inject: javadoc:MessageId:/client/org/apache/pulsar/client/api/MessageId}.
The following is an example.
```java
-
byte[] msgIdBytes = // Some message ID byte array
MessageId id = MessageId.fromByteArray(msgIdBytes);
Reader reader = pulsarClient.newReader()
@@ -1418,7 +1331,6 @@ while (true) {
Message message = reader.readNext();
// Process message
}
-
```
In the example above, a `Reader` object is instantiated for a specific topic and message (by ID); the reader iterates over each message in the topic after the message is identified by `msgIdBytes` (how that value is obtained depends on the application).
@@ -1443,31 +1355,28 @@ When you create a reader, you can use the `loadConf` configuration. The followin
### Sticky key range reader
-In sticky key range reader, broker will only dispatch messages which hash of the message key contains by the specified key hash range. Multiple key hash ranges can be specified on a reader.
+In a sticky key range reader, broker only dispatches messages which hash of the message key contains by the specified key hash range. Multiple key hash ranges can be specified on a reader.
The following is an example to create a sticky key range reader.
```java
-
pulsarClient.newReader()
.topic(topic)
.startMessageId(MessageId.earliest)
.keyHashRange(Range.of(0, 10000), Range.of(20001, 30000))
.create();
-
```
-Total hash range size is 65536, so the max end of the range should be less than or equal to 65535.
+The total hash range size is 65536, so the max end of the range should be less than or equal to 65535.
### Configure chunking
-Configuring chuncking for readers is similar to that for consumers. See [configure chunking for consumers](#configure-chunking) for more information.
+Configuring chunking for readers is similar to that for consumers. See [configure chunking for consumers](#configure-chunking) for more information.
The following is an example of how to configure message chunking for a reader.
```java
-
Reader<byte[]> reader = pulsarClient.newReader()
.topic(topicName)
.startMessageId(MessageId.earliest)
@@ -1475,7 +1384,6 @@ Reader<byte[]> reader = pulsarClient.newReader()
.autoAckOldestChunkedMessageOnQueueFull(true)
.expireTimeOfIncompleteChunkedMessage(12, TimeUnit.MILLISECONDS)
.create();
-
```
### Create reader with interceptor
@@ -1491,10 +1399,9 @@ Pulsar reader interceptor works on top of Pulsar consumer interceptor. The plugi
To perceive triggered events and perform customized processing, you can add `ReaderInterceptor` when creating a `Reader` as follows.
```java
-
PulsarClient pulsarClient = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Reader<byte[]> reader = pulsarClient.newReader()
- .topic(“t1”)
+ .topic("t1")
.autoUpdatePartitionsInterval(5, TimeUnit.SECONDS)
.intercept(new ReaderInterceptor<byte[]>() {
@Override
@@ -1514,7 +1421,6 @@ Reader<byte[]> reader = pulsarClient.newReader()
})
.startMessageId(MessageId.earliest)
.create();
-
```
## TableView
@@ -1537,11 +1443,9 @@ The following figure illustrates the dynamic construction of a TableView updated
The following is an example of how to configure a TableView.
```java
-
TableView<String> tv = client.newTableViewBuilder(Schema.STRING)
.topic("my-tableview")
.create()
-
```
You can use the available parameters in the `loadConf` configuration or related [API](/api/client/2.10.0-SNAPSHOT/org/apache/pulsar/client/api/TableViewBuilder.html) to customize your TableView.
@@ -1558,13 +1462,11 @@ You can register listeners for both existing messages on a topic and new message
The following is an example of how to register listeners with TableView.
```java
-
// Register listeners for all existing and incoming messages
tv.forEachAndListen((key, value) -> /*operations on all existing and incoming messages*/)
// Register action for all existing messages
tv.forEach((key, value) -> /*operations on all existing messages*/)
-
```
## Schema
@@ -1572,11 +1474,9 @@ tv.forEach((key, value) -> /*operations on all existing messages*/)
In Pulsar, all message data consists of byte arrays "under the hood." [Message schemas](schema-get-started.md) enable you to use other types of data when constructing and handling messages (from simple types like strings to more complex, application-specific types). If you construct, say, a [producer](#producer) without specifying a schema, then the producer can only produce messages of type `byte[]`. The following is an example.
```java
-
Producer<byte[]> producer = client.newProducer()
.topic(topic)
.create();
-
```
The producer above is equivalent to a `Producer<byte[]>` (in fact, you should *always* explicitly specify the type). If you'd like to use a producer for a different type of data, you'll need to specify a **schema** that informs Pulsar which data type will be transmitted over the [topic](reference-terminology.md#topic).
@@ -1586,7 +1486,6 @@ The producer above is equivalent to a `Producer<byte[]>` (in fact, you should *a
Let's say that you have a `SensorReading` class that you'd like to transmit over a Pulsar topic:
```java
-
public class SensorReading {
public float temperature;
@@ -1606,17 +1505,14 @@ public class SensorReading {
this.temperature = temperature;
}
}
-
```
You could then create a `Producer<SensorReading>` (or `Consumer<SensorReading>`) like this:
```java
-
Producer<SensorReading> producer = client.newProducer(JSONSchema.of(SensorReading.class))
.topic("sensor-readings")
.create();
-
```
The following schema formats are currently available for Java:
@@ -1624,66 +1520,54 @@ The following schema formats are currently available for Java:
* No schema or the byte array schema (which can be applied using `Schema.BYTES`):
```java
-
Producer<byte[]> bytesProducer = client.newProducer(Schema.BYTES)
.topic("some-raw-bytes-topic")
.create();
-
```
Or, equivalently:
```java
-
Producer<byte[]> bytesProducer = client.newProducer()
.topic("some-raw-bytes-topic")
.create();
-
```
* `String` for normal UTF-8-encoded string data. Apply the schema using `Schema.STRING`:
```java
-
Producer<String> stringProducer = client.newProducer(Schema.STRING)
.topic("some-string-topic")
.create();
-
```
* Create JSON schemas for POJOs using `Schema.JSON`. The following is an example.
```java
-
Producer<MyPojo> pojoProducer = client.newProducer(Schema.JSON(MyPojo.class))
.topic("some-pojo-topic")
.create();
-
```
* Generate Protobuf schemas using `Schema.PROTOBUF`. The following example shows how to create the Protobuf schema and use it to instantiate a new producer:
```java
-
Producer<MyProtobuf> protobufProducer = client.newProducer(Schema.PROTOBUF(MyProtobuf.class))
.topic("some-protobuf-topic")
.create();
-
```
* Define Avro schemas with `Schema.AVRO`. The following code snippet demonstrates how to create and use Avro schema.
```java
-
Producer<MyAvro> avroProducer = client.newProducer(Schema.AVRO(MyAvro.class))
.topic("some-avro-topic")
.create();
-
```
### ProtobufNativeSchema example
-For example of ProtobufNativeSchema, see [`SchemaDefinition` in `Complex type`](schema-understand.md#complex-type).
+For examples of ProtobufNativeSchema, see [`SchemaDefinition` in `Complex type`](schema-understand.md#complex-type).
## Authentication
@@ -1696,7 +1580,6 @@ To use [TLS](security-tls-authentication.md), `enableTls` method is deprecated a
The following is an example.
```java
-
Map<String, String> authParams = new HashMap();
authParams.put("tlsCertFile", "/path/to/client-cert.pem");
authParams.put("tlsKeyFile", "/path/to/client-key.pem");
@@ -1709,7 +1592,6 @@ PulsarClient client = PulsarClient.builder()
.tlsTrustCertsFilePath("/path/to/cacert.pem")
.authentication(tlsAuth)
.build();
-
```
### Athenz
@@ -1724,7 +1606,6 @@ To use [Athenz](security-athenz.md) as an authentication provider, you need to [
You can also set an optional `keyId`. The following is an example.
```java
-
Map<String, String> authParams = new HashMap();
authParams.put("tenantDomain", "shopping"); // Tenant domain name
authParams.put("tenantService", "some_app"); // Tenant service name
@@ -1740,14 +1621,13 @@ PulsarClient client = PulsarClient.builder()
.tlsTrustCertsFilePath("/path/to/cacert.pem")
.authentication(athenzAuth)
.build();
-
```
-> #### Supported pattern formats
-> The `privateKey` parameter supports the following three pattern formats:
-> * `file:///path/to/file`
-> * `file:/path/to/file`
-> * `data:application/x-pem-file;base64,<base64-encoded value>`
+#### Supported pattern formats
+The `privateKey` parameter supports the following three pattern formats:
+* `file:///path/to/file`
+* `file:/path/to/file`
+* `data:application/x-pem-file;base64,<base64-encoded value>`
### Oauth2
@@ -1756,25 +1636,21 @@ The following example shows how to use [Oauth2](security-oauth2.md) as an authen
You can use the factory method to configure authentication for Pulsar Java client.
```java
-
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://broker.example.com:6650/")
.authentication(
AuthenticationFactoryOAuth2.clientCredentials(this.issuerUrl, this.credentialsUrl, this.audience))
.build();
-
```
In addition, you can also use the encoded parameters to configure authentication for Pulsar Java client.
```java
-
Authentication auth = AuthenticationFactory
.create(AuthenticationOAuth2.class.getName(), "{"type":"client_credentials","privateKey":"...","issuerUrl":"...","audience":"..."}");
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://broker.example.com:6650/")
.authentication(auth)
.build();
-
```
diff --git a/site2/website-next/docs/client-libraries-node.md b/site2/website-next/docs/client-libraries-node.md
index fd31e1c3f06..ab38c04ee2f 100644
--- a/site2/website-next/docs/client-libraries-node.md
+++ b/site2/website-next/docs/client-libraries-node.md
@@ -35,9 +35,7 @@ If an incompatible version of the C++ client is installed, you may fail to build
Install the `pulsar-client` library via [npm](https://www.npmjs.com/):
```shell
-
-$ npm install pulsar-client
-
+npm install pulsar-client
```
:::note
@@ -70,12 +68,11 @@ pulsar+ssl://pulsar.us-west.example.com:6651
## Create a client
-In order to interact with Pulsar, you first need a client object. You can create a client instance using a `new` operator and the `Client` method, passing in a client options object (more on configuration [below](#client-configuration)).
+To interact with Pulsar, you first need a client object. You can create a client instance using a `new` operator and the `Client` method, passing in a client options object (more on configuration [below](#client-configuration)).
Here is an example:
```javascript
-
const Pulsar = require('pulsar-client');
(async () => {
@@ -85,7 +82,6 @@ const Pulsar = require('pulsar-client');
await client.close();
})();
-
```
### Client configuration
@@ -113,7 +109,6 @@ Pulsar producers publish messages to Pulsar topics. You can [configure](#produce
Here is an example:
```javascript
-
const producer = await client.createProducer({
topic: 'my-topic', // or 'my-tenant/my-namespace/my-topic' to specify topic's tenant and namespace
});
@@ -123,12 +118,11 @@ await producer.send({
});
await producer.close();
-
```
> #### Promise operation
-> When you create a new Pulsar producer, the operation returns `Promise` object and get producer instance or an error through executor function.
-> In this example, using await operator instead of executor function.
+> When you create a new Pulsar producer, the operation returns `Promise` object and get producer instance or an error through executor function.
+> In this example, use await operator instead of executor function.
### Producer operations
@@ -166,7 +160,6 @@ Pulsar Node.js producers have the following methods available:
This example creates a Node.js producer for the `my-topic` topic and sends 10 messages to that topic:
```javascript
-
const Pulsar = require('pulsar-client');
(async () => {
@@ -193,7 +186,6 @@ const Pulsar = require('pulsar-client');
await producer.close();
await client.close();
})();
-
```
## Consumers
@@ -203,7 +195,6 @@ Pulsar consumers subscribe to one or more Pulsar topics and listen for incoming
Here is an example:
```javascript
-
const consumer = await client.subscribe({
topic: 'my-topic',
subscription: 'my-subscription',
@@ -214,12 +205,11 @@ console.log(msg.getData().toString());
consumer.acknowledge(msg);
await consumer.close();
-
```
> #### Promise operation
-> When you create a new Pulsar consumer, the operation returns `Promise` object and get consumer instance or an error through executor function.
-> In this example, using await operator instead of executor function.
+> When you create a new Pulsar consumer, the operation returns `Promise` object and get consumer instance or an error through executor function.
+> In this example, use await operator instead of executor function.
### Consumer operations
@@ -262,7 +252,6 @@ Pulsar Node.js consumers have the following methods available:
This example creates a Node.js consumer with the `my-subscription` subscription on the `my-topic` topic, receives messages, prints the content that arrive, and acknowledges each message to the Pulsar broker for 10 times:
```javascript
-
const Pulsar = require('pulsar-client');
(async () => {
@@ -288,13 +277,11 @@ const Pulsar = require('pulsar-client');
await consumer.close();
await client.close();
})();
-
```
-Instead a consumer can be created with `listener` to process messages.
+Instead, a consumer can be created with `listener` to process messages.
```javascript
-
// Create a consumer
const consumer = await client.subscribe({
topic: 'my-topic',
@@ -305,14 +292,13 @@ const consumer = await client.subscribe({
msgConsumer.acknowledge(msg);
},
});
-
```
:::note
Pulsar Node.js client uses [AsyncWorker](https://github.com/nodejs/node-addon-api/blob/main/doc/async_worker). Asynchronous operations such as creating consumers/producers and receiving/sending messages are performed in worker threads.
Until completion of these operations, worker threads are blocked.
-Since there are only 4 worker threads by default, a called method may never complete.
+Since there are only 4 worker threads by default, a called method may never be complete.
To avoid this situation, you can set `UV_THREADPOOL_SIZE` to increase the number of worker threads, or define `listener` instead of calling `receive()` many times.
:::
@@ -324,7 +310,6 @@ Pulsar readers process messages from Pulsar topics. Readers are different from c
Here is an example:
```javascript
-
const reader = await client.createReader({
topic: 'my-topic',
startMessageId: Pulsar.MessageId.earliest(),
@@ -334,7 +319,6 @@ const msg = await reader.readNext();
console.log(msg.getData().toString());
await reader.close();
-
```
### Reader operations
@@ -365,7 +349,6 @@ Pulsar Node.js readers have the following methods available:
This example creates a Node.js reader with the `my-topic` topic, reads messages, and prints the content that arrive for 10 times:
```javascript
-
const Pulsar = require('pulsar-client');
(async () => {
@@ -390,17 +373,15 @@ const Pulsar = require('pulsar-client');
await reader.close();
await client.close();
})();
-
```
## Messages
-In Pulsar Node.js client, you have to construct producer message object for producer.
+In Pulsar Node.js client, you have to construct producer message objects for producers.
-Here is an example message:
+Here is an example of a message:
```javascript
-
const msg = {
data: Buffer.from('Hello, Pulsar'),
partitionKey: 'key1',
@@ -415,7 +396,6 @@ const msg = {
}
await producer.send(msg);
-
```
The following keys are available for producer message objects:
@@ -433,9 +413,9 @@ The following keys are available for producer message objects:
### Message object operations
-In Pulsar Node.js client, you can receive (or read) message object as consumer (or reader).
+In Pulsar Node.js client, you can receive (or read) message objects as consumers (or readers).
-The message object have the following methods available:
+The message object has the following methods available:
| Method | Description | Return type |
| :----- | :---------- | :---------- |
@@ -450,16 +430,16 @@ The message object have the following methods available:
### Message ID object operations
-In Pulsar Node.js client, you can get message id object from message object.
+In Pulsar Node.js client, you can get message id objects from message objects.
-The message id object have the following methods available:
+The message id object has the following methods available:
| Method | Description | Return type |
| :----- | :---------- | :---------- |
| `serialize()` | Serialize the message id into a Buffer for storing. | `Buffer` |
| `toString()` | Get message id as String. | `String` |
-The client has static method of message id object. You can access it as `Pulsar.MessageId.someStaticMethod` too.
+The client has a static method of message id object. You can access it as `Pulsar.MessageId.someStaticMethod`.
The following static methods are available for the message id object:
@@ -475,13 +455,11 @@ The following static methods are available for the message id object:
### Configuration
-If you want to use the end-to-end encryption feature in the Node.js client, you need to configure `publicKeyPath` for producer and `privateKeyPath` for consumer.
-
-```
+If you want to use the end-to-end encryption feature in the Node.js client, you need to configure `publicKeyPath` for producer and `privateKeyPath` for consumers.
+```conf
publicKeyPath: "./public.pem"
privateKeyPath: "./private.pem"
-
```
### Tutorial
@@ -499,10 +477,8 @@ This section provides step-by-step instructions on how to use the end-to-end enc
**Input**
```shell
-
openssl genrsa -out private.pem 2048
openssl rsa -in private.pem -pubout -out public.pem
-
```
2. Create a producer to send encrypted messages.
@@ -510,7 +486,6 @@ This section provides step-by-step instructions on how to use the end-to-end enc
**Input**
```javascript
-
const Pulsar = require('pulsar-client');
(async () => {
@@ -543,7 +518,6 @@ This section provides step-by-step instructions on how to use the end-to-end enc
await producer.close();
await client.close();
})();
-
```
3. Create a consumer to receive encrypted messages.
@@ -551,7 +525,6 @@ This section provides step-by-step instructions on how to use the end-to-end enc
**Input**
```javascript
-
const Pulsar = require('pulsar-client');
(async () => {
@@ -581,7 +554,6 @@ This section provides step-by-step instructions on how to use the end-to-end enc
await consumer.close();
await client.close();
})();
-
```
4. Run the consumer to receive encrypted messages.
@@ -589,9 +561,7 @@ This section provides step-by-step instructions on how to use the end-to-end enc
**Input**
```shell
-
node consumer.js
-
```
5. In a new terminal tab, run the producer to produce encrypted messages.
@@ -599,9 +569,7 @@ This section provides step-by-step instructions on how to use the end-to-end enc
**Input**
```shell
-
node producer.js
-
```
Now you can see the producer sends messages and the consumer receives messages successfully.
@@ -611,7 +579,6 @@ This section provides step-by-step instructions on how to use the end-to-end enc
This is from the producer side.
```
-
Sent message: my-message-0
Sent message: my-message-1
Sent message: my-message-2
@@ -622,13 +589,11 @@ This section provides step-by-step instructions on how to use the end-to-end enc
Sent message: my-message-7
Sent message: my-message-8
Sent message: my-message-9
-
```
This is from the consumer side.
```
-
my-message-0
my-message-1
my-message-2
@@ -639,6 +604,5 @@ This section provides step-by-step instructions on how to use the end-to-end enc
my-message-7
my-message-8
my-message-9
-
```
diff --git a/site2/website-next/docs/client-libraries-python.md b/site2/website-next/docs/client-libraries-python.md
index 32c9d784757..6884f689b31 100644
--- a/site2/website-next/docs/client-libraries-python.md
+++ b/site2/website-next/docs/client-libraries-python.md
@@ -25,21 +25,21 @@ You can install the [`pulsar-client`](https://pypi.python.org/pypi/pulsar-client
To install the `pulsar-client` library as a pre-built package using the [pip](https://pip.pypa.io/en/stable/) package manager:
```shell
-$ pip install pulsar-client==@pulsar:version_number@
+pip install pulsar-client==@pulsar:version_number@
```
### Optional dependencies
-If you install the client libraries on Linux to support services like Pulsar functions or Avro serialization, you can install optional components alongside the `pulsar-client` library.
+If you install the client libraries on Linux to support services like Pulsar functions or Avro serialization, you can install optional components alongside the `pulsar-client` library.
```shell
# avro serialization
-$ pip install 'pulsar-client[avro]==@pulsar:version_number@'
+pip install 'pulsar-client[avro]==@pulsar:version_number@'
# functions runtime
-$ pip install 'pulsar-client[functions]==@pulsar:version_number@'
+pip install 'pulsar-client[functions]==@pulsar:version_number@'
# all optional components
-$ pip install 'pulsar-client[all]==@pulsar:version_number@'
+pip install 'pulsar-client[all]==@pulsar:version_number@'
```
Installation via PyPi is available for the following Python versions:
@@ -57,9 +57,9 @@ To install the `pulsar-client` library by building from source, follow [instruct
To install the built Python bindings:
```shell
-$ git clone https://github.com/apache/pulsar
-$ cd pulsar/pulsar-client-cpp/python
-$ sudo pip install .
+git clone https://github.com/apache/pulsar
+cd pulsar/pulsar-client-cpp/python
+sudo pip install .
```
## Connection URLs
@@ -133,7 +133,7 @@ while True:
client.close()
```
-This example shows how to configure negative acknowledgement.
+This example shows how to configure negative acknowledgment.
```python
from pulsar import Client, schema
@@ -212,7 +212,7 @@ client = pulsar.Client('pulsar://localhost:6650', listener_name='external')
### Supported schema types
-You can use different builtin schema types in Pulsar. All the definitions are in the `pulsar.schema` package.
+You can use different built-in schema types in Pulsar. All the definitions are in the `pulsar.schema` package.
| Schema | Notes |
| ------ | ----- |
@@ -225,10 +225,7 @@ You can use different builtin schema types in Pulsar. All the definitions are in
The schema definition is done through a class that inherits from `pulsar.schema.Record`.
-This class has a number of fields which can be of either
-`pulsar.schema.Field` type or another nested `Record`. All the
-fields are specified in the `pulsar.schema` package. The fields
-are matching the AVRO fields types.
+This class has a number of fields that can be of either `pulsar.schema.Field` type or another nested `Record`. All the fields are specified in the `pulsar.schema` package. The fields are matching the AVRO field types.
| Field Type | Python Type | Notes |
| ---------- | ----------- | ----- |
@@ -250,7 +247,7 @@ When adding a field, you can use these parameters in the constructor.
| Argument | Default | Notes |
| ---------- | --------| ----- |
-| `default` | `None` | Set a default value for the field. Eg: `a = Integer(default=5)` |
+| `default` | `None` | Set a default value for the field, such as `a = Integer(default=5)`. |
| `required` | `False` | Mark the field as "required". It is set in the schema accordingly. |
#### Schema definition examples
@@ -295,7 +292,7 @@ class Example(Record):
##### Set namespace for Avro schema
-Set the namespace for Avro Record schema using the special field `_avro_namespace`.
+Set the namespace for the Avro Record schema using the special field `_avro_namespace`.
```python
class NamespaceDemo(Record):
@@ -522,7 +519,7 @@ consumer = client.subscribe(
### Configuration
-To use the end-to-end encryption feature in the Python client, you need to configure `publicKeyPath` for producer and `privateKeyPath` for consumer.
+To use the end-to-end encryption feature in the Python client, you need to configure `publicKeyPath` for producers and `privateKeyPath` for consumers.
```
publicKeyPath: "./public.pem"
diff --git a/site2/website-next/docs/client-libraries-rest.md b/site2/website-next/docs/client-libraries-rest.md
index 8fb89d6283a..a079882ff8b 100644
--- a/site2/website-next/docs/client-libraries-rest.md
+++ b/site2/website-next/docs/client-libraries-rest.md
@@ -13,17 +13,13 @@ To connect to Pulsar, you need to specify a URL.
- Produce messages to non-partitioned or partitioned topics
```
-
brokerUrl:{8080/8081}/topics/{persistent/non-persistent}/{my-tenant}/{my-namespace}/{my-topic}
-
```
- Produce messages to specific partitions of partitioned topics
```
-
brokerUrl:{8080/8081}/topics/{persistent/non-persistent}/{my-tenant}/{my-namespace}/{my-topic}/partitions/{partition-number}
-
```
## Producer
@@ -82,7 +78,6 @@ Below is an example of sending messages to topics using JSON schema via REST.
Assume that you send messages representing the following class.
```java
-
class Seller {
public String state;
public String street;
@@ -96,13 +91,11 @@ Assume that you send messages representing the following class.
public GPU gpu;
public Seller seller;
}
-
```
Send messages to topics with JSON schema using the command below.
```shell
-
curl --location --request POST 'brokerUrl:{8080/8081}/topics/{persistent/non-persistent}/{my-tenant}/{my-namespace}/{my-topic}' \
--header 'Content-Type: application/json' \
--data-raw '{
@@ -128,6 +121,5 @@ curl --location --request POST 'brokerUrl:{8080/8081}/topics/{persistent/non-per
}
`
// Sample message
-
```
diff --git a/site2/website-next/docs/client-libraries-websocket.md b/site2/website-next/docs/client-libraries-websocket.md
index c5455b05388..e128d7ef6ba 100644
--- a/site2/website-next/docs/client-libraries-websocket.md
+++ b/site2/website-next/docs/client-libraries-websocket.md
@@ -23,9 +23,7 @@ In non-standalone mode, there are two ways to deploy the WebSocket service:
In this mode, the WebSocket service will run within the same HTTP service that's already running in the broker. To enable this mode, set the [`webSocketServiceEnabled`](reference-configuration.md#broker-webSocketServiceEnabled) parameter in the [`conf/broker.conf`](reference-configuration.md#broker) configuration file in your installation.
```properties
-
webSocketServiceEnabled=true
-
```
### As a separate component
@@ -39,11 +37,9 @@ In this mode, the WebSocket service will be run from a Pulsar [broker](reference
Here's an example:
```properties
-
configurationMetadataStoreUrl=zk1:2181,zk2:2181,zk3:2181
webServicePort=8080
clusterName=my-cluster
-
```
### Security settings
@@ -51,13 +47,11 @@ clusterName=my-cluster
To enable TLS encryption on WebSocket service:
```properties
-
tlsEnabled=true
tlsAllowInsecureConnection=false
tlsCertificateFilePath=/path/to/client-websocket.cert.pem
tlsKeyFilePath=/path/to/client-websocket.key-pk8.pem
tlsTrustCertsFilePath=/path/to/ca.cert.pem
-
```
To enable encryption at rest on WebSocket service, add CryptoKeyReaderFactory factory class in classpath which will create CryptoKeyReader for WebSocket and that helps to load encryption keys for producer/consumer.
@@ -71,9 +65,7 @@ cryptoKeyReaderFactoryClassName=org.apache.pulsar.MyCryptoKeyReaderFactoryClassI
When the configuration is set, you can start the service using the [`pulsar-daemon`](reference-cli-tools.md#pulsar-daemon) tool:
```shell
-
-$ bin/pulsar-daemon start websocket
-
+bin/pulsar-daemon start websocket
```
## API Reference
@@ -86,12 +78,10 @@ All exchanges via the WebSocket API use JSON.
#### Browser javascript WebSocket client
-Use the query param `token` transport the authentication token.
+Use the query param `token` to transport the authentication token.
```http
-
ws://broker-service-url:8080/path?token=token
-
```
### Producer endpoint
@@ -99,9 +89,7 @@ ws://broker-service-url:8080/path?token=token
The producer endpoint requires you to specify a tenant, namespace, and topic in the URL:
```http
-
ws://broker-service-url:8080/ws/v2/producer/persistent/:tenant/:namespace/:topic
-
```
##### Query param
@@ -125,13 +113,11 @@ Key | Type | Required? | Explanation
#### Publishing a message
```json
-
{
"payload": "SGVsbG8gV29ybGQ=",
"properties": {"key1": "value1", "key2": "value2"},
"context": "1"
}
-
```
Key | Type | Required? | Explanation
@@ -146,25 +132,21 @@ Key | Type | Required? | Explanation
##### Example success response
```json
-
{
"result": "ok",
"messageId": "CAAQAw==",
"context": "1"
}
-
```
##### Example failure response
```json
-
{
"result": "send-error:3",
"errorMsg": "Failed to de-serialize from JSON",
"context": "1"
}
-
```
Key | Type | Required? | Explanation
@@ -179,9 +161,7 @@ Key | Type | Required? | Explanation
The consumer endpoint requires you to specify a tenant, namespace, and topic, as well as a subscription, in the URL:
```http
-
ws://broker-service-url:8080/ws/v2/consumer/persistent/:tenant/:namespace/:topic/:subscription
-
```
##### Query param
@@ -199,16 +179,19 @@ Key | Type | Required? | Explanation
`negativeAckRedeliveryDelay` | int | no | When a message is negatively acknowledged, the delay time before the message is redelivered (in milliseconds). The default value is 60000.
`token` | string | no | Authentication token, this is used for the browser javascript client
-NB: these parameter (except `pullMode`) apply to the internal consumer of the WebSocket service.
-So messages will be subject to the redelivery settings as soon as the get into the receive queue,
+:::note
+
+These parameters (except `pullMode`) apply to the internal consumers of the WebSocket service.
+So messages will be subject to the redelivery settings as soon as they get into the receive queue,
even if the client doesn't consume on the WebSocket.
+:::
+
##### Receiving messages
Server will push messages on the WebSocket session:
```json
-
{
"messageId": "CAMQADAA",
"payload": "hvXcJvHW7kOSrUn17P2q71RA5SdiXwZBqw==",
@@ -231,7 +214,6 @@ Server will push messages on the WebSocket session:
}
}
}
-
```
Below are the parameters in the WebSocket consumer response.
@@ -271,11 +253,9 @@ Consumer needs to acknowledge the successful processing of the message to
have the Pulsar broker delete it.
```json
-
{
"messageId": "CAAQAw=="
}
-
```
Key | Type | Required? | Explanation
@@ -285,12 +265,10 @@ Key | Type | Required? | Explanation
#### Negatively acknowledging messages
```json
-
{
"type": "negativeAcknowledge",
"messageId": "CAAQAw=="
}
-
```
Key | Type | Required? | Explanation
@@ -301,9 +279,8 @@ Key | Type | Required? | Explanation
##### Push Mode
-By default (`pullMode=false`), the consumer endpoint will use the `receiverQueueSize` parameter both to size its
-internal receive queue and to limit the number of unacknowledged messages that are passed to the WebSocket client.
-In this mode, if you don't send acknowledgements, the Pulsar WebSocket service will stop sending messages after reaching
+By default (`pullMode=false`), the consumer endpoint will use the `receiverQueueSize` parameter both to size its internal receive queue and to limit the number of unacknowledged messages that are passed to the WebSocket client.
+In this mode, if you don't send acknowledgments, the Pulsar WebSocket service will stop sending messages after reaching
`receiverQueueSize` unacked messages sent to the WebSocket client.
##### Pull Mode
@@ -312,12 +289,10 @@ If you set `pullMode` to `true`, the WebSocket client will need to send `permit`
Pulsar WebSocket service to send more messages.
```json
-
{
"type": "permit",
"permitMessages": 100
}
-
```
Key | Type | Required? | Explanation
@@ -325,20 +300,18 @@ Key | Type | Required? | Explanation
`type`| string | yes | Type of command. Must be `permit`
`permitMessages`| int | yes | Number of messages to permit
-NB: in this mode it's possible to acknowledge messages in a different connection.
+> In this mode it's possible to acknowledge messages in a different connection.
-#### Check if reach end of topic
+#### Check if reach the end of topic
-Consumer can check if it has reached end of topic by sending `isEndOfTopic` request.
+Consumers can check if it has reached the end of a topic by sending the `isEndOfTopic` request.
**Request**
```json
-
{
"type": "isEndOfTopic"
}
-
```
Key | Type | Required? | Explanation
@@ -348,11 +321,9 @@ Key | Type | Required? | Explanation
**Response**
```json
-
{
"endOfTopic": "true/false"
}
-
```
### Reader endpoint
@@ -360,9 +331,7 @@ Key | Type | Required? | Explanation
The reader endpoint requires you to specify a tenant, namespace, and topic in the URL:
```http
-
ws://broker-service-url:8080/ws/v2/reader/persistent/:tenant/:namespace/:topic
-
```
##### Query param
@@ -379,7 +348,6 @@ Key | Type | Required? | Explanation
Server will push messages on the WebSocket session:
```json
-
{
"messageId": "CAAQAw==",
"payload": "SGVsbG8gV29ybGQ=",
@@ -387,7 +355,6 @@ Server will push messages on the WebSocket session:
"publishTime": "2016-08-30 16:45:57.785",
"redeliveryCount": 4
}
-
```
Key | Type | Required? | Explanation
@@ -403,32 +370,28 @@ Key | Type | Required? | Explanation
**In WebSocket**, Reader needs to acknowledge the successful processing of the message to
have the Pulsar WebSocket service update the number of pending messages.
-If you don't send acknowledgements, Pulsar WebSocket service will stop sending messages after reaching the pendingMessages limit.
+If you don't send acknowledgments, Pulsar WebSocket service will stop sending messages after reaching the pendingMessages limit.
```json
-
{
"messageId": "CAAQAw=="
}
-
```
Key | Type | Required? | Explanation
:---|:-----|:----------|:-----------
`messageId`| string | yes | Message ID of the processed message
-#### Check if reach end of topic
+#### Check if reach the end of topic
-Consumer can check if it has reached end of topic by sending `isEndOfTopic` request.
+Consumers can check if it has reached the end of topic by sending the `isEndOfTopic` request.
**Request**
```json
-
{
"type": "isEndOfTopic"
}
-
```
Key | Type | Required? | Explanation
@@ -438,11 +401,9 @@ Key | Type | Required? | Explanation
**Response**
```json
-
{
"endOfTopic": "true/false"
}
-
```
### Error codes
@@ -472,9 +433,7 @@ Below you'll find code examples for the Pulsar WebSocket API in [Python](#python
This example uses the [`websocket-client`](https://pypi.python.org/pypi/websocket-client) package. You can install it using [pip](https://pypi.python.org/pypi/pip):
```shell
-
-$ pip install websocket-client
-
+pip install websocket-client
```
You can also download it from [PyPI](https://pypi.python.org/pypi/websocket-client).
@@ -484,7 +443,6 @@ You can also download it from [PyPI](https://pypi.python.org/pypi/websocket-clie
Here's an example Python producer that sends a simple message to a Pulsar [topic](reference-terminology.md#topic):
```python
-
import websocket, base64, json
# If set enableTLS to true, your have to set tlsEnabled to true in conf/websocket.conf.
@@ -519,7 +477,6 @@ if response['result'] == 'ok':
else:
print('Failed to publish message:', response)
ws.close()
-
```
#### Python consumer
@@ -527,7 +484,6 @@ ws.close()
Here's an example Python consumer that listens on a Pulsar topic and prints the message ID whenever a message arrives:
```python
-
import websocket, base64, json
# If set enableTLS to true, your have to set tlsEnabled to true in conf/websocket.conf.
@@ -550,7 +506,6 @@ while True:
ws.send(json.dumps({'messageId' : msg['messageId']}))
ws.close()
-
```
#### Python reader
@@ -558,7 +513,6 @@ ws.close()
Here's an example Python reader that listens on a Pulsar topic and prints the message ID whenever a message arrives:
```python
-
import websocket, base64, json
# If set enableTLS to true, your have to set tlsEnabled to true in conf/websocket.conf.
@@ -580,7 +534,6 @@ while True:
ws.send(json.dumps({'messageId' : msg['messageId']}))
ws.close()
-
```
### Node.js
@@ -588,9 +541,7 @@ ws.close()
This example uses the [`ws`](https://websockets.github.io/ws/) package. You can install it using [npm](https://www.npmjs.com/):
```shell
-
-$ npm install ws
-
+npm install ws
```
#### Node.js producer
@@ -598,7 +549,6 @@ $ npm install ws
Here's an example Node.js producer that sends a simple message to a Pulsar topic:
```javascript
-
const WebSocket = require('ws');
// If set enableTLS to true, your have to set tlsEnabled to true in conf/websocket.conf.
@@ -623,7 +573,6 @@ ws.on('open', function() {
ws.on('message', function(message) {
console.log('received ack: %s', message);
});
-
```
#### Node.js consumer
@@ -631,7 +580,6 @@ ws.on('message', function(message) {
Here's an example Node.js consumer that listens on the same topic used by the producer above:
```javascript
-
const WebSocket = require('ws');
// If set enableTLS to true, your have to set tlsEnabled to true in conf/websocket.conf.
@@ -645,13 +593,11 @@ ws.on('message', function(message) {
var ackMsg = {"messageId" : receiveMsg.messageId};
ws.send(JSON.stringify(ackMsg));
});
-
```
#### NodeJS reader
```javascript
-
const WebSocket = require('ws');
// If set enableTLS to true, your have to set tlsEnabled to true in conf/websocket.conf.
@@ -665,6 +611,5 @@ ws.on('message', function(message) {
var ackMsg = {"messageId" : receiveMsg.messageId};
ws.send(JSON.stringify(ackMsg));
});
-
```
diff --git a/site2/website-next/docs/client-libraries.md b/site2/website-next/docs/client-libraries.md
index 5aed5c52636..d5842bdd0c1 100644
--- a/site2/website-next/docs/client-libraries.md
+++ b/site2/website-next/docs/client-libraries.md
@@ -4,7 +4,7 @@ title: Pulsar client libraries
sidebar_label: "Overview"
---
-Pulsar supports the following language specific client libraries:
+Pulsar supports the following language-specific client libraries:
| Language | Documentation | Release note | Code repo |
| --------- | -------------------------------------------------------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
@@ -28,7 +28,7 @@ Pulsar client feature matrix for different languages is listed on [Pulsar Featur
## Third-party clients
-Besides the official released clients, multiple projects on developing Pulsar clients are available in different languages.
+Besides the officially released clients, multiple projects on developing Pulsar clients are available in different languages.
> Want your repository listed here? Just submit a PR to the [pulsar repository](https://github.com/apache/pulsar/edit/master/site2/docs/client-libraries.md).
diff --git a/site2/website-next/docs/concepts-architecture-overview.md b/site2/website-next/docs/concepts-architecture-overview.md
index a0f99205c23..116b904d19d 100644
--- a/site2/website-next/docs/concepts-architecture-overview.md
+++ b/site2/website-next/docs/concepts-architecture-overview.md
@@ -12,11 +12,11 @@ In a Pulsar cluster:
* A BookKeeper cluster consisting of one or more bookies handles [persistent storage](#persistent-storage) of messages.
* A ZooKeeper cluster specific to that cluster handles coordination tasks between Pulsar clusters.
-The diagram below provides an illustration of a Pulsar cluster:
+The diagram below illustrates a Pulsar cluster:
-At the broader instance level, an instance-wide ZooKeeper cluster called the configuration store handles coordination tasks involving multiple clusters, for example [geo-replication](concepts-replication.md).
+At the broader instance level, an instance-wide ZooKeeper cluster called the configuration store handles coordination tasks involving multiple clusters, for example, [geo-replication](concepts-replication.md).
## Brokers
@@ -39,7 +39,7 @@ A Pulsar instance consists of one or more Pulsar *clusters*. Clusters, in turn,
* A ZooKeeper quorum used for cluster-level configuration and coordination
* An ensemble of bookies used for [persistent storage](#persistent-storage) of messages
-Clusters can replicate amongst themselves using [geo-replication](concepts-replication.md).
+Clusters can replicate among themselves using [geo-replication](concepts-replication.md).
> For a guide to managing Pulsar clusters, see the [clusters](admin-api-clusters.md) guide.
@@ -47,7 +47,7 @@ Clusters can replicate amongst themselves using [geo-replication](concepts-repli
The Pulsar metadata store maintains all the metadata of a Pulsar cluster, such as topic metadata, schema, broker load data, and so on. Pulsar uses [Apache ZooKeeper](https://zookeeper.apache.org/) for metadata storage, cluster configuration, and coordination. The Pulsar metadata store can be deployed on a separate ZooKeeper cluster or deployed on an existing ZooKeeper cluster. You can use one ZooKeeper cluster for both Pulsar metadata store and BookKeeper metadata store. If you want to d [...]
-> Pulsar also supports more metadata backend services, including [ETCD](https://etcd.io/) and [RocksDB](http://rocksdb.org/) (for standalone Pulsar only).
+> Pulsar also supports more metadata backend services, including [etcd](https://etcd.io/) and [RocksDB](http://rocksdb.org/) (for standalone Pulsar only).
In a Pulsar instance:
@@ -57,33 +57,31 @@ In a Pulsar instance:
## Configuration store
-The configuration store maintains all the configurations of a Pulsar instance, such as clusters, tenants, namespaces, partitioned topic related configurations, and so on. A Pulsar instance can have a single local cluster, multiple local clusters, or multiple cross-region clusters. Consequently, the configuration store can share the configurations across multiple clusters under a Pulsar instance. The configuration store can be deployed on a separate ZooKeeper cluster or deployed on an exi [...]
+The configuration store maintains all the configurations of a Pulsar instance, such as clusters, tenants, namespaces, partitioned topic-related configurations, and so on. A Pulsar instance can have a single local cluster, multiple local clusters, or multiple cross-region clusters. Consequently, the configuration store can share the configurations across multiple clusters under a Pulsar instance. The configuration store can be deployed on a separate ZooKeeper cluster or deployed on an exi [...]
## Persistent storage
Pulsar provides guaranteed message delivery for applications. If a message successfully reaches a Pulsar broker, it will be delivered to its intended target.
-This guarantee requires that non-acknowledged messages are stored in a durable manner until they can be delivered to and acknowledged by consumers. This mode of messaging is commonly called *persistent messaging*. In Pulsar, N copies of all messages are stored and synced on disk, for example 4 copies across two servers with mirrored [RAID](https://en.wikipedia.org/wiki/RAID) volumes on each server.
+This guarantee requires that non-acknowledged messages are stored durably until they can be delivered to and acknowledged by consumers. This mode of messaging is commonly called *persistent messaging*. In Pulsar, N copies of all messages are stored and synced on disk, for example, 4 copies across two servers with mirrored [RAID](https://en.wikipedia.org/wiki/RAID) volumes on each server.
### Apache BookKeeper
-Pulsar uses a system called [Apache BookKeeper](http://bookkeeper.apache.org/) for persistent message storage. BookKeeper is a distributed [write-ahead log](https://en.wikipedia.org/wiki/Write-ahead_logging) (WAL) system that provides a number of crucial advantages for Pulsar:
+Pulsar uses a system called [Apache BookKeeper](http://bookkeeper.apache.org/) for persistent message storage. BookKeeper is a distributed [write-ahead log](https://en.wikipedia.org/wiki/Write-ahead_logging) (WAL) system that provides several crucial advantages for Pulsar:
* It enables Pulsar to utilize many independent logs, called [ledgers](#ledgers). Multiple ledgers can be created for topics over time.
* It offers very efficient storage for sequential data that handles entry replication.
* It guarantees read consistency of ledgers in the presence of various system failures.
* It offers even distribution of I/O across bookies.
* It's horizontally scalable in both capacity and throughput. Capacity can be immediately increased by adding more bookies to a cluster.
-* Bookies are designed to handle thousands of ledgers with concurrent reads and writes. By using multiple disk devices---one for journal and another for general storage--bookies are able to isolate the effects of read operations from the latency of ongoing write operations.
+* Bookies are designed to handle thousands of ledgers with concurrent reads and writes. By using multiple disk devices---one for journal and another for general storage--bookies can isolate the effects of reading operations from the latency of ongoing write operations.
In addition to message data, *cursors* are also persistently stored in BookKeeper. Cursors are [subscription](reference-terminology.md#subscription) positions for [consumers](reference-terminology.md#consumer). BookKeeper enables Pulsar to store consumer position in a scalable fashion.
At the moment, Pulsar supports persistent message storage. This accounts for the `persistent` in all topic names. Here's an example:
```http
-
persistent://my-tenant/my-namespace/my-topic
-
```
> Pulsar also supports ephemeral ([non-persistent](concepts-messaging.md#non-persistent-topics.md)) message storage.
@@ -130,12 +128,10 @@ The **Pulsar proxy** provides a solution to this problem by acting as a single g
Architecturally, the Pulsar proxy gets all the information it requires from ZooKeeper. When starting the proxy on a machine, you only need to provide metadata store connection strings for the cluster-specific and instance-wide configuration store clusters. Here's an example:
```bash
-
-$ cd /path/to/pulsar/directory
-$ bin/pulsar proxy \
- --metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181 \
- --configuration-metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181
-
+cd /path/to/pulsar/directory
+bin/pulsar proxy \
+--metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181 \
+--configuration-metadata-store zk:my-zk-1:2181,my-zk-2:2181,my-zk-3:2181
```
> #### Pulsar proxy docs
@@ -160,11 +156,9 @@ The diagram below illustrates Pulsar service discovery:
In this diagram, the Pulsar cluster is addressable via a single DNS name: `pulsar-cluster.acme.com`. A [Python client](client-libraries-python.md), for example, could access this Pulsar cluster like this:
```python
-
from pulsar import Client
client = Client('pulsar://pulsar-cluster.acme.com:6650')
-
```
:::note
diff --git a/site2/website-next/docs/concepts-clients.md b/site2/website-next/docs/concepts-clients.md
index c0fe61cd06e..c987ae95fa3 100644
--- a/site2/website-next/docs/concepts-clients.md
+++ b/site2/website-next/docs/concepts-clients.md
@@ -17,36 +17,37 @@ Under the hood, the current official Pulsar client libraries support transparent
Before an application creates a producer/consumer, the Pulsar client library needs to initiate a setup phase including two steps:
1. The client attempts to determine the owner of the topic by sending an HTTP lookup request to the broker. The request could reach one of the active brokers which, by looking at the (cached) zookeeper metadata knows who is serving the topic or, in case nobody is serving it, tries to assign it to the least loaded broker.
-1. Once the client library has the broker address, it creates a TCP connection (or reuse an existing connection from the pool) and authenticates it. Within this connection, client and broker exchange binary commands from a custom protocol. At this point the client sends a command to create producer/consumer to the broker, which will comply after having validated the authorization policy.
+1. Once the client library has the broker address, it creates a TCP connection (or reuses an existing connection from the pool) and authenticates it. Within this connection, the client and broker exchange binary commands from a custom protocol. At this point, the client sends a command to create producer/consumer to the broker, which will comply after having validated the authorization policy.
Whenever the TCP connection breaks, the client immediately re-initiates this setup phase and keeps trying with exponential backoff to re-establish the producer or consumer until the operation succeeds.
## Reader interface
-In Pulsar, the "standard" [consumer interface](concepts-messaging.md#consumers) involves using consumers to listen on [topics](reference-terminology.md#topic), process incoming messages, and finally acknowledge those messages when they are processed. Whenever a new subscription is created, it is initially positioned at the end of the topic (by default), and consumers associated with that subscription begin reading with the first message created afterwards. Whenever a consumer connects t [...]
+In Pulsar, the "standard" [consumer interface](concepts-messaging.md#consumers) involves using consumers to listen on [topics](reference-terminology.md#topic), process incoming messages, and finally acknowledge those messages when they are processed. Whenever a new subscription is created, it is initially positioned at the end of the topic (by default), and consumers associated with that subscription begin reading with the first message created afterward. Whenever a consumer connects to [...]
The **reader interface** for Pulsar enables applications to manually manage cursors. When you use a reader to connect to a topic---rather than a consumer---you need to specify *which* message the reader begins reading from when it connects to a topic. When connecting to a topic, the reader interface enables you to begin with:
-* The **earliest** available message in the topic
-* The **latest** available message in the topic
-* Some other message between the earliest and the latest. If you select this option, you'll need to explicitly provide a message ID. Your application will be responsible for "knowing" this message ID in advance, perhaps fetching it from a persistent data store or cache.
+* The **earliest** available message in the topic.
+* The **latest** available message in the topic.
+* Some other messages between the earliest and the latest. If you select this option, you'll need to explicitly provide a message ID. Your application will be responsible for "knowing" this message ID in advance, perhaps fetching it from a persistent data store or cache.
-The reader interface is helpful for use cases like using Pulsar to provide effectively-once processing semantics for a stream processing system. For this use case, it's essential that the stream processing system be able to "rewind" topics to a specific message and begin reading there. The reader interface provides Pulsar clients with the low-level abstraction necessary to "manually position" themselves within a topic.
+The reader interface is helpful for use cases like using Pulsar to provide effectively-once processing semantics for a stream processing system. For this use case, the stream processing system must be able to "rewind" topics to a specific message and begin reading there. The reader interface provides Pulsar clients with the low-level abstraction necessary to "manually position" themselves within a topic.
Internally, the reader interface is implemented as a consumer using an exclusive, non-durable subscription to the topic with a randomly-allocated name.
-[ **IMPORTANT** ]
+:::tip
-Unlike subscription/consumer, readers are non-durable in nature and does not prevent data in a topic from being deleted, thus it is ***strongly*** advised that [data retention](cookbooks-retention-expiry.md) be configured. If data retention for a topic is not configured for an adequate amount of time, messages that the reader has not yet read might be deleted . This causes the readers to essentially skip messages. Configuring the data retention for a topic guarantees the reader with a c [...]
+Unlike subscription/consumer, readers are non-durable in nature and do not prevent data in a topic from being deleted, thus it is ***strongly*** advised that [data retention](cookbooks-retention-expiry.md) be configured. If data retention for a topic is not configured for an adequate amount of time, messages that the reader has not yet read might be deleted. This causes the readers to essentially skip messages. Configuring the data retention for a topic guarantees the reader with a cert [...]
Please also note that a reader can have a "backlog", but the metric is only used for users to know how behind the reader is. The metric is not considered for any backlog quota calculations.
+:::
+
Here's a Java example that begins reading from the earliest available message on a topic:
```java
-
import org.apache.pulsar.client.api.Message;
import org.apache.pulsar.client.api.MessageId;
import org.apache.pulsar.client.api.Reader;
@@ -62,30 +63,25 @@ while (true) {
// Process the message
}
-
```
To create a reader that reads from the latest available message:
```java
... 31278 lines suppressed ...