You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by ur...@apache.org on 2022/03/31 12:08:25 UTC
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#02eb31b)
This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new 75acae2 Docs sync done from apache/pulsar(#02eb31b)
75acae2 is described below
commit 75acae2fa07a0af026cd819bbf122e98059d0ab7
Author: Pulsar Site Updater <de...@pulsar.apache.org>
AuthorDate: Thu Mar 31 12:08:21 2022 +0000
Docs sync done from apache/pulsar(#02eb31b)
---
site2/docs/deploy-aws.md | 2 +-
site2/docs/developing-binary-protocol.md | 2 +-
site2/docs/getting-started-docker.md | 2 +-
site2/docs/getting-started-helm.md | 2 +-
site2/website-next/docs/deploy-aws.md | 2 +-
.../developing-binary-protocol.md} | 2 +-
.../getting-started-docker.md} | 2 +-
.../getting-started-helm.md} | 2 +-
.../versioned_docs/version-2.2.0/deploy-aws.md | 2 +-
.../version-2.2.0/develop-binary-protocol.md | 15 +-
...y-protocol.md => developing-binary-protocol.md} | 4 +-
.../version-2.2.0/standalone-docker.md | 2 +-
.../versioned_docs/version-2.2.1/deploy-aws.md | 2 +-
.../version-2.2.1/develop-binary-protocol.md | 111 ++--
...ndalone-docker.md => getting-started-docker.md} | 4 +-
.../version-2.2.1/standalone-docker.md | 217 ++++----
.../versioned_docs/version-2.3.0/deploy-aws.md | 2 +-
.../version-2.3.0/develop-binary-protocol.md | 111 ++--
.../version-2.3.0/standalone-docker.md | 2 +-
.../versioned_docs/version-2.3.1/deploy-aws.md | 2 +-
.../version-2.3.1/develop-binary-protocol.md | 111 ++--
...ndalone-docker.md => getting-started-docker.md} | 4 +-
.../version-2.3.1/standalone-docker.md | 196 ++++---
.../versioned_docs/version-2.3.2/deploy-aws.md | 2 +-
.../version-2.3.2/develop-binary-protocol.md | 111 ++--
...ndalone-docker.md => getting-started-docker.md} | 4 +-
.../version-2.3.2/standalone-docker.md | 169 +++---
.../versioned_docs/version-2.4.0/deploy-aws.md | 2 +-
.../version-2.4.0/develop-binary-protocol.md | 11 +-
...y-protocol.md => developing-binary-protocol.md} | 4 +-
.../version-2.4.0/standalone-docker.md | 2 +-
.../versioned_docs/version-2.4.1/deploy-aws.md | 2 +-
.../version-2.4.1/develop-binary-protocol.md | 111 ++--
.../version-2.4.1/standalone-docker.md | 2 +-
.../versioned_docs/version-2.4.2/deploy-aws.md | 2 +-
.../version-2.4.2/develop-binary-protocol.md | 111 ++--
.../version-2.4.2/standalone-docker.md | 2 +-
.../versioned_docs/version-2.5.0/deploy-aws.md | 2 +-
.../version-2.5.0/develop-binary-protocol.md | 111 ++--
.../getting-started-docker.md} | 4 +-
...{kubernetes-helm.md => getting-started-helm.md} | 4 +-
.../version-2.5.0/kubernetes-helm.md | 113 ++--
.../version-2.5.0/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.5.1/deploy-aws.md | 2 +-
.../version-2.5.1/develop-binary-protocol.md | 4 +-
.../developing-binary-protocol.md} | 4 +-
.../version-2.5.1/kubernetes-helm.md | 4 +-
.../version-2.5.1/standalone-docker.md | 4 +-
.../versioned_docs/version-2.5.2/deploy-aws.md | 2 +-
.../version-2.5.2/develop-binary-protocol.md | 4 +-
.../developing-binary-protocol.md} | 4 +-
.../version-2.5.2/kubernetes-helm.md | 1 +
.../version-2.5.2/standalone-docker.md | 2 +-
.../versioned_docs/version-2.6.0/deploy-aws.md | 2 +-
.../version-2.6.0/develop-binary-protocol.md | 4 +-
...y-protocol.md => developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
...{kubernetes-helm.md => getting-started-helm.md} | 4 +-
.../version-2.6.0/kubernetes-helm.md | 54 +-
.../version-2.6.0/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.6.1/deploy-aws.md | 2 +-
.../version-2.6.1/develop-binary-protocol.md | 4 +-
.../developing-binary-protocol.md} | 4 +-
...ndalone-docker.md => getting-started-docker.md} | 4 +-
...{kubernetes-helm.md => getting-started-helm.md} | 4 +-
.../version-2.6.1/kubernetes-helm.md | 53 +-
.../version-2.6.1/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.6.2/deploy-aws.md | 2 +-
.../version-2.6.2/develop-binary-protocol.md | 4 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.6.2/kubernetes-helm.md | 53 +-
.../version-2.6.2/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.6.3/deploy-aws.md | 2 +-
.../version-2.6.3/develop-binary-protocol.md | 4 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.6.3/kubernetes-helm.md | 53 +-
.../version-2.6.3/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.6.4/deploy-aws.md | 2 +-
.../version-2.6.4/develop-binary-protocol.md | 4 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
...{kubernetes-helm.md => getting-started-helm.md} | 4 +-
.../version-2.6.4/kubernetes-helm.md | 42 +-
.../version-2.6.4/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.7.0/deploy-aws.md | 2 +-
.../version-2.7.0/develop-binary-protocol.md | 2 +-
...y-protocol.md => developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.7.0/kubernetes-helm.md | 19 +-
.../version-2.7.0/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.7.1/deploy-aws.md | 2 +-
.../version-2.7.1/develop-binary-protocol.md | 2 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
...{kubernetes-helm.md => getting-started-helm.md} | 4 +-
.../version-2.7.1/kubernetes-helm.md | 19 +-
.../version-2.7.1/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.7.2/deploy-aws.md | 2 +-
.../version-2.7.2/develop-binary-protocol.md | 581 ---------------------
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.7.2/kubernetes-helm.md | 19 +-
.../version-2.7.2/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.7.3/deploy-aws.md | 2 +-
.../version-2.7.3/develop-binary-protocol.md | 2 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.7.3/kubernetes-helm.md | 19 +-
.../version-2.7.3/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.7.4/deploy-aws.md | 2 +-
.../version-2.7.4/develop-binary-protocol.md | 2 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.7.4/kubernetes-helm.md | 19 +-
.../version-2.7.4/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.8.0/deploy-aws.md | 2 +-
.../version-2.8.0/develop-binary-protocol.md | 2 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.8.0/kubernetes-helm.md | 19 +-
.../version-2.8.0/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.8.1/deploy-aws.md | 2 +-
.../version-2.8.1/develop-binary-protocol.md | 2 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.8.1/kubernetes-helm.md | 19 +-
.../version-2.8.1/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.8.2/deploy-aws.md | 2 +-
.../version-2.8.2/develop-binary-protocol.md | 2 +-
.../developing-binary-protocol.md} | 4 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 4 +-
.../version-2.8.2/kubernetes-helm.md | 19 +-
.../version-2.8.2/standalone-docker.md | 149 ++++--
.../versioned_docs/version-2.9.0/deploy-aws.md | 2 +-
.../developing-binary-protocol.md} | 6 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 3 +-
.../versioned_docs/version-2.9.1/deploy-aws.md | 2 +-
.../developing-binary-protocol.md} | 6 +-
.../getting-started-docker.md} | 4 +-
.../getting-started-helm.md} | 3 +-
.../version-2.1.0-incubating/deploy-aws.md | 2 +-
.../developing-binary-protocol.md | 4 +-
.../getting-started-docker.md | 4 +-
.../versioned_docs/version-2.2.0/deploy-aws.md | 2 +-
.../version-2.2.0/developing-binary-protocol.md | 4 +-
.../version-2.2.1/getting-started-docker.md | 4 +-
.../version-2.3.1/getting-started-docker.md | 4 +-
.../version-2.3.2/getting-started-docker.md | 4 +-
.../version-2.4.0/developing-binary-protocol.md | 4 +-
.../versioned_docs/version-2.5.0/deploy-aws.md | 2 +-
.../version-2.5.0/getting-started-docker.md | 4 +-
.../version-2.5.0/getting-started-helm.md | 4 +-
.../version-2.5.1/developing-binary-protocol.md | 4 +-
.../version-2.5.1/kubernetes-helm.md | 4 +-
.../version-2.5.1/standalone-docker.md | 4 +-
.../version-2.5.2/developing-binary-protocol.md | 4 +-
.../versioned_docs/version-2.6.0/deploy-aws.md | 2 +-
.../version-2.6.0/developing-binary-protocol.md | 4 +-
.../version-2.6.0/getting-started-docker.md | 4 +-
.../version-2.6.0/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.6.1/deploy-aws.md | 2 +-
.../version-2.6.1/developing-binary-protocol.md | 4 +-
.../version-2.6.1/getting-started-docker.md | 4 +-
.../version-2.6.1/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.6.2/deploy-aws.md | 2 +-
.../version-2.6.2/developing-binary-protocol.md | 4 +-
.../version-2.6.2/getting-started-docker.md | 4 +-
.../version-2.6.2/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.6.3/deploy-aws.md | 2 +-
.../version-2.6.3/developing-binary-protocol.md | 4 +-
.../version-2.6.3/getting-started-docker.md | 4 +-
.../version-2.6.3/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.6.4/deploy-aws.md | 2 +-
.../version-2.6.4/developing-binary-protocol.md | 4 +-
.../version-2.6.4/getting-started-docker.md | 4 +-
.../version-2.6.4/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.7.0/deploy-aws.md | 2 +-
.../version-2.7.0/developing-binary-protocol.md | 4 +-
.../version-2.7.0/getting-started-docker.md | 4 +-
.../version-2.7.0/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.7.1/deploy-aws.md | 2 +-
.../version-2.7.1/developing-binary-protocol.md | 4 +-
.../version-2.7.1/getting-started-docker.md | 4 +-
.../version-2.7.1/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.7.2/deploy-aws.md | 2 +-
.../version-2.7.2/develop-binary-protocol.md | 4 +-
.../version-2.7.2/developing-binary-protocol.md | 4 +-
.../version-2.7.2/getting-started-docker.md | 4 +-
.../version-2.7.2/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.7.3/deploy-aws.md | 2 +-
.../version-2.7.3/developing-binary-protocol.md | 4 +-
.../version-2.7.3/getting-started-docker.md | 4 +-
.../version-2.7.3/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.7.4/deploy-aws.md | 2 +-
.../version-2.7.4/developing-binary-protocol.md | 4 +-
.../version-2.7.4/getting-started-docker.md | 4 +-
.../version-2.7.4/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.8.0/deploy-aws.md | 2 +-
.../version-2.8.0/developing-binary-protocol.md | 4 +-
.../version-2.8.0/getting-started-docker.md | 4 +-
.../version-2.8.0/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.8.1/deploy-aws.md | 2 +-
.../version-2.8.1/developing-binary-protocol.md | 4 +-
.../version-2.8.1/getting-started-docker.md | 4 +-
.../version-2.8.1/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.8.2/deploy-aws.md | 2 +-
.../version-2.8.2/developing-binary-protocol.md | 4 +-
.../version-2.8.2/getting-started-docker.md | 4 +-
.../version-2.8.2/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.8.3/deploy-aws.md | 2 +-
.../version-2.8.3/developing-binary-protocol.md | 4 +-
.../version-2.8.3/getting-started-docker.md | 4 +-
.../version-2.8.3/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.9.0/deploy-aws.md | 2 +-
.../version-2.9.0/developing-binary-protocol.md | 4 +-
.../version-2.9.0/getting-started-docker.md | 4 +-
.../version-2.9.0/getting-started-helm.md | 4 +-
.../versioned_docs/version-2.9.1/deploy-aws.md | 2 +-
.../version-2.9.1/developing-binary-protocol.md | 4 +-
.../version-2.9.1/getting-started-docker.md | 4 +-
.../version-2.9.1/getting-started-helm.md | 4 +-
233 files changed, 2587 insertions(+), 2620 deletions(-)
diff --git a/site2/docs/deploy-aws.md b/site2/docs/deploy-aws.md
index 802c16f..417bc12 100644
--- a/site2/docs/deploy-aws.md
+++ b/site2/docs/deploy-aws.md
@@ -30,7 +30,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/docs/developing-binary-protocol.md b/site2/docs/developing-binary-protocol.md
index 3caf769..e6c14c6 100644
--- a/site2/docs/developing-binary-protocol.md
+++ b/site2/docs/developing-binary-protocol.md
@@ -1,5 +1,5 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
---
diff --git a/site2/docs/getting-started-docker.md b/site2/docs/getting-started-docker.md
index 0d8fb11..af654b2 100644
--- a/site2/docs/getting-started-docker.md
+++ b/site2/docs/getting-started-docker.md
@@ -1,5 +1,5 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
---
diff --git a/site2/docs/getting-started-helm.md b/site2/docs/getting-started-helm.md
index b147f55..3fa1485 100644
--- a/site2/docs/getting-started-helm.md
+++ b/site2/docs/getting-started-helm.md
@@ -1,5 +1,5 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
---
diff --git a/site2/website-next/docs/deploy-aws.md b/site2/website-next/docs/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/docs/deploy-aws.md
+++ b/site2/website-next/docs/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.3.1/develop-binary-protocol.md b/site2/website-next/docs/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.3.1/develop-binary-protocol.md
copy to site2/website-next/docs/developing-binary-protocol.md
index 63e43dd..61f15d5 100644
--- a/site2/website-next/versioned_docs/version-2.3.1/develop-binary-protocol.md
+++ b/site2/website-next/docs/developing-binary-protocol.md
@@ -1,5 +1,5 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
---
diff --git a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md b/site2/website-next/docs/getting-started-docker.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
copy to site2/website-next/docs/getting-started-docker.md
index f636d2d..cea482e 100644
--- a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
+++ b/site2/website-next/docs/getting-started-docker.md
@@ -1,5 +1,5 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
---
diff --git a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md b/site2/website-next/docs/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
copy to site2/website-next/docs/getting-started-helm.md
index d435371..b72e3f5 100644
--- a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
+++ b/site2/website-next/docs/getting-started-helm.md
@@ -1,5 +1,5 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
---
diff --git a/site2/website-next/versioned_docs/version-2.2.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.2.0/deploy-aws.md
index a5eac0a..5c25f94 100644
--- a/site2/website-next/versioned_docs/version-2.2.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.2.0/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You'll also need to have the Terraform and Ansible configurations for Pulsar locally on your machine. They're contained in Pulsar's [GitHub repository](https://github.com/apache/pulsar), which you can fetch using Git:
diff --git a/site2/website-next/versioned_docs/version-2.2.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.2.0/develop-binary-protocol.md
index cf95010..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.2.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.2.0/develop-binary-protocol.md
@@ -57,11 +57,11 @@ Message metadata is stored alongside the application-specified payload as a seri
| Field | Description |
|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `producer_name` | The name of the producer that published the message |
-| `sequence_id` | The sequence ID of the message, assigned by producer} |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -298,8 +298,11 @@ A `Flow` command gives additional *permits* to send messages to the consumer.
A typical consumer implementation will use a queue to accumulate these messages
before the application is ready to consume them.
-After the application has dequeued a number of message, the consumer will
-send additional number of permits to allow the broker to push more messages.
+After the application has dequeued half of the messages in the queue, the consumer
+sends permits to the broker to ask for more messages (equals to half of the messages in the queue).
+
+For example, if the queue size is 1000 and the consumer consumes 500 messages in the queue.
+Then the consumer sends permits to the broker to ask for 500 messages.
##### Command Subscribe
@@ -318,7 +321,7 @@ message CommandSubscribe {
Parameters:
* `topic` → Complete topic name to where you want to create the consumer on
* `subscription` → Subscription name
- * `subType` → Subscription type: Exclusive, Shared, Failover
+ * `subType` → Subscription type: Exclusive, Shared, Failover, Key_Shared
* `consumer_id` → Client generated consumer identifier. Needs to be unique
within the same connection
* `request_id` → Identifier for this request. Used to match the response with
@@ -455,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.2.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.2.0/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.2.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.2.0/developing-binary-protocol.md
index cf95010..14e992c 100644
--- a/site2/website-next/versioned_docs/version-2.2.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.2.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.2.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.2.0/standalone-docker.md
index f636d2d..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.2.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.2.0/standalone-docker.md
@@ -2,6 +2,7 @@
id: standalone-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: standalone-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.2.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.2.1/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.2.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.2.1/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.2.1/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.2.1/develop-binary-protocol.md
index 63e43dd..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.2.1/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.2.1/develop-binary-protocol.md
@@ -2,6 +2,7 @@
id: develop-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
+original_id: develop-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -28,63 +29,42 @@ The Pulsar protocol allows for two types of commands:
Simple (payload-free) commands have this basic structure:
-| Component | Description | Size (in bytes) |
-|:--------------|:----------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | The size of the protobuf-serialized command | 4 |
-| `message` | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| Component | Description | Size (in bytes) |
+|:------------|:----------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
### Payload commands
Payload commands have this basic structure:
-| Component | Required or optional| Description | Size (in bytes) |
-|:-----------------------------------|:----------|:--------------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | Required | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | Required | The size of the protobuf-serialized command | 4 |
-| `message` | Required | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| `magicNumberOfBrokerEntryMetadata` | Optional | A 2-byte byte array (`0x0e02`) identifying the broker entry metadata <br /> **Note**: `magicNumberOfBrokerEntryMetadata` , `brokerEntryMetadataSize`, and `brokerEntryMetadata` should be used **together**. | 2 |
-| `brokerEntryMetadataSize` | Optional | The size of the broker entry metadata | 4 |
-| `brokerEntryMetadata` | Optional | The broker entry metadata stored as a binary protobuf message | |
-| `magicNumber` | Required | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| `checksum` | Required | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| `metadataSize` | Required | The size of the message [metadata](#message-metadata) | 4 |
-| `metadata` | Required | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| `payload` | Required | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Broker entry metadata
-
-Broker entry metadata is stored alongside the message metadata as a serialized protobuf message.
-It is created by the broker when the message arrived at the broker and passed without changes to the consumer if configured.
-
-| Field | Required or optional | Description |
-|:-------------------|:----------------|:------------------------------------------------------------------------------------------------------------------------------|
-| `broker_timestamp` | Optional | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC) |
-| `index` | Optional | The index of the message. It is assigned by the broker.
-
-If you want to use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
-
-If you want to use broker entry metadata for **consumers**:
-
-1. Use the client protocol version [18 or later](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
-
-2. Configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter and set the [`enableExposingBrokerEntryMetadataToClient`](reference-configuration.md#broker) parameter to `true` in the `broker.conf` file.
+| Component | Description | Size (in bytes) |
+|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
+| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
+| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
+| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
+| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
## Message metadata
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed without changes to the consumer.
+Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-| Field | Required or optional | Description |
-|:-------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | Required | The name of the producer that published the message |
-| `sequence_id` | Required | The sequence ID of the message, assigned by producer |
-| `publish_time` | Required | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | Required | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` | Optional | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` | Optional | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
-| `compression` | Optional | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` | Optional | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` | Optional | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
+| Field | Description |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `producer_name` | The name of the producer that published the message |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
+| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
+| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
+| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
+| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
+| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
+| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
### Batch messages
@@ -96,19 +76,19 @@ object.
For a single batch, the payload format will look like this:
-| Field | Required or optional | Description |
-|:----------------|:---------------------|:-----------------------------------------------------------|
-| `metadataSizeN` | Required |The size of the single message metadata serialized Protobuf |
-| `metadataN` | Required |Single message metadata |
-| `payloadN` | Required |Message payload passed by application |
+| Field | Description |
+|:--------------|:------------------------------------------------------------|
+| metadataSizeN | The size of the single message metadata serialized Protobuf |
+| metadataN | Single message metadata |
+| payloadN | Message payload passed by application |
Each metadata field looks like this;
-| Field | Required or optional | Description |
-|:----------------|:----------------------|:--------------------------------------------------------|
-| `properties` | Required | Application-defined properties |
-| `partition key` | Optional | Key to indicate the hashing to a particular partition |
-| `payload_size` | Required | Size of the payload for the single message in the batch |
+| Field | Description |
+|:---------------------------|:--------------------------------------------------------|
+| properties | Application-defined properties |
+| partition key *(optional)* | Key to indicate the hashing to a particular partition |
+| payload_size | Size of the payload for the single message in the batch |
When compression is enabled, the whole batch will be compressed at once.
@@ -190,10 +170,6 @@ messages to the broker, referring to the producer id negotiated before.
-If the client does not receive a response indicating producer creation success or failure,
-the client should first send a command to close the original producer before sending a
-command to re-attempt producer creation.
-
##### Command Producer
```protobuf
@@ -240,10 +216,8 @@ Parameters:
##### Command Send
Command `Send` is used to publish a new message within the context of an
-already existing producer. If a producer has not yet been created for the
-connection, the broker will terminate the connection. This command is used
-in a frame that includes command as well as message payload, for which the
-complete format is specified in the [payload commands](#payload-commands) section.
+already existing producer. This command is used in a frame that includes command
+as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
```protobuf
@@ -299,11 +273,6 @@ When receiving a `CloseProducer` command, the broker will stop accepting any
more messages for the producer, wait until all pending messages are persisted
and then reply `Success` to the client.
-If the client does not receive a response to a `Producer` command within a timeout,
-the client must first send a `CloseProducer` command before sending another
-`Producer` command. The client does not need to await a response to the `CloseProducer`
-command before sending the next `Producer` command.
-
The broker can send a `CloseProducer` command to client when it's performing
a graceful failover (eg: broker is being restarted, or the topic is being unloaded
by load balancer to be transferred to a different broker).
diff --git a/site2/website-next/versioned_docs/version-2.2.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.2.1/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.2.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.2.1/getting-started-docker.md
index 76635fb..317596d 100644
--- a/site2/website-next/versioned_docs/version-2.2.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.2.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Start a standalone cluster with Docker
sidebar_label: "Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For the purposes of local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.2.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.2.1/standalone-docker.md
index 76635fb..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.2.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.2.1/standalone-docker.md
@@ -1,77 +1,62 @@
---
id: standalone-docker
-title: Start a standalone cluster with Docker
-sidebar_label: "Pulsar in Docker"
+title: Set up a standalone Pulsar in Docker
+sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For the purposes of local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you don't have Docker installed, you can download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
-## Starting Pulsar inside Docker
+## Start Pulsar in Docker
-```shell
-
-$ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- -v $PWD/data:/pulsar/data \
- apachepulsar/pulsar:{{site.current_version}} \
- bin/pulsar standalone
-
-```
+* For MacOS, Linux, and Windows:
-Under Windows, you should use something like the following docker command:
-
-```shell
-
-$ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- -v "$PWD/data:/pulsar/data".ToLower() \
- apachepulsar/pulsar:{{site.current_version}} \
- bin/pulsar standalone
-
-```
+ ```shell
+
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
+
+ ```
A few things to note about this command:
- * `$PWD/data` : The docker host directory under the Windows operating system must be lowercase.`$PWD/data` can provide you with the specified directory, for example: `E:/data`.
- * `-v $PWD/data:/pulsar/data`: This will make the process inside the container to store the
- data and metadata in the filesystem outside the container, in order to not start "fresh" every
- time the container is restarted.
+ * The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
+time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
+ * For Docker on Windows make sure to configure it to use Linux containers
-If Pulsar has been successfully started, you should see `INFO`-level log messages like this:
+If you start Pulsar successfully, you will see `INFO`-level log messages like this:
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
-> #### Automatically created namespace
-> When you start a local standalone cluster, Pulsar will automatically create a `public/default`
-namespace that you can use for development purposes. All Pulsar topics are managed within namespaces.
-For more info, see [Topics](concepts-messaging.md#topics).
+:::tip
+
+When you start a local standalone cluster, a `public/default`
+
+:::
+namespace is created automatically. The namespace is used for development purposes. All Pulsar topics are managed within namespaces.
+For more information, see [Topics](concepts-messaging.md#topics).
-## Start publishing and consuming messages
+## Use Pulsar in Docker
-Pulsar currently offers client libraries for [Java](client-libraries-java.md), [Go](client-libraries-go.md), [Python](client-libraries-python)
+Pulsar offers client libraries for [Java](client-libraries-java.md), [Go](client-libraries-go.md), [Python](client-libraries-python)
and [C++](client-libraries-cpp). If you're running a local standalone cluster, you can
-use one of these root URLs for interacting with your cluster:
+use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-Here's an example that lets you quickly get started with Pulsar by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
-You can install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
+Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
```shell
@@ -79,7 +64,9 @@ $ pip install pulsar-client
```
-First create a consumer and subscribe to the topic:
+### Consume a message
+
+Create a consumer and subscribe to the topic:
```python
@@ -98,7 +85,9 @@ client.close()
```
-Now we can start a producer to send some test messages:
+### Produce a message
+
+Now start a producer to send some test messages:
```python
@@ -116,8 +105,8 @@ client.close()
## Get the topic statistics
-In Pulsar you can use REST, Java, or command-line tools to control every aspect of the system.
-You can find detailed documentation of all the APIs in the [Admin API Overview](admin-api-overview).
+In Pulsar, you can use REST, Java, or command-line tools to control every aspect of the system.
+For details on APIs, refer to [Admin API Overview](admin-api-overview).
In the simplest example, you can use curl to probe the stats for a particular topic:
@@ -127,56 +116,98 @@ $ curl http://localhost:8080/admin/v2/persistent/public/default/my-topic/stats |
```
-The output will be something like this:
+The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.3.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.3.0/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.3.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.3.0/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.3.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.3.0/develop-binary-protocol.md
index 63e43dd..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.3.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.3.0/develop-binary-protocol.md
@@ -2,6 +2,7 @@
id: develop-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
+original_id: develop-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -28,63 +29,42 @@ The Pulsar protocol allows for two types of commands:
Simple (payload-free) commands have this basic structure:
-| Component | Description | Size (in bytes) |
-|:--------------|:----------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | The size of the protobuf-serialized command | 4 |
-| `message` | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| Component | Description | Size (in bytes) |
+|:------------|:----------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
### Payload commands
Payload commands have this basic structure:
-| Component | Required or optional| Description | Size (in bytes) |
-|:-----------------------------------|:----------|:--------------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | Required | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | Required | The size of the protobuf-serialized command | 4 |
-| `message` | Required | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| `magicNumberOfBrokerEntryMetadata` | Optional | A 2-byte byte array (`0x0e02`) identifying the broker entry metadata <br /> **Note**: `magicNumberOfBrokerEntryMetadata` , `brokerEntryMetadataSize`, and `brokerEntryMetadata` should be used **together**. | 2 |
-| `brokerEntryMetadataSize` | Optional | The size of the broker entry metadata | 4 |
-| `brokerEntryMetadata` | Optional | The broker entry metadata stored as a binary protobuf message | |
-| `magicNumber` | Required | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| `checksum` | Required | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| `metadataSize` | Required | The size of the message [metadata](#message-metadata) | 4 |
-| `metadata` | Required | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| `payload` | Required | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Broker entry metadata
-
-Broker entry metadata is stored alongside the message metadata as a serialized protobuf message.
-It is created by the broker when the message arrived at the broker and passed without changes to the consumer if configured.
-
-| Field | Required or optional | Description |
-|:-------------------|:----------------|:------------------------------------------------------------------------------------------------------------------------------|
-| `broker_timestamp` | Optional | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC) |
-| `index` | Optional | The index of the message. It is assigned by the broker.
-
-If you want to use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
-
-If you want to use broker entry metadata for **consumers**:
-
-1. Use the client protocol version [18 or later](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
-
-2. Configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter and set the [`enableExposingBrokerEntryMetadataToClient`](reference-configuration.md#broker) parameter to `true` in the `broker.conf` file.
+| Component | Description | Size (in bytes) |
+|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
+| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
+| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
+| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
+| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
## Message metadata
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed without changes to the consumer.
+Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-| Field | Required or optional | Description |
-|:-------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | Required | The name of the producer that published the message |
-| `sequence_id` | Required | The sequence ID of the message, assigned by producer |
-| `publish_time` | Required | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | Required | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` | Optional | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` | Optional | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
-| `compression` | Optional | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` | Optional | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` | Optional | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
+| Field | Description |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `producer_name` | The name of the producer that published the message |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
+| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
+| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
+| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
+| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
+| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
+| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
### Batch messages
@@ -96,19 +76,19 @@ object.
For a single batch, the payload format will look like this:
-| Field | Required or optional | Description |
-|:----------------|:---------------------|:-----------------------------------------------------------|
-| `metadataSizeN` | Required |The size of the single message metadata serialized Protobuf |
-| `metadataN` | Required |Single message metadata |
-| `payloadN` | Required |Message payload passed by application |
+| Field | Description |
+|:--------------|:------------------------------------------------------------|
+| metadataSizeN | The size of the single message metadata serialized Protobuf |
+| metadataN | Single message metadata |
+| payloadN | Message payload passed by application |
Each metadata field looks like this;
-| Field | Required or optional | Description |
-|:----------------|:----------------------|:--------------------------------------------------------|
-| `properties` | Required | Application-defined properties |
-| `partition key` | Optional | Key to indicate the hashing to a particular partition |
-| `payload_size` | Required | Size of the payload for the single message in the batch |
+| Field | Description |
+|:---------------------------|:--------------------------------------------------------|
+| properties | Application-defined properties |
+| partition key *(optional)* | Key to indicate the hashing to a particular partition |
+| payload_size | Size of the payload for the single message in the batch |
When compression is enabled, the whole batch will be compressed at once.
@@ -190,10 +170,6 @@ messages to the broker, referring to the producer id negotiated before.
-If the client does not receive a response indicating producer creation success or failure,
-the client should first send a command to close the original producer before sending a
-command to re-attempt producer creation.
-
##### Command Producer
```protobuf
@@ -240,10 +216,8 @@ Parameters:
##### Command Send
Command `Send` is used to publish a new message within the context of an
-already existing producer. If a producer has not yet been created for the
-connection, the broker will terminate the connection. This command is used
-in a frame that includes command as well as message payload, for which the
-complete format is specified in the [payload commands](#payload-commands) section.
+already existing producer. This command is used in a frame that includes command
+as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
```protobuf
@@ -299,11 +273,6 @@ When receiving a `CloseProducer` command, the broker will stop accepting any
more messages for the producer, wait until all pending messages are persisted
and then reply `Success` to the client.
-If the client does not receive a response to a `Producer` command within a timeout,
-the client must first send a `CloseProducer` command before sending another
-`Producer` command. The client does not need to await a response to the `CloseProducer`
-command before sending the next `Producer` command.
-
The broker can send a `CloseProducer` command to client when it's performing
a graceful failover (eg: broker is being restarted, or the topic is being unloaded
by load balancer to be transferred to a different broker).
diff --git a/site2/website-next/versioned_docs/version-2.3.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.3.0/standalone-docker.md
index f636d2d..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.3.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.3.0/standalone-docker.md
@@ -2,6 +2,7 @@
id: standalone-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: standalone-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.3.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.3.1/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.3.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.3.1/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.3.1/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.3.1/develop-binary-protocol.md
index 63e43dd..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.3.1/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.3.1/develop-binary-protocol.md
@@ -2,6 +2,7 @@
id: develop-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
+original_id: develop-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -28,63 +29,42 @@ The Pulsar protocol allows for two types of commands:
Simple (payload-free) commands have this basic structure:
-| Component | Description | Size (in bytes) |
-|:--------------|:----------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | The size of the protobuf-serialized command | 4 |
-| `message` | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| Component | Description | Size (in bytes) |
+|:------------|:----------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
### Payload commands
Payload commands have this basic structure:
-| Component | Required or optional| Description | Size (in bytes) |
-|:-----------------------------------|:----------|:--------------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | Required | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | Required | The size of the protobuf-serialized command | 4 |
-| `message` | Required | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| `magicNumberOfBrokerEntryMetadata` | Optional | A 2-byte byte array (`0x0e02`) identifying the broker entry metadata <br /> **Note**: `magicNumberOfBrokerEntryMetadata` , `brokerEntryMetadataSize`, and `brokerEntryMetadata` should be used **together**. | 2 |
-| `brokerEntryMetadataSize` | Optional | The size of the broker entry metadata | 4 |
-| `brokerEntryMetadata` | Optional | The broker entry metadata stored as a binary protobuf message | |
-| `magicNumber` | Required | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| `checksum` | Required | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| `metadataSize` | Required | The size of the message [metadata](#message-metadata) | 4 |
-| `metadata` | Required | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| `payload` | Required | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Broker entry metadata
-
-Broker entry metadata is stored alongside the message metadata as a serialized protobuf message.
-It is created by the broker when the message arrived at the broker and passed without changes to the consumer if configured.
-
-| Field | Required or optional | Description |
-|:-------------------|:----------------|:------------------------------------------------------------------------------------------------------------------------------|
-| `broker_timestamp` | Optional | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC) |
-| `index` | Optional | The index of the message. It is assigned by the broker.
-
-If you want to use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
-
-If you want to use broker entry metadata for **consumers**:
-
-1. Use the client protocol version [18 or later](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
-
-2. Configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter and set the [`enableExposingBrokerEntryMetadataToClient`](reference-configuration.md#broker) parameter to `true` in the `broker.conf` file.
+| Component | Description | Size (in bytes) |
+|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
+| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
+| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
+| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
+| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
## Message metadata
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed without changes to the consumer.
+Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-| Field | Required or optional | Description |
-|:-------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | Required | The name of the producer that published the message |
-| `sequence_id` | Required | The sequence ID of the message, assigned by producer |
-| `publish_time` | Required | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | Required | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` | Optional | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` | Optional | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
-| `compression` | Optional | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` | Optional | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` | Optional | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
+| Field | Description |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `producer_name` | The name of the producer that published the message |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
+| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
+| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
+| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
+| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
+| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
+| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
### Batch messages
@@ -96,19 +76,19 @@ object.
For a single batch, the payload format will look like this:
-| Field | Required or optional | Description |
-|:----------------|:---------------------|:-----------------------------------------------------------|
-| `metadataSizeN` | Required |The size of the single message metadata serialized Protobuf |
-| `metadataN` | Required |Single message metadata |
-| `payloadN` | Required |Message payload passed by application |
+| Field | Description |
+|:--------------|:------------------------------------------------------------|
+| metadataSizeN | The size of the single message metadata serialized Protobuf |
+| metadataN | Single message metadata |
+| payloadN | Message payload passed by application |
Each metadata field looks like this;
-| Field | Required or optional | Description |
-|:----------------|:----------------------|:--------------------------------------------------------|
-| `properties` | Required | Application-defined properties |
-| `partition key` | Optional | Key to indicate the hashing to a particular partition |
-| `payload_size` | Required | Size of the payload for the single message in the batch |
+| Field | Description |
+|:---------------------------|:--------------------------------------------------------|
+| properties | Application-defined properties |
+| partition key *(optional)* | Key to indicate the hashing to a particular partition |
+| payload_size | Size of the payload for the single message in the batch |
When compression is enabled, the whole batch will be compressed at once.
@@ -190,10 +170,6 @@ messages to the broker, referring to the producer id negotiated before.
-If the client does not receive a response indicating producer creation success or failure,
-the client should first send a command to close the original producer before sending a
-command to re-attempt producer creation.
-
##### Command Producer
```protobuf
@@ -240,10 +216,8 @@ Parameters:
##### Command Send
Command `Send` is used to publish a new message within the context of an
-already existing producer. If a producer has not yet been created for the
-connection, the broker will terminate the connection. This command is used
-in a frame that includes command as well as message payload, for which the
-complete format is specified in the [payload commands](#payload-commands) section.
+already existing producer. This command is used in a frame that includes command
+as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
```protobuf
@@ -299,11 +273,6 @@ When receiving a `CloseProducer` command, the broker will stop accepting any
more messages for the producer, wait until all pending messages are persisted
and then reply `Success` to the client.
-If the client does not receive a response to a `Producer` command within a timeout,
-the client must first send a `CloseProducer` command before sending another
-`Producer` command. The client does not need to await a response to the `CloseProducer`
-command before sending the next `Producer` command.
-
The broker can send a `CloseProducer` command to client when it's performing
a graceful failover (eg: broker is being restarted, or the topic is being unloaded
by load balancer to be transferred to a different broker).
diff --git a/site2/website-next/versioned_docs/version-2.3.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.3.1/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.3.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.3.1/getting-started-docker.md
index 4a125f6..01c81b1 100644
--- a/site2/website-next/versioned_docs/version-2.3.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.3.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Start a standalone cluster with Docker
sidebar_label: "Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.3.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.3.1/standalone-docker.md
index 4a125f6..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.3.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.3.1/standalone-docker.md
@@ -1,64 +1,50 @@
---
id: standalone-docker
-title: Start a standalone cluster with Docker
-sidebar_label: "Pulsar in Docker"
+title: Set up a standalone Pulsar in Docker
+sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
-## Start Pulsar inside Docker
+## Start Pulsar in Docker
-```shell
-
-$ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- -v $PWD/data:/pulsar/data \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
-
-```
-
-For Windows, enter something like the following docker command:
+* For MacOS, Linux, and Windows:
-```shell
-
-$ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- -v "$PWD/data:/pulsar/data".ToLower() \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
-
-```
+ ```shell
+
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
+
+ ```
A few things to note about this command:
- * `$PWD/data` : The docker host directory in Windows operating system must be lowercase.`$PWD/data` provides you with the specified directory, for example: `E:/data`.
- * `-v $PWD/data:/pulsar/data`: This makes the process inside the container to store the
- data and metadata in the filesystem outside the container, in order not to start "fresh" every time the container is restarted.
+ * The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
+time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
+ * For Docker on Windows make sure to configure it to use Linux containers
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
-> #### Automatically created namespace
-> When you start a local standalone cluster, a `public/default`
+:::tip
+
+When you start a local standalone cluster, a `public/default`
+
+:::
+
namespace is created automatically. The namespace is used for development purposes. All Pulsar topics are managed within namespaces.
For more information, see [Topics](concepts-messaging.md#topics).
-
-## Start publishing and consuming messages
+## Use Pulsar in Docker
Pulsar offers client libraries for [Java](client-libraries-java.md), [Go](client-libraries-go.md), [Python](client-libraries-python)
and [C++](client-libraries-cpp). If you're running a local standalone cluster, you can
@@ -67,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -78,6 +64,8 @@ $ pip install pulsar-client
```
+### Consume a message
+
Create a consumer and subscribe to the topic:
```python
@@ -97,6 +85,8 @@ client.close()
```
+### Produce a message
+
Now start a producer to send some test messages:
```python
@@ -131,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.3.2/deploy-aws.md b/site2/website-next/versioned_docs/version-2.3.2/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.3.2/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.3.2/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.3.2/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.3.2/develop-binary-protocol.md
index 63e43dd..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.3.2/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.3.2/develop-binary-protocol.md
@@ -2,6 +2,7 @@
id: develop-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
+original_id: develop-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -28,63 +29,42 @@ The Pulsar protocol allows for two types of commands:
Simple (payload-free) commands have this basic structure:
-| Component | Description | Size (in bytes) |
-|:--------------|:----------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | The size of the protobuf-serialized command | 4 |
-| `message` | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| Component | Description | Size (in bytes) |
+|:------------|:----------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
### Payload commands
Payload commands have this basic structure:
-| Component | Required or optional| Description | Size (in bytes) |
-|:-----------------------------------|:----------|:--------------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | Required | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | Required | The size of the protobuf-serialized command | 4 |
-| `message` | Required | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| `magicNumberOfBrokerEntryMetadata` | Optional | A 2-byte byte array (`0x0e02`) identifying the broker entry metadata <br /> **Note**: `magicNumberOfBrokerEntryMetadata` , `brokerEntryMetadataSize`, and `brokerEntryMetadata` should be used **together**. | 2 |
-| `brokerEntryMetadataSize` | Optional | The size of the broker entry metadata | 4 |
-| `brokerEntryMetadata` | Optional | The broker entry metadata stored as a binary protobuf message | |
-| `magicNumber` | Required | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| `checksum` | Required | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| `metadataSize` | Required | The size of the message [metadata](#message-metadata) | 4 |
-| `metadata` | Required | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| `payload` | Required | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Broker entry metadata
-
-Broker entry metadata is stored alongside the message metadata as a serialized protobuf message.
-It is created by the broker when the message arrived at the broker and passed without changes to the consumer if configured.
-
-| Field | Required or optional | Description |
-|:-------------------|:----------------|:------------------------------------------------------------------------------------------------------------------------------|
-| `broker_timestamp` | Optional | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC) |
-| `index` | Optional | The index of the message. It is assigned by the broker.
-
-If you want to use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
-
-If you want to use broker entry metadata for **consumers**:
-
-1. Use the client protocol version [18 or later](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
-
-2. Configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter and set the [`enableExposingBrokerEntryMetadataToClient`](reference-configuration.md#broker) parameter to `true` in the `broker.conf` file.
+| Component | Description | Size (in bytes) |
+|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
+| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
+| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
+| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
+| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
## Message metadata
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed without changes to the consumer.
+Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-| Field | Required or optional | Description |
-|:-------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | Required | The name of the producer that published the message |
-| `sequence_id` | Required | The sequence ID of the message, assigned by producer |
-| `publish_time` | Required | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | Required | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` | Optional | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` | Optional | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
-| `compression` | Optional | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` | Optional | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` | Optional | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
+| Field | Description |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `producer_name` | The name of the producer that published the message |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
+| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
+| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
+| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
+| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
+| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
+| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
### Batch messages
@@ -96,19 +76,19 @@ object.
For a single batch, the payload format will look like this:
-| Field | Required or optional | Description |
-|:----------------|:---------------------|:-----------------------------------------------------------|
-| `metadataSizeN` | Required |The size of the single message metadata serialized Protobuf |
-| `metadataN` | Required |Single message metadata |
-| `payloadN` | Required |Message payload passed by application |
+| Field | Description |
+|:--------------|:------------------------------------------------------------|
+| metadataSizeN | The size of the single message metadata serialized Protobuf |
+| metadataN | Single message metadata |
+| payloadN | Message payload passed by application |
Each metadata field looks like this;
-| Field | Required or optional | Description |
-|:----------------|:----------------------|:--------------------------------------------------------|
-| `properties` | Required | Application-defined properties |
-| `partition key` | Optional | Key to indicate the hashing to a particular partition |
-| `payload_size` | Required | Size of the payload for the single message in the batch |
+| Field | Description |
+|:---------------------------|:--------------------------------------------------------|
+| properties | Application-defined properties |
+| partition key *(optional)* | Key to indicate the hashing to a particular partition |
+| payload_size | Size of the payload for the single message in the batch |
When compression is enabled, the whole batch will be compressed at once.
@@ -190,10 +170,6 @@ messages to the broker, referring to the producer id negotiated before.
-If the client does not receive a response indicating producer creation success or failure,
-the client should first send a command to close the original producer before sending a
-command to re-attempt producer creation.
-
##### Command Producer
```protobuf
@@ -240,10 +216,8 @@ Parameters:
##### Command Send
Command `Send` is used to publish a new message within the context of an
-already existing producer. If a producer has not yet been created for the
-connection, the broker will terminate the connection. This command is used
-in a frame that includes command as well as message payload, for which the
-complete format is specified in the [payload commands](#payload-commands) section.
+already existing producer. This command is used in a frame that includes command
+as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
```protobuf
@@ -299,11 +273,6 @@ When receiving a `CloseProducer` command, the broker will stop accepting any
more messages for the producer, wait until all pending messages are persisted
and then reply `Success` to the client.
-If the client does not receive a response to a `Producer` command within a timeout,
-the client must first send a `CloseProducer` command before sending another
-`Producer` command. The client does not need to await a response to the `CloseProducer`
-command before sending the next `Producer` command.
-
The broker can send a `CloseProducer` command to client when it's performing
a graceful failover (eg: broker is being restarted, or the topic is being unloaded
by load balancer to be transferred to a different broker).
diff --git a/site2/website-next/versioned_docs/version-2.3.2/standalone-docker.md b/site2/website-next/versioned_docs/version-2.3.2/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.3.2/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.3.2/getting-started-docker.md
index e39d94f..0b5a2fe 100644
--- a/site2/website-next/versioned_docs/version-2.3.2/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.3.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.3.2/standalone-docker.md b/site2/website-next/versioned_docs/version-2.3.2/standalone-docker.md
index e39d94f..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.3.2/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.3.2/standalone-docker.md
@@ -5,51 +5,32 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
-* For MacOS and Linux:
+* For MacOS, Linux, and Windows:
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- -v $PWD/data:/pulsar/data \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
-
- ```
-
-* For Windows:
-
- ```shell
-
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- -v "$PWD/data:/pulsar/data".ToLower() \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
A few things to note about this command:
- * `$PWD/data` : The docker host directory in Windows operating system must be lowercase.`$PWD/data` provides you with the specified directory, for example: `E:/data`.
- * `-v $PWD/data:/pulsar/data`: This makes the process inside the container to store the
- data and metadata in the filesystem outside the container, in order not to start "fresh" every time the container is restarted.
+ * The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
+time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
+ * For Docker on Windows make sure to configure it to use Linux containers
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -72,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -140,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.4.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.4.0/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.4.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.4.0/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.4.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.4.0/develop-binary-protocol.md
index 59e342c..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.4.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.4.0/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -298,8 +298,11 @@ A `Flow` command gives additional *permits* to send messages to the consumer.
A typical consumer implementation will use a queue to accumulate these messages
before the application is ready to consume them.
-After the application has dequeued a number of message, the consumer will
-send additional number of permits to allow the broker to push more messages.
+After the application has dequeued half of the messages in the queue, the consumer
+sends permits to the broker to ask for more messages (equals to half of the messages in the queue).
+
+For example, if the queue size is 1000 and the consumer consumes 500 messages in the queue.
+Then the consumer sends permits to the broker to ask for 500 messages.
##### Command Subscribe
@@ -455,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.4.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.4.0/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.4.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.4.0/developing-binary-protocol.md
index 59e342c..4c526b9 100644
--- a/site2/website-next/versioned_docs/version-2.4.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.4.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
index f636d2d..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
@@ -2,6 +2,7 @@
id: standalone-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: standalone-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.4.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.4.1/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.4.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.4.1/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.4.1/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.4.1/develop-binary-protocol.md
index 63e43dd..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.4.1/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.4.1/develop-binary-protocol.md
@@ -2,6 +2,7 @@
id: develop-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
+original_id: develop-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -28,63 +29,42 @@ The Pulsar protocol allows for two types of commands:
Simple (payload-free) commands have this basic structure:
-| Component | Description | Size (in bytes) |
-|:--------------|:----------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | The size of the protobuf-serialized command | 4 |
-| `message` | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| Component | Description | Size (in bytes) |
+|:------------|:----------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
### Payload commands
Payload commands have this basic structure:
-| Component | Required or optional| Description | Size (in bytes) |
-|:-----------------------------------|:----------|:--------------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | Required | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | Required | The size of the protobuf-serialized command | 4 |
-| `message` | Required | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| `magicNumberOfBrokerEntryMetadata` | Optional | A 2-byte byte array (`0x0e02`) identifying the broker entry metadata <br /> **Note**: `magicNumberOfBrokerEntryMetadata` , `brokerEntryMetadataSize`, and `brokerEntryMetadata` should be used **together**. | 2 |
-| `brokerEntryMetadataSize` | Optional | The size of the broker entry metadata | 4 |
-| `brokerEntryMetadata` | Optional | The broker entry metadata stored as a binary protobuf message | |
-| `magicNumber` | Required | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| `checksum` | Required | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| `metadataSize` | Required | The size of the message [metadata](#message-metadata) | 4 |
-| `metadata` | Required | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| `payload` | Required | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Broker entry metadata
-
-Broker entry metadata is stored alongside the message metadata as a serialized protobuf message.
-It is created by the broker when the message arrived at the broker and passed without changes to the consumer if configured.
-
-| Field | Required or optional | Description |
-|:-------------------|:----------------|:------------------------------------------------------------------------------------------------------------------------------|
-| `broker_timestamp` | Optional | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC) |
-| `index` | Optional | The index of the message. It is assigned by the broker.
-
-If you want to use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
-
-If you want to use broker entry metadata for **consumers**:
-
-1. Use the client protocol version [18 or later](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
-
-2. Configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter and set the [`enableExposingBrokerEntryMetadataToClient`](reference-configuration.md#broker) parameter to `true` in the `broker.conf` file.
+| Component | Description | Size (in bytes) |
+|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
+| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
+| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
+| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
+| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
## Message metadata
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed without changes to the consumer.
+Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-| Field | Required or optional | Description |
-|:-------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | Required | The name of the producer that published the message |
-| `sequence_id` | Required | The sequence ID of the message, assigned by producer |
-| `publish_time` | Required | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | Required | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` | Optional | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` | Optional | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
-| `compression` | Optional | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` | Optional | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` | Optional | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
+| Field | Description |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `producer_name` | The name of the producer that published the message |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
+| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
+| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
+| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
+| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
+| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
+| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
### Batch messages
@@ -96,19 +76,19 @@ object.
For a single batch, the payload format will look like this:
-| Field | Required or optional | Description |
-|:----------------|:---------------------|:-----------------------------------------------------------|
-| `metadataSizeN` | Required |The size of the single message metadata serialized Protobuf |
-| `metadataN` | Required |Single message metadata |
-| `payloadN` | Required |Message payload passed by application |
+| Field | Description |
+|:--------------|:------------------------------------------------------------|
+| metadataSizeN | The size of the single message metadata serialized Protobuf |
+| metadataN | Single message metadata |
+| payloadN | Message payload passed by application |
Each metadata field looks like this;
-| Field | Required or optional | Description |
-|:----------------|:----------------------|:--------------------------------------------------------|
-| `properties` | Required | Application-defined properties |
-| `partition key` | Optional | Key to indicate the hashing to a particular partition |
-| `payload_size` | Required | Size of the payload for the single message in the batch |
+| Field | Description |
+|:---------------------------|:--------------------------------------------------------|
+| properties | Application-defined properties |
+| partition key *(optional)* | Key to indicate the hashing to a particular partition |
+| payload_size | Size of the payload for the single message in the batch |
When compression is enabled, the whole batch will be compressed at once.
@@ -190,10 +170,6 @@ messages to the broker, referring to the producer id negotiated before.
-If the client does not receive a response indicating producer creation success or failure,
-the client should first send a command to close the original producer before sending a
-command to re-attempt producer creation.
-
##### Command Producer
```protobuf
@@ -240,10 +216,8 @@ Parameters:
##### Command Send
Command `Send` is used to publish a new message within the context of an
-already existing producer. If a producer has not yet been created for the
-connection, the broker will terminate the connection. This command is used
-in a frame that includes command as well as message payload, for which the
-complete format is specified in the [payload commands](#payload-commands) section.
+already existing producer. This command is used in a frame that includes command
+as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
```protobuf
@@ -299,11 +273,6 @@ When receiving a `CloseProducer` command, the broker will stop accepting any
more messages for the producer, wait until all pending messages are persisted
and then reply `Success` to the client.
-If the client does not receive a response to a `Producer` command within a timeout,
-the client must first send a `CloseProducer` command before sending another
-`Producer` command. The client does not need to await a response to the `CloseProducer`
-command before sending the next `Producer` command.
-
The broker can send a `CloseProducer` command to client when it's performing
a graceful failover (eg: broker is being restarted, or the topic is being unloaded
by load balancer to be transferred to a different broker).
diff --git a/site2/website-next/versioned_docs/version-2.4.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.4.1/standalone-docker.md
index f636d2d..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.4.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.4.1/standalone-docker.md
@@ -2,6 +2,7 @@
id: standalone-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: standalone-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.4.2/deploy-aws.md b/site2/website-next/versioned_docs/version-2.4.2/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.4.2/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.4.2/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.4.2/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.4.2/develop-binary-protocol.md
index 63e43dd..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.4.2/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.4.2/develop-binary-protocol.md
@@ -2,6 +2,7 @@
id: develop-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
+original_id: develop-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -28,63 +29,42 @@ The Pulsar protocol allows for two types of commands:
Simple (payload-free) commands have this basic structure:
-| Component | Description | Size (in bytes) |
-|:--------------|:----------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | The size of the protobuf-serialized command | 4 |
-| `message` | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| Component | Description | Size (in bytes) |
+|:------------|:----------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
### Payload commands
Payload commands have this basic structure:
-| Component | Required or optional| Description | Size (in bytes) |
-|:-----------------------------------|:----------|:--------------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | Required | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | Required | The size of the protobuf-serialized command | 4 |
-| `message` | Required | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| `magicNumberOfBrokerEntryMetadata` | Optional | A 2-byte byte array (`0x0e02`) identifying the broker entry metadata <br /> **Note**: `magicNumberOfBrokerEntryMetadata` , `brokerEntryMetadataSize`, and `brokerEntryMetadata` should be used **together**. | 2 |
-| `brokerEntryMetadataSize` | Optional | The size of the broker entry metadata | 4 |
-| `brokerEntryMetadata` | Optional | The broker entry metadata stored as a binary protobuf message | |
-| `magicNumber` | Required | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| `checksum` | Required | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| `metadataSize` | Required | The size of the message [metadata](#message-metadata) | 4 |
-| `metadata` | Required | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| `payload` | Required | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Broker entry metadata
-
-Broker entry metadata is stored alongside the message metadata as a serialized protobuf message.
-It is created by the broker when the message arrived at the broker and passed without changes to the consumer if configured.
-
-| Field | Required or optional | Description |
-|:-------------------|:----------------|:------------------------------------------------------------------------------------------------------------------------------|
-| `broker_timestamp` | Optional | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC) |
-| `index` | Optional | The index of the message. It is assigned by the broker.
-
-If you want to use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
-
-If you want to use broker entry metadata for **consumers**:
-
-1. Use the client protocol version [18 or later](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
-
-2. Configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter and set the [`enableExposingBrokerEntryMetadataToClient`](reference-configuration.md#broker) parameter to `true` in the `broker.conf` file.
+| Component | Description | Size (in bytes) |
+|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
+| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
+| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
+| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
+| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
## Message metadata
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed without changes to the consumer.
+Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-| Field | Required or optional | Description |
-|:-------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | Required | The name of the producer that published the message |
-| `sequence_id` | Required | The sequence ID of the message, assigned by producer |
-| `publish_time` | Required | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | Required | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` | Optional | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` | Optional | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
-| `compression` | Optional | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` | Optional | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` | Optional | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
+| Field | Description |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `producer_name` | The name of the producer that published the message |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
+| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
+| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
+| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
+| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
+| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
+| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
### Batch messages
@@ -96,19 +76,19 @@ object.
For a single batch, the payload format will look like this:
-| Field | Required or optional | Description |
-|:----------------|:---------------------|:-----------------------------------------------------------|
-| `metadataSizeN` | Required |The size of the single message metadata serialized Protobuf |
-| `metadataN` | Required |Single message metadata |
-| `payloadN` | Required |Message payload passed by application |
+| Field | Description |
+|:--------------|:------------------------------------------------------------|
+| metadataSizeN | The size of the single message metadata serialized Protobuf |
+| metadataN | Single message metadata |
+| payloadN | Message payload passed by application |
Each metadata field looks like this;
-| Field | Required or optional | Description |
-|:----------------|:----------------------|:--------------------------------------------------------|
-| `properties` | Required | Application-defined properties |
-| `partition key` | Optional | Key to indicate the hashing to a particular partition |
-| `payload_size` | Required | Size of the payload for the single message in the batch |
+| Field | Description |
+|:---------------------------|:--------------------------------------------------------|
+| properties | Application-defined properties |
+| partition key *(optional)* | Key to indicate the hashing to a particular partition |
+| payload_size | Size of the payload for the single message in the batch |
When compression is enabled, the whole batch will be compressed at once.
@@ -190,10 +170,6 @@ messages to the broker, referring to the producer id negotiated before.
-If the client does not receive a response indicating producer creation success or failure,
-the client should first send a command to close the original producer before sending a
-command to re-attempt producer creation.
-
##### Command Producer
```protobuf
@@ -240,10 +216,8 @@ Parameters:
##### Command Send
Command `Send` is used to publish a new message within the context of an
-already existing producer. If a producer has not yet been created for the
-connection, the broker will terminate the connection. This command is used
-in a frame that includes command as well as message payload, for which the
-complete format is specified in the [payload commands](#payload-commands) section.
+already existing producer. This command is used in a frame that includes command
+as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
```protobuf
@@ -299,11 +273,6 @@ When receiving a `CloseProducer` command, the broker will stop accepting any
more messages for the producer, wait until all pending messages are persisted
and then reply `Success` to the client.
-If the client does not receive a response to a `Producer` command within a timeout,
-the client must first send a `CloseProducer` command before sending another
-`Producer` command. The client does not need to await a response to the `CloseProducer`
-command before sending the next `Producer` command.
-
The broker can send a `CloseProducer` command to client when it's performing
a graceful failover (eg: broker is being restarted, or the topic is being unloaded
by load balancer to be transferred to a different broker).
diff --git a/site2/website-next/versioned_docs/version-2.4.2/standalone-docker.md b/site2/website-next/versioned_docs/version-2.4.2/standalone-docker.md
index f636d2d..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.4.2/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.4.2/standalone-docker.md
@@ -2,6 +2,7 @@
id: standalone-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: standalone-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.5.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.5.0/deploy-aws.md
index 4845db0..542f40b 100644
--- a/site2/website-next/versioned_docs/version-2.5.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.5.0/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.5.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.5.0/develop-binary-protocol.md
index 63e43dd..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.5.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.5.0/develop-binary-protocol.md
@@ -2,6 +2,7 @@
id: develop-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
+original_id: develop-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -28,63 +29,42 @@ The Pulsar protocol allows for two types of commands:
Simple (payload-free) commands have this basic structure:
-| Component | Description | Size (in bytes) |
-|:--------------|:----------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | The size of the protobuf-serialized command | 4 |
-| `message` | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| Component | Description | Size (in bytes) |
+|:------------|:----------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
### Payload commands
Payload commands have this basic structure:
-| Component | Required or optional| Description | Size (in bytes) |
-|:-----------------------------------|:----------|:--------------------------------------------------------------------------------------------|:----------------|
-| `totalSize` | Required | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| `commandSize` | Required | The size of the protobuf-serialized command | 4 |
-| `message` | Required | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| `magicNumberOfBrokerEntryMetadata` | Optional | A 2-byte byte array (`0x0e02`) identifying the broker entry metadata <br /> **Note**: `magicNumberOfBrokerEntryMetadata` , `brokerEntryMetadataSize`, and `brokerEntryMetadata` should be used **together**. | 2 |
-| `brokerEntryMetadataSize` | Optional | The size of the broker entry metadata | 4 |
-| `brokerEntryMetadata` | Optional | The broker entry metadata stored as a binary protobuf message | |
-| `magicNumber` | Required | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| `checksum` | Required | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| `metadataSize` | Required | The size of the message [metadata](#message-metadata) | 4 |
-| `metadata` | Required | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| `payload` | Required | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Broker entry metadata
-
-Broker entry metadata is stored alongside the message metadata as a serialized protobuf message.
-It is created by the broker when the message arrived at the broker and passed without changes to the consumer if configured.
-
-| Field | Required or optional | Description |
-|:-------------------|:----------------|:------------------------------------------------------------------------------------------------------------------------------|
-| `broker_timestamp` | Optional | The timestamp when a message arrived at the broker (`id est` as the number of milliseconds since January 1st, 1970 in UTC) |
-| `index` | Optional | The index of the message. It is assigned by the broker.
-
-If you want to use broker entry metadata for **brokers**, configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter in the `broker.conf` file.
-
-If you want to use broker entry metadata for **consumers**:
-
-1. Use the client protocol version [18 or later](https://github.com/apache/pulsar/blob/ca37e67211feda4f7e0984e6414e707f1c1dfd07/pulsar-common/src/main/proto/PulsarApi.proto#L259).
-
-2. Configure the [`brokerEntryMetadataInterceptors`](reference-configuration.md#broker) parameter and set the [`enableExposingBrokerEntryMetadataToClient`](reference-configuration.md#broker) parameter to `true` in the `broker.conf` file.
+| Component | Description | Size (in bytes) |
+|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
+| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
+| commandSize | The size of the protobuf-serialized command | 4 |
+| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
+| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
+| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
+| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
+| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
+| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
## Message metadata
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed without changes to the consumer.
+Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-| Field | Required or optional | Description |
-|:-------------------------|:----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | Required | The name of the producer that published the message |
-| `sequence_id` | Required | The sequence ID of the message, assigned by producer |
-| `publish_time` | Required | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | Required | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` | Optional | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` | Optional | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
-| `compression` | Optional | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` | Optional | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` | Optional | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
+| Field | Description |
+|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `producer_name` | The name of the producer that published the message |
+| `sequence_id` | The sequence ID of the message, assigned by producer |
+| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
+| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
+| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
+| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
+| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
+| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
### Batch messages
@@ -96,19 +76,19 @@ object.
For a single batch, the payload format will look like this:
-| Field | Required or optional | Description |
-|:----------------|:---------------------|:-----------------------------------------------------------|
-| `metadataSizeN` | Required |The size of the single message metadata serialized Protobuf |
-| `metadataN` | Required |Single message metadata |
-| `payloadN` | Required |Message payload passed by application |
+| Field | Description |
+|:--------------|:------------------------------------------------------------|
+| metadataSizeN | The size of the single message metadata serialized Protobuf |
+| metadataN | Single message metadata |
+| payloadN | Message payload passed by application |
Each metadata field looks like this;
-| Field | Required or optional | Description |
-|:----------------|:----------------------|:--------------------------------------------------------|
-| `properties` | Required | Application-defined properties |
-| `partition key` | Optional | Key to indicate the hashing to a particular partition |
-| `payload_size` | Required | Size of the payload for the single message in the batch |
+| Field | Description |
+|:---------------------------|:--------------------------------------------------------|
+| properties | Application-defined properties |
+| partition key *(optional)* | Key to indicate the hashing to a particular partition |
+| payload_size | Size of the payload for the single message in the batch |
When compression is enabled, the whole batch will be compressed at once.
@@ -190,10 +170,6 @@ messages to the broker, referring to the producer id negotiated before.
-If the client does not receive a response indicating producer creation success or failure,
-the client should first send a command to close the original producer before sending a
-command to re-attempt producer creation.
-
##### Command Producer
```protobuf
@@ -240,10 +216,8 @@ Parameters:
##### Command Send
Command `Send` is used to publish a new message within the context of an
-already existing producer. If a producer has not yet been created for the
-connection, the broker will terminate the connection. This command is used
-in a frame that includes command as well as message payload, for which the
-complete format is specified in the [payload commands](#payload-commands) section.
+already existing producer. This command is used in a frame that includes command
+as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
```protobuf
@@ -299,11 +273,6 @@ When receiving a `CloseProducer` command, the broker will stop accepting any
more messages for the producer, wait until all pending messages are persisted
and then reply `Success` to the client.
-If the client does not receive a response to a `Producer` command within a timeout,
-the client must first send a `CloseProducer` command before sending another
-`Producer` command. The client does not need to await a response to the `CloseProducer`
-command before sending the next `Producer` command.
-
The broker can send a `CloseProducer` command to client when it's performing
a graceful failover (eg: broker is being restarted, or the topic is being unloaded
by load balancer to be transferred to a different broker).
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.5.0/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.5.0/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.5.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.5.0/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.5.0/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.5.0/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.5.0/getting-started-helm.md
index 81ef10c..a160874 100644
--- a/site2/website-next/versioned_docs/version-2.5.0/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.5.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running
diff --git a/site2/website-next/versioned_docs/version-2.5.0/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.5.0/kubernetes-helm.md
index 81ef10c..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.5.0/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.5.0/kubernetes-helm.md
@@ -5,8 +5,7 @@ sidebar_label: "Run Pulsar in Kubernetes"
original_id: kubernetes-helm
---
-This section guides you through every step of installing and running
-Apache Pulsar with Helm on Kubernetes quickly, including
+This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
- Install the Apache Pulsar on Kubernetes using Helm
- Start and stop Apache Pulsar
@@ -14,7 +13,7 @@ Apache Pulsar with Helm on Kubernetes quickly, including
- Produce and consume messages using Pulsar clients
- Monitor Apache Pulsar status with Prometheus and Grafana
-For deploying a Pulsar cluster for production usage, please read the documentation on [how to configure and install a Pulsar Helm chart](helm-deploy).
+For deploying a Pulsar cluster for production usage, read the documentation on [how to configure and install a Pulsar Helm chart](helm-deploy).
## Prerequisite
@@ -24,7 +23,7 @@ For deploying a Pulsar cluster for production usage, please read the documentati
:::tip
-For the following steps, step 2 and step 3 are for developers and step 4 and step 5 are for administrators.
+For the following steps, step 2 and step 3 are for **developers** and step 4 and step 5 are for **administrators**.
:::
@@ -32,9 +31,9 @@ For the following steps, step 2 and step 3 are for developers and step 4 and ste
Before installing a Pulsar Helm chart, you have to create a Kubernetes cluster. You can follow [the instructions](helm-prepare) to prepare a Kubernetes cluster.
-We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start guide.
+We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start guide. To prepare a Kubernetes cluster, follow these steps:
-1. Create a kubernetes cluster on Minikube.
+1. Create a Kubernetes cluster on Minikube.
```bash
@@ -42,7 +41,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
- The `<k8s-version>` can be any [Kubernetes version supported by your minikube installation](https://minikube.sigs.k8s.io/docs/reference/configuration/kubernetes/). Example: `v1.16.1.
+ The `<k8s-version>` can be any [Kubernetes version supported by your Minikube installation](https://minikube.sigs.k8s.io/docs/reference/configuration/kubernetes/), such as `v1.16.1`.
2. Set `kubectl` to use Minikube.
@@ -52,7 +51,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. In order to use the [Kubernetes Dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) with local Kubernetes cluster on Minikube, enter the command below:
+3. To use the [Kubernetes Dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/) with the local Kubernetes cluster on Minikube, enter the command below:
```bash
@@ -62,9 +61,23 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
The command automatically triggers opening a webpage in your browser.
-## Step 1: Install Pulsar Helm Chart
+## Step 1: Install Pulsar Helm chart
-1. Clone the Pulsar Helm chart repository.
+1. Add Pulsar charts repo.
+
+ ```bash
+
+ helm repo add apache https://pulsar.apache.org/charts
+
+ ```
+
+ ```bash
+
+ helm repo update
+
+ ```
+
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -73,30 +86,36 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run `prepare_helm_release.sh` to create secrets required for installing Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
./scripts/pulsar/prepare_helm_release.sh \
-n pulsar \
-k pulsar-mini \
- --control-center-admin pulsar \
- --control-center-password pulsar \
-c
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
helm install \
--values examples/values-minikube.yaml \
- pulsar-mini charts/pulsar
+ --set initialize=true \
+ --namespace pulsar \
+ pulsar-mini apache/pulsar
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -104,7 +123,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
- If all pods start up successfully, you can see `STATUS` changes to `Running` or `Completed`.
+ If all pods start up successfully, you can see that the `STATUS` is changed to `Running` or `Completed`.
**Output**
@@ -124,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
@@ -150,7 +169,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 2: Use pulsar-admin to create Pulsar tenants/namespaces/topics
-`pulsar-admin` is the CLI tool for Pulsar. In this step, you can use `pulsar-admin` to create resources including tenants, namespaces, and topics.
+`pulsar-admin` is the CLI (command-Line Interface) tool for Pulsar. In this step, you can use `pulsar-admin` to create resources, including tenants, namespaces, and topics.
1. Enter the `toolset` container.
@@ -238,7 +257,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
You can use the Pulsar client to create producers and consumers to produce and consume messages.
-By default the Helm chart expose the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to get the IP address of the proxy service.
+By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to check the proxy service.
```bash
@@ -254,28 +273,48 @@ pulsar-mini-proxy LoadBalancer 10.97.240.109 <pending> 80:32
```
-This output tells what are the node ports that Pulsar cluster's binary port and http port are exposed to. The port after `80:` is the http port while the port after `6650:` is the binary port.
+This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are mapped to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+
+Then you can find the IP address and exposed ports of your Minikube server by running the following command.
+
+```bash
+
+minikube service pulsar-mini-proxy -n pulsar
+
+```
-Then you can find the ip address of your minikube server by running the following command.
+**Output**
```bash
-minikube ip
+|-----------|-------------------|-------------|-------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|-------------------------|
+| pulsar | pulsar-mini-proxy | http/80 | http://172.17.0.4:32305 |
+| | | pulsar/6650 | http://172.17.0.4:31816 |
+|-----------|-------------------|-------------|-------------------------|
+🏃 Starting tunnel for service pulsar-mini-proxy.
+|-----------|-------------------|-------------|------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|------------------------|
+| pulsar | pulsar-mini-proxy | | http://127.0.0.1:61853 |
+| | | | http://127.0.0.1:61854 |
+|-----------|-------------------|-------------|------------------------|
```
-At this point, you will get the service urls to connect to your Pulsar client.
+At this point, you can get the service URLs to connect to your Pulsar client. Here are URL examples:
```
-webServiceUrl=http://$(minikube ip):<exposed-http-port>/
-brokerServiceUrl=pulsar://$(minikube ip):<exposed-binary-port>/
+webServiceUrl=http://127.0.0.1:61853/
+brokerServiceUrl=pulsar://127.0.0.1:61854/
```
-Then proceed with the following steps:
+Then you can proceed with the following steps:
-1. Download the Apache Pulsar tarball from [downloads page](https://pulsar.apache.org/en/download/).
+1. Download the Apache Pulsar tarball from the [downloads page](https://pulsar.apache.org/en/download/).
2. Decompress the tarball based on your download file.
@@ -299,7 +338,7 @@ Then proceed with the following steps:
4. Configure the Pulsar client.
- In the `${PULSAR_HOME}/conf/client.conf` file, replace `webServiceUrl` and `brokerServiceUrl` with the service urls you get from the above steps.
+ In the `${PULSAR_HOME}/conf/client.conf` file, replace `webServiceUrl` and `brokerServiceUrl` with the service URLs you get from the above steps.
5. Create a subscription to consume messages from `apache/pulsar/test-topic`.
@@ -319,7 +358,7 @@ Then proceed with the following steps:
7. Verify the results.
- - From producer side
+ - From the producer side
**Output**
@@ -331,7 +370,7 @@ Then proceed with the following steps:
```
- - From consumer side
+ - From the consumer side
**Output**
@@ -370,11 +409,11 @@ Then proceed with the following steps:
```bash
- minikube service pulsar-mini-pulsar-mananger
+ minikube service -n pulsar pulsar-mini-pulsar-manager
```
-2. The pulsar manager UI will be open in your browser. You can use username `pulsar` and password `pulsar` to log into Pulsar Manager.
+2. The Pulsar Manager UI will be open in your browser. You can use the username `pulsar` and password `pulsar` to log into Pulsar Manager.
3. In Pulsar Manager UI, you can create an environment.
@@ -383,9 +422,9 @@ Then proceed with the following steps:
- Type `http://pulsar-mini-broker:8080` for the field `Service URL` in the popup window.
- Click `Confirm` button in the popup window.
-4. After successfully created an environment, you will be redirected to the `tenants` page of that environment. Then you can create `tenants`, `namespaces` and `topics` using Pulsar Manager.
+4. After successfully created an environment, you are redirected to the `tenants` page of that environment. Then you can create `tenants`, `namespaces` and `topics` using the Pulsar Manager.
-## Step 5: Use Prometheus and Grafana to monitor the cluster
+## Step 5: Use Prometheus and Grafana to monitor cluster
Grafana is an open-source visualization tool, which can be used for visualizing time series data into dashboards.
@@ -397,6 +436,6 @@ Grafana is an open-source visualization tool, which can be used for visualizing
```
-2. The Grafana UI will be open in your browser. You can use username `pulsar` and password `pulsar` to log into Grafana Dashboard.
+2. The Grafana UI is open in your browser. You can use the username `pulsar` and password `pulsar` to log into the Grafana Dashboard.
-3. You will be able to view dashboards for different components of a Pulsar cluster.
\ No newline at end of file
+3. You can view dashboards for different components of a Pulsar cluster.
diff --git a/site2/website-next/versioned_docs/version-2.5.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.5.0/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.5.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.5.0/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.5.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.5.1/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.5.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.5.1/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.5.1/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.5.1/develop-binary-protocol.md
index 31b5b32..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.5.1/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.5.1/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -458,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.5.1/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.5.1/developing-binary-protocol.md
index 31b5b32..790ee88 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.5.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.5.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.5.1/kubernetes-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.5.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.5.1/kubernetes-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.5.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.5.1/standalone-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.5.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.5.1/standalone-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.5.2/deploy-aws.md b/site2/website-next/versioned_docs/version-2.5.2/deploy-aws.md
index 2034749..5db846f 100644
--- a/site2/website-next/versioned_docs/version-2.5.2/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.5.2/deploy-aws.md
@@ -34,7 +34,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.5.2/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.5.2/develop-binary-protocol.md
index 31b5b32..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.5.2/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.5.2/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -458,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.5.2/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.5.2/developing-binary-protocol.md
index 31b5b32..790ee88 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.5.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
index d435371..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
@@ -2,6 +2,7 @@
id: kubernetes-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
+original_id: kubernetes-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.5.2/standalone-docker.md b/site2/website-next/versioned_docs/version-2.5.2/standalone-docker.md
index f636d2d..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.5.2/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.5.2/standalone-docker.md
@@ -2,6 +2,7 @@
id: standalone-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: standalone-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.6.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.6.0/deploy-aws.md
index 4845db0..542f40b 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.6.0/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
index 31b5b32..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -458,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.0/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.6.0/developing-binary-protocol.md
index 31b5b32..790ee88 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.0/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.6.0/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.6.0/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.0/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.6.0/getting-started-helm.md
index fadc855..83d4aa6 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.6.0/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.0/kubernetes-helm.md
index fadc855..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.0/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,17 +97,25 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
helm install \
--values examples/values-minikube.yaml \
+ --set initialize=true \
+ --namespace pulsar \
pulsar-mini apache/pulsar
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -135,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
@@ -249,7 +257,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
You can use the Pulsar client to create producers and consumers to produce and consume messages.
-By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to get the IP address of the proxy service.
+By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to check the proxy service.
```bash
@@ -265,22 +273,42 @@ pulsar-mini-proxy LoadBalancer 10.97.240.109 <pending> 80:32
```
-This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are exposed to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are mapped to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+
+Then you can find the IP address and exposed ports of your Minikube server by running the following command.
+
+```bash
+
+minikube service pulsar-mini-proxy -n pulsar
+
+```
-Then you can find the IP address of your Minikube server by running the following command.
+**Output**
```bash
-minikube ip
+|-----------|-------------------|-------------|-------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|-------------------------|
+| pulsar | pulsar-mini-proxy | http/80 | http://172.17.0.4:32305 |
+| | | pulsar/6650 | http://172.17.0.4:31816 |
+|-----------|-------------------|-------------|-------------------------|
+🏃 Starting tunnel for service pulsar-mini-proxy.
+|-----------|-------------------|-------------|------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|------------------------|
+| pulsar | pulsar-mini-proxy | | http://127.0.0.1:61853 |
+| | | | http://127.0.0.1:61854 |
+|-----------|-------------------|-------------|------------------------|
```
-At this point, you can get the service URLs to connect to your Pulsar client.
+At this point, you can get the service URLs to connect to your Pulsar client. Here are URL examples:
```
-webServiceUrl=http://$(minikube ip):<exposed-http-port>/
-brokerServiceUrl=pulsar://$(minikube ip):<exposed-binary-port>/
+webServiceUrl=http://127.0.0.1:61853/
+brokerServiceUrl=pulsar://127.0.0.1:61854/
```
diff --git a/site2/website-next/versioned_docs/version-2.6.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.0/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.0/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.6.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.6.1/deploy-aws.md
index 4845db0..542f40b 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.6.1/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.6.1/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.1/develop-binary-protocol.md
index 31b5b32..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.1/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -458,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.1/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.6.1/developing-binary-protocol.md
index 31b5b32..790ee88 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.1/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.6.1/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.1/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.6.1/getting-started-helm.md
index 1394086..ba59a23 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.1/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
index 1394086..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
@@ -254,7 +257,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
You can use the Pulsar client to create producers and consumers to produce and consume messages.
-By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to get the IP address of the proxy service.
+By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to check the proxy service.
```bash
@@ -270,22 +273,42 @@ pulsar-mini-proxy LoadBalancer 10.97.240.109 <pending> 80:32
```
-This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are exposed to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are mapped to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+
+Then you can find the IP address and exposed ports of your Minikube server by running the following command.
+
+```bash
+
+minikube service pulsar-mini-proxy -n pulsar
+
+```
-Then you can find the IP address of your Minikube server by running the following command.
+**Output**
```bash
-minikube ip
+|-----------|-------------------|-------------|-------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|-------------------------|
+| pulsar | pulsar-mini-proxy | http/80 | http://172.17.0.4:32305 |
+| | | pulsar/6650 | http://172.17.0.4:31816 |
+|-----------|-------------------|-------------|-------------------------|
+🏃 Starting tunnel for service pulsar-mini-proxy.
+|-----------|-------------------|-------------|------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|------------------------|
+| pulsar | pulsar-mini-proxy | | http://127.0.0.1:61853 |
+| | | | http://127.0.0.1:61854 |
+|-----------|-------------------|-------------|------------------------|
```
-At this point, you can get the service URLs to connect to your Pulsar client.
+At this point, you can get the service URLs to connect to your Pulsar client. Here are URL examples:
```
-webServiceUrl=http://$(minikube ip):<exposed-http-port>/
-brokerServiceUrl=pulsar://$(minikube ip):<exposed-binary-port>/
+webServiceUrl=http://127.0.0.1:61853/
+brokerServiceUrl=pulsar://127.0.0.1:61854/
```
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.6.2/deploy-aws.md b/site2/website-next/versioned_docs/version-2.6.2/deploy-aws.md
index 4845db0..542f40b 100644
--- a/site2/website-next/versioned_docs/version-2.6.2/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.6.2/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.6.2/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.2/develop-binary-protocol.md
index 31b5b32..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.6.2/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.2/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -458,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.2/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.6.2/developing-binary-protocol.md
index 31b5b32..790ee88 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.2/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.6.2/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.2/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.6.2/getting-started-helm.md
index 1394086..ba59a23 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.2/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.6.2/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.2/kubernetes-helm.md
index 1394086..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.6.2/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.2/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
@@ -254,7 +257,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
You can use the Pulsar client to create producers and consumers to produce and consume messages.
-By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to get the IP address of the proxy service.
+By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to check the proxy service.
```bash
@@ -270,22 +273,42 @@ pulsar-mini-proxy LoadBalancer 10.97.240.109 <pending> 80:32
```
-This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are exposed to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are mapped to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+
+Then you can find the IP address and exposed ports of your Minikube server by running the following command.
+
+```bash
+
+minikube service pulsar-mini-proxy -n pulsar
+
+```
-Then you can find the IP address of your Minikube server by running the following command.
+**Output**
```bash
-minikube ip
+|-----------|-------------------|-------------|-------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|-------------------------|
+| pulsar | pulsar-mini-proxy | http/80 | http://172.17.0.4:32305 |
+| | | pulsar/6650 | http://172.17.0.4:31816 |
+|-----------|-------------------|-------------|-------------------------|
+🏃 Starting tunnel for service pulsar-mini-proxy.
+|-----------|-------------------|-------------|------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|------------------------|
+| pulsar | pulsar-mini-proxy | | http://127.0.0.1:61853 |
+| | | | http://127.0.0.1:61854 |
+|-----------|-------------------|-------------|------------------------|
```
-At this point, you can get the service URLs to connect to your Pulsar client.
+At this point, you can get the service URLs to connect to your Pulsar client. Here are URL examples:
```
-webServiceUrl=http://$(minikube ip):<exposed-http-port>/
-brokerServiceUrl=pulsar://$(minikube ip):<exposed-binary-port>/
+webServiceUrl=http://127.0.0.1:61853/
+brokerServiceUrl=pulsar://127.0.0.1:61854/
```
diff --git a/site2/website-next/versioned_docs/version-2.6.2/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.2/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.6.2/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.2/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.6.3/deploy-aws.md b/site2/website-next/versioned_docs/version-2.6.3/deploy-aws.md
index 4845db0..542f40b 100644
--- a/site2/website-next/versioned_docs/version-2.6.3/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.6.3/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.6.3/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.3/develop-binary-protocol.md
index 31b5b32..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.6.3/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.3/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -458,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.3/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.6.3/developing-binary-protocol.md
index 31b5b32..790ee88 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.3/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.3/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.6.3/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.3/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.3/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.6.3/getting-started-helm.md
index 1394086..ba59a23 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.3/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.6.3/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.3/kubernetes-helm.md
index 1394086..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.6.3/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.3/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
@@ -254,7 +257,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
You can use the Pulsar client to create producers and consumers to produce and consume messages.
-By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to get the IP address of the proxy service.
+By default, the Pulsar Helm chart exposes the Pulsar cluster through a Kubernetes `LoadBalancer`. In Minikube, you can use the following command to check the proxy service.
```bash
@@ -270,22 +273,42 @@ pulsar-mini-proxy LoadBalancer 10.97.240.109 <pending> 80:32
```
-This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are exposed to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+This output tells what are the node ports that Pulsar cluster's binary port and HTTP port are mapped to. The port after `80:` is the HTTP port while the port after `6650:` is the binary port.
+
+Then you can find the IP address and exposed ports of your Minikube server by running the following command.
+
+```bash
+
+minikube service pulsar-mini-proxy -n pulsar
+
+```
-Then you can find the IP address of your Minikube server by running the following command.
+**Output**
```bash
-minikube ip
+|-----------|-------------------|-------------|-------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|-------------------------|
+| pulsar | pulsar-mini-proxy | http/80 | http://172.17.0.4:32305 |
+| | | pulsar/6650 | http://172.17.0.4:31816 |
+|-----------|-------------------|-------------|-------------------------|
+🏃 Starting tunnel for service pulsar-mini-proxy.
+|-----------|-------------------|-------------|------------------------|
+| NAMESPACE | NAME | TARGET PORT | URL |
+|-----------|-------------------|-------------|------------------------|
+| pulsar | pulsar-mini-proxy | | http://127.0.0.1:61853 |
+| | | | http://127.0.0.1:61854 |
+|-----------|-------------------|-------------|------------------------|
```
-At this point, you can get the service URLs to connect to your Pulsar client.
+At this point, you can get the service URLs to connect to your Pulsar client. Here are URL examples:
```
-webServiceUrl=http://$(minikube ip):<exposed-http-port>/
-brokerServiceUrl=pulsar://$(minikube ip):<exposed-binary-port>/
+webServiceUrl=http://127.0.0.1:61853/
+brokerServiceUrl=pulsar://127.0.0.1:61854/
```
diff --git a/site2/website-next/versioned_docs/version-2.6.3/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.3/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.6.3/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.3/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.6.4/deploy-aws.md b/site2/website-next/versioned_docs/version-2.6.4/deploy-aws.md
index 7ae3bb0..071effef 100644
--- a/site2/website-next/versioned_docs/version-2.6.4/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.6.4/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.6.4/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.4/develop-binary-protocol.md
index 31b5b32..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.6.4/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.4/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
@@ -458,7 +458,7 @@ Topic lookup needs to be performed each time a client needs to create or
reconnect a producer or a consumer. Lookup is used to discover which particular
broker is serving the topic we are about to use.
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
+Lookup can be done with a REST call as described in the [admin API](admin-api-topics.md#lookup-of-topic)
docs.
Since Pulsar-1.16 it is also possible to perform the lookup within the binary
diff --git a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.6.4/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.6.4/developing-binary-protocol.md
index 31b5b32..790ee88 100644
--- a/site2/website-next/versioned_docs/version-2.6.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.6.4/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.4/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.6.4/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.4/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.6.4/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.4/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.6.4/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.6.4/getting-started-helm.md
index d412bbf..77cc3f4 100644
--- a/site2/website-next/versioned_docs/version-2.6.4/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.4/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.6.4/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.6.4/kubernetes-helm.md
index d412bbf..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.6.4/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.6.4/kubernetes-helm.md
@@ -63,39 +63,59 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-1. Clone the Pulsar Helm chart repository.
+1. Add Pulsar charts repo.
```bash
- git clone https://github.com/apache/pulsar
- cd deployment/kubernetes/helm/
+ helm repo add apache https://pulsar.apache.org/charts
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+ ```bash
+
+ helm repo update
+
+ ```
+
+2. Clone the Pulsar Helm chart repository.
+
+ ```bash
+
+ git clone https://github.com/apache/pulsar-helm-chart
+ cd pulsar-helm-chart
+
+ ```
+
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
./scripts/pulsar/prepare_helm_release.sh \
-n pulsar \
-k pulsar-mini \
- --control-center-admin pulsar \
- --control-center-password pulsar \
-c
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
helm install \
--values examples/values-minikube.yaml \
- pulsar-mini pulsar
+ --set initialize=true \
+ --namespace pulsar \
+ pulsar-mini apache/pulsar
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -123,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
@@ -155,7 +175,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```bash
- kubectl exec -it -n pulsar pulsar-mini-toolset-0 /bin/bash
+ kubectl exec -it -n pulsar pulsar-mini-toolset-0 -- /bin/bash
```
diff --git a/site2/website-next/versioned_docs/version-2.6.4/standalone-docker.md b/site2/website-next/versioned_docs/version-2.6.4/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.6.4/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.6.4/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.7.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.7.0/deploy-aws.md
index 6323051..1c8f1a0 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.7.0/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
index b233f10..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.0/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.7.0/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.0/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.7.0/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.0/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.7.0/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.7.0/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.0/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.0/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.7.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.0/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.0/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.7.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.7.1/deploy-aws.md
index 7ae3bb0..071effef 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.7.1/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.7.1/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.1/develop-binary-protocol.md
index b233f10..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.1/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.1/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.7.1/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.1/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.7.1/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.1/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.7.1/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.1/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.7.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.1/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.1/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.7.2/deploy-aws.md b/site2/website-next/versioned_docs/version-2.7.2/deploy-aws.md
index 6323051..1c8f1a0 100644
--- a/site2/website-next/versioned_docs/version-2.7.2/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.7.2/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.7.2/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.2/develop-binary-protocol.md
deleted file mode 100644
index 31b5b32..0000000
--- a/site2/website-next/versioned_docs/version-2.7.2/develop-binary-protocol.md
+++ /dev/null
@@ -1,581 +0,0 @@
----
-id: develop-binary-protocol
-title: Pulsar binary protocol specification
-sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
----
-
-Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
-
-Clients and brokers exchange *commands* with each other. Commands are formatted as binary [protocol buffer](https://developers.google.com/protocol-buffers/) (aka *protobuf*) messages. The format of protobuf commands is specified in the [`PulsarApi.proto`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto) file and also documented in the [Protobuf interface](#protobuf-interface) section below.
-
-> ### Connection sharing
-> Commands for different producers and consumers can be interleaved and sent through the same connection without restriction.
-
-All commands associated with Pulsar's protocol are contained in a [`BaseCommand`](#pulsar.proto.BaseCommand) protobuf message that includes a [`Type`](#pulsar.proto.Type) [enum](https://developers.google.com/protocol-buffers/docs/proto#enum) with all possible subcommands as optional fields. `BaseCommand` messages can specify only one subcommand.
-
-## Framing
-
-Since protobuf doesn't provide any sort of message frame, all messages in the Pulsar protocol are prepended with a 4-byte field that specifies the size of the frame. The maximum allowable size of a single frame is 5 MB.
-
-The Pulsar protocol allows for two types of commands:
-
-1. **Simple commands** that do not carry a message payload.
-2. **Payload commands** that bear a payload that is used when publishing or delivering messages. In payload commands, the protobuf command data is followed by protobuf [metadata](#message-metadata) and then the payload, which is passed in raw format outside of protobuf. All sizes are passed as 4-byte unsigned big endian integers.
-
-> Message payloads are passed in raw format rather than protobuf format for efficiency reasons.
-
-### Simple commands
-
-Simple (payload-free) commands have this basic structure:
-
-| Component | Description | Size (in bytes) |
-|:------------|:----------------------------------------------------------------------------------------|:----------------|
-| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| commandSize | The size of the protobuf-serialized command | 4 |
-| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-
-### Payload commands
-
-Payload commands have this basic structure:
-
-| Component | Description | Size (in bytes) |
-|:-------------|:--------------------------------------------------------------------------------------------|:----------------|
-| totalSize | The size of the frame, counting everything that comes after it (in bytes) | 4 |
-| commandSize | The size of the protobuf-serialized command | 4 |
-| message | The protobuf message serialized in a raw binary format (rather than in protobuf format) | |
-| magicNumber | A 2-byte byte array (`0x0e01`) identifying the current format | 2 |
-| checksum | A [CRC32-C checksum](http://www.evanjones.ca/crc32c.html) of everything that comes after it | 4 |
-| metadataSize | The size of the message [metadata](#message-metadata) | 4 |
-| metadata | The message [metadata](#message-metadata) stored as a binary protobuf message | |
-| payload | Anything left in the frame is considered the payload and can include any sequence of bytes | |
-
-## Message metadata
-
-Message metadata is stored alongside the application-specified payload as a serialized protobuf message. Metadata is created by the producer and passed on unchanged to the consumer.
-
-| Field | Description |
-|:-------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `producer_name` | The name of the producer that published the message |
-| `sequence_id` | The sequence ID of the message, assigned by producer |
-| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
-| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
-| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
-| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
-| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
-| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
-
-### Batch messages
-
-When using batch messages, the payload will be containing a list of entries,
-each of them with its individual metadata, defined by the `SingleMessageMetadata`
-object.
-
-
-For a single batch, the payload format will look like this:
-
-
-| Field | Description |
-|:--------------|:------------------------------------------------------------|
-| metadataSizeN | The size of the single message metadata serialized Protobuf |
-| metadataN | Single message metadata |
-| payloadN | Message payload passed by application |
-
-Each metadata field looks like this;
-
-| Field | Description |
-|:---------------------------|:--------------------------------------------------------|
-| properties | Application-defined properties |
-| partition key *(optional)* | Key to indicate the hashing to a particular partition |
-| payload_size | Size of the payload for the single message in the batch |
-
-When compression is enabled, the whole batch will be compressed at once.
-
-## Interactions
-
-### Connection establishment
-
-After opening a TCP connection to a broker, typically on port 6650, the client
-is responsible to initiate the session.
-
-
-
-After receiving a `Connected` response from the broker, the client can
-consider the connection ready to use. Alternatively, if the broker doesn't
-validate the client authentication, it will reply with an `Error` command and
-close the TCP connection.
-
-Example:
-
-```protobuf
-
-message CommandConnect {
- "client_version" : "Pulsar-Client-Java-v1.15.2",
- "auth_method_name" : "my-authentication-plugin",
- "auth_data" : "my-auth-data",
- "protocol_version" : 6
-}
-
-```
-
-Fields:
- * `client_version` → String based identifier. Format is not enforced
- * `auth_method_name` → *(optional)* Name of the authentication plugin if auth
- enabled
- * `auth_data` → *(optional)* Plugin specific authentication data
- * `protocol_version` → Indicates the protocol version supported by the
- client. Broker will not send commands introduced in newer revisions of the
- protocol. Broker might be enforcing a minimum version
-
-```protobuf
-
-message CommandConnected {
- "server_version" : "Pulsar-Broker-v1.15.2",
- "protocol_version" : 6
-}
-
-```
-
-Fields:
- * `server_version` → String identifier of broker version
- * `protocol_version` → Protocol version supported by the broker. Client
- must not attempt to send commands introduced in newer revisions of the
- protocol
-
-### Keep Alive
-
-To identify prolonged network partitions between clients and brokers or cases
-in which a machine crashes without interrupting the TCP connection on the remote
-end (eg: power outage, kernel panic, hard reboot...), we have introduced a
-mechanism to probe for the availability status of the remote peer.
-
-Both clients and brokers are sending `Ping` commands periodically and they will
-close the socket if a `Pong` response is not received within a timeout (default
-used by broker is 60s).
-
-A valid implementation of a Pulsar client is not required to send the `Ping`
-probe, though it is required to promptly reply after receiving one from the
-broker in order to prevent the remote side from forcibly closing the TCP connection.
-
-
-### Producer
-
-In order to send messages, a client needs to establish a producer. When creating
-a producer, the broker will first verify that this particular client is
-authorized to publish on the topic.
-
-Once the client gets confirmation of the producer creation, it can publish
-messages to the broker, referring to the producer id negotiated before.
-
-
-
-##### Command Producer
-
-```protobuf
-
-message CommandProducer {
- "topic" : "persistent://my-property/my-cluster/my-namespace/my-topic",
- "producer_id" : 1,
- "request_id" : 1
-}
-
-```
-
-Parameters:
- * `topic` → Complete topic name to where you want to create the producer on
- * `producer_id` → Client generated producer identifier. Needs to be unique
- within the same connection
- * `request_id` → Identifier for this request. Used to match the response with
- the originating request. Needs to be unique within the same connection
- * `producer_name` → *(optional)* If a producer name is specified, the name will
- be used, otherwise the broker will generate a unique name. Generated
- producer name is guaranteed to be globally unique. Implementations are
- expected to let the broker generate a new producer name when the producer
- is initially created, then reuse it when recreating the producer after
- reconnections.
-
-The broker will reply with either `ProducerSuccess` or `Error` commands.
-
-##### Command ProducerSuccess
-
-```protobuf
-
-message CommandProducerSuccess {
- "request_id" : 1,
- "producer_name" : "generated-unique-producer-name"
-}
-
-```
-
-Parameters:
- * `request_id` → Original id of the `CreateProducer` request
- * `producer_name` → Generated globally unique producer name or the name
- specified by the client, if any.
-
-##### Command Send
-
-Command `Send` is used to publish a new message within the context of an
-already existing producer. This command is used in a frame that includes command
-as well as message payload, for which the complete format is specified in the [payload commands](#payload-commands) section.
-
-```protobuf
-
-message CommandSend {
- "producer_id" : 1,
- "sequence_id" : 0,
- "num_messages" : 1
-}
-
-```
-
-Parameters:
- * `producer_id` → id of an existing producer
- * `sequence_id` → each message has an associated sequence id which is expected
- to be implemented with a counter starting at 0. The `SendReceipt` that
- acknowledges the effective publishing of a messages will refer to it by
- its sequence id.
- * `num_messages` → *(optional)* Used when publishing a batch of messages at
- once.
-
-##### Command SendReceipt
-
-After a message has been persisted on the configured number of replicas, the
-broker will send the acknowledgment receipt to the producer.
-
-```protobuf
-
-message CommandSendReceipt {
- "producer_id" : 1,
- "sequence_id" : 0,
- "message_id" : {
- "ledgerId" : 123,
- "entryId" : 456
- }
-}
-
-```
-
-Parameters:
- * `producer_id` → id of producer originating the send request
- * `sequence_id` → sequence id of the published message
- * `message_id` → message id assigned by the system to the published message
- Unique within a single cluster. Message id is composed of 2 longs, `ledgerId`
- and `entryId`, that reflect that this unique id is assigned when appending
- to a BookKeeper ledger
-
-
-##### Command CloseProducer
-
-**Note**: *This command can be sent by either producer or broker*.
-
-When receiving a `CloseProducer` command, the broker will stop accepting any
-more messages for the producer, wait until all pending messages are persisted
-and then reply `Success` to the client.
-
-The broker can send a `CloseProducer` command to client when it's performing
-a graceful failover (eg: broker is being restarted, or the topic is being unloaded
-by load balancer to be transferred to a different broker).
-
-When receiving the `CloseProducer`, the client is expected to go through the
-service discovery lookup again and recreate the producer again. The TCP
-connection is not affected.
-
-### Consumer
-
-A consumer is used to attach to a subscription and consume messages from it.
-After every reconnection, a client needs to subscribe to the topic. If a
-subscription is not already there, a new one will be created.
-
-
-
-#### Flow control
-
-After the consumer is ready, the client needs to *give permission* to the
-broker to push messages. This is done with the `Flow` command.
-
-A `Flow` command gives additional *permits* to send messages to the consumer.
-A typical consumer implementation will use a queue to accumulate these messages
-before the application is ready to consume them.
-
-After the application has dequeued half of the messages in the queue, the consumer
-sends permits to the broker to ask for more messages (equals to half of the messages in the queue).
-
-For example, if the queue size is 1000 and the consumer consumes 500 messages in the queue.
-Then the consumer sends permits to the broker to ask for 500 messages.
-
-##### Command Subscribe
-
-```protobuf
-
-message CommandSubscribe {
- "topic" : "persistent://my-property/my-cluster/my-namespace/my-topic",
- "subscription" : "my-subscription-name",
- "subType" : "Exclusive",
- "consumer_id" : 1,
- "request_id" : 1
-}
-
-```
-
-Parameters:
- * `topic` → Complete topic name to where you want to create the consumer on
- * `subscription` → Subscription name
- * `subType` → Subscription type: Exclusive, Shared, Failover, Key_Shared
- * `consumer_id` → Client generated consumer identifier. Needs to be unique
- within the same connection
- * `request_id` → Identifier for this request. Used to match the response with
- the originating request. Needs to be unique within the same connection
- * `consumer_name` → *(optional)* Clients can specify a consumer name. This
- name can be used to track a particular consumer in the stats. Also, in
- Failover subscription type, the name is used to decide which consumer is
- elected as *master* (the one receiving messages): consumers are sorted by
- their consumer name and the first one is elected master.
-
-##### Command Flow
-
-```protobuf
-
-message CommandFlow {
- "consumer_id" : 1,
- "messagePermits" : 1000
-}
-
-```
-
-Parameters:
-* `consumer_id` → Id of an already established consumer
-* `messagePermits` → Number of additional permits to grant to the broker for
- pushing more messages
-
-##### Command Message
-
-Command `Message` is used by the broker to push messages to an existing consumer,
-within the limits of the given permits.
-
-
-This command is used in a frame that includes the message payload as well, for
-which the complete format is specified in the [payload commands](#payload-commands)
-section.
-
-```protobuf
-
-message CommandMessage {
- "consumer_id" : 1,
- "message_id" : {
- "ledgerId" : 123,
- "entryId" : 456
- }
-}
-
-```
-
-##### Command Ack
-
-An `Ack` is used to signal to the broker that a given message has been
-successfully processed by the application and can be discarded by the broker.
-
-In addition, the broker will also maintain the consumer position based on the
-acknowledged messages.
-
-```protobuf
-
-message CommandAck {
- "consumer_id" : 1,
- "ack_type" : "Individual",
- "message_id" : {
- "ledgerId" : 123,
- "entryId" : 456
- }
-}
-
-```
-
-Parameters:
- * `consumer_id` → Id of an already established consumer
- * `ack_type` → Type of acknowledgment: `Individual` or `Cumulative`
- * `message_id` → Id of the message to acknowledge
- * `validation_error` → *(optional)* Indicates that the consumer has discarded
- the messages due to: `UncompressedSizeCorruption`,
- `DecompressionError`, `ChecksumMismatch`, `BatchDeSerializeError`
-
-##### Command CloseConsumer
-
-***Note***: **This command can be sent by either producer or broker*.
-
-This command behaves the same as [`CloseProducer`](#command-closeproducer)
-
-##### Command RedeliverUnacknowledgedMessages
-
-A consumer can ask the broker to redeliver some or all of the pending messages
-that were pushed to that particular consumer and not yet acknowledged.
-
-The protobuf object accepts a list of message ids that the consumer wants to
-be redelivered. If the list is empty, the broker will redeliver all the
-pending messages.
-
-On redelivery, messages can be sent to the same consumer or, in the case of a
-shared subscription, spread across all available consumers.
-
-
-##### Command ReachedEndOfTopic
-
-This is sent by a broker to a particular consumer, whenever the topic
-has been "terminated" and all the messages on the subscription were
-acknowledged.
-
-The client should use this command to notify the application that no more
-messages are coming from the consumer.
-
-##### Command ConsumerStats
-
-This command is sent by the client to retrieve Subscriber and Consumer level
-stats from the broker.
-Parameters:
- * `request_id` → Id of the request, used to correlate the request
- and the response.
- * `consumer_id` → Id of an already established consumer.
-
-##### Command ConsumerStatsResponse
-
-This is the broker's response to ConsumerStats request by the client.
-It contains the Subscriber and Consumer level stats of the `consumer_id` sent in the request.
-If the `error_code` or the `error_message` field is set it indicates that the request has failed.
-
-##### Command Unsubscribe
-
-This command is sent by the client to unsubscribe the `consumer_id` from the associated topic.
-Parameters:
- * `request_id` → Id of the request.
- * `consumer_id` → Id of an already established consumer which needs to unsubscribe.
-
-
-## Service discovery
-
-### Topic lookup
-
-Topic lookup needs to be performed each time a client needs to create or
-reconnect a producer or a consumer. Lookup is used to discover which particular
-broker is serving the topic we are about to use.
-
-Lookup can be done with a REST call as described in the [admin API](admin-api-persistent-topics.md#lookup-of-topic)
-docs.
-
-Since Pulsar-1.16 it is also possible to perform the lookup within the binary
-protocol.
-
-For the sake of example, let's assume we have a service discovery component
-running at `pulsar://broker.example.com:6650`
-
-Individual brokers will be running at `pulsar://broker-1.example.com:6650`,
-`pulsar://broker-2.example.com:6650`, ...
-
-A client can use a connection to the discovery service host to issue a
-`LookupTopic` command. The response can either be a broker hostname to
-connect to, or a broker hostname to which retry the lookup.
-
-The `LookupTopic` command has to be used in a connection that has already
-gone through the `Connect` / `Connected` initial handshake.
-
-
-
-```protobuf
-
-message CommandLookupTopic {
- "topic" : "persistent://my-property/my-cluster/my-namespace/my-topic",
- "request_id" : 1,
- "authoritative" : false
-}
-
-```
-
-Fields:
- * `topic` → Topic name to lookup
- * `request_id` → Id of the request that will be passed with its response
- * `authoritative` → Initial lookup request should use false. When following a
- redirect response, client should pass the same value contained in the
- response
-
-##### LookupTopicResponse
-
-Example of response with successful lookup:
-
-```protobuf
-
-message CommandLookupTopicResponse {
- "request_id" : 1,
- "response" : "Connect",
- "brokerServiceUrl" : "pulsar://broker-1.example.com:6650",
- "brokerServiceUrlTls" : "pulsar+ssl://broker-1.example.com:6651",
- "authoritative" : true
-}
-
-```
-
-Example of lookup response with redirection:
-
-```protobuf
-
-message CommandLookupTopicResponse {
- "request_id" : 1,
- "response" : "Redirect",
- "brokerServiceUrl" : "pulsar://broker-2.example.com:6650",
- "brokerServiceUrlTls" : "pulsar+ssl://broker-2.example.com:6651",
- "authoritative" : true
-}
-
-```
-
-In this second case, we need to reissue the `LookupTopic` command request
-to `broker-2.example.com` and this broker will be able to give a definitive
-answer to the lookup request.
-
-### Partitioned topics discovery
-
-Partitioned topics metadata discovery is used to find out if a topic is a
-"partitioned topic" and how many partitions were set up.
-
-If the topic is marked as "partitioned", the client is expected to create
-multiple producers or consumers, one for each partition, using the `partition-X`
-suffix.
-
-This information only needs to be retrieved the first time a producer or
-consumer is created. There is no need to do this after reconnections.
-
-The discovery of partitioned topics metadata works very similar to the topic
-lookup. The client send a request to the service discovery address and the
-response will contain actual metadata.
-
-##### Command PartitionedTopicMetadata
-
-```protobuf
-
-message CommandPartitionedTopicMetadata {
- "topic" : "persistent://my-property/my-cluster/my-namespace/my-topic",
- "request_id" : 1
-}
-
-```
-
-Fields:
- * `topic` → the topic for which to check the partitions metadata
- * `request_id` → Id of the request that will be passed with its response
-
-
-##### Command PartitionedTopicMetadataResponse
-
-Example of response with metadata:
-
-```protobuf
-
-message CommandPartitionedTopicMetadataResponse {
- "request_id" : 1,
- "response" : "Success",
- "partitions" : 32
-}
-
-```
-
-## Protobuf interface
-
-All Pulsar's Protobuf definitions can be found {@inject: github:here:/pulsar-common/src/main/proto/PulsarApi.proto}.
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.2/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.7.2/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.2/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.7.2/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.2/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.7.2/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.2/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.7.2/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.2/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.7.2/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.2/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.7.2/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.2/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.7.2/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.2/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.7.3/deploy-aws.md b/site2/website-next/versioned_docs/version-2.7.3/deploy-aws.md
index 6323051..1c8f1a0 100644
--- a/site2/website-next/versioned_docs/version-2.7.3/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.7.3/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.7.3/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.3/develop-binary-protocol.md
index b233f10..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.7.3/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.3/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.3/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.7.3/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.3/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.3/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.7.3/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.3/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.3/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.7.3/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.3/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.7.3/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.3/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.7.3/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.3/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.7.3/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.3/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.7.3/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.3/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.7.4/deploy-aws.md b/site2/website-next/versioned_docs/version-2.7.4/deploy-aws.md
index 6323051..1c8f1a0 100644
--- a/site2/website-next/versioned_docs/version-2.7.4/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.7.4/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.7.4/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.4/develop-binary-protocol.md
index b233f10..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.7.4/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.4/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.7.4/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.7.4/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.7.4/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.4/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.7.4/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.4/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.4/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.7.4/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.4/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.7.4/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.7.4/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.7.4/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.7.4/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.7.4/standalone-docker.md b/site2/website-next/versioned_docs/version-2.7.4/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.7.4/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.7.4/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.8.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.8.0/deploy-aws.md
index 7ae3bb0..071effef 100644
--- a/site2/website-next/versioned_docs/version-2.8.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.8.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.8.0/develop-binary-protocol.md
index b233f10..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.8.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.8.0/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.8.0/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.8.0/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.8.0/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.8.0/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.8.0/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.8.0/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.8.0/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.8.0/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.8.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.8.0/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.8.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.8.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.8.1/deploy-aws.md
index 7ae3bb0..071effef 100644
--- a/site2/website-next/versioned_docs/version-2.8.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.8.1/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.8.1/develop-binary-protocol.md
index b233f10..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.8.1/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.8.1/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.8.1/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.8.1/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.8.1/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.8.1/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.8.1/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.8.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.8.1/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.8.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.8.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.8.1/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.8.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.8.2/deploy-aws.md b/site2/website-next/versioned_docs/version-2.8.2/deploy-aws.md
index 6323051..1c8f1a0 100644
--- a/site2/website-next/versioned_docs/version-2.8.2/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.8.2/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.8.2/develop-binary-protocol.md
index b233f10..74ef751 100644
--- a/site2/website-next/versioned_docs/version-2.8.2/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/develop-binary-protocol.md
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.8.2/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.8.2/developing-binary-protocol.md
index b233f10..37be0d4 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md b/site2/website-next/versioned_docs/version-2.8.2/getting-started-docker.md
similarity index 98%
copy from site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.8.2/getting-started-docker.md
index 05ac2a1..02ca2e9 100644
--- a/site2/website-next/versioned_docs/version-2.6.1/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.8.2/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.8.2/getting-started-helm.md
index bbbd307..3f21e80 100644
--- a/site2/website-next/versioned_docs/version-2.7.1/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.8.2/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.8.2/kubernetes-helm.md
index bbbd307..403d975 100644
--- a/site2/website-next/versioned_docs/version-2.8.2/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/kubernetes-helm.md
@@ -63,7 +63,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
@@ -77,7 +77,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
@@ -86,7 +86,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
@@ -97,10 +97,13 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
- > You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+ :::note
+
+ You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
+
+ :::
```bash
@@ -112,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
@@ -140,7 +143,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
diff --git a/site2/website-next/versioned_docs/version-2.8.2/standalone-docker.md b/site2/website-next/versioned_docs/version-2.8.2/standalone-docker.md
index 05ac2a1..d02efd2 100644
--- a/site2/website-next/versioned_docs/version-2.8.2/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/standalone-docker.md
@@ -5,11 +5,9 @@ sidebar_label: "Run Pulsar in Docker"
original_id: standalone-docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -17,13 +15,7 @@ and follow the instructions for your OS.
```shell
- $ docker run -it \
- -p 6650:6650 \
- -p 8080:8080 \
- --mount source=pulsardata,target=/pulsar/data \
- --mount source=pulsarconf,target=/pulsar/conf \
- apachepulsar/pulsar:@pulsar:version@ \
- bin/pulsar standalone
+ $ docker run -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:@pulsar:version@ bin/pulsar standalone
```
@@ -36,8 +28,9 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
```
-2017-08-09 22:34:04,030 - INFO - [main:WebService@213] - Web Service started at http://127.0.0.1:8080
-2017-08-09 22:34:04,038 - INFO - [main:PulsarService@335] - messaging service is ready, bootstrap service on port=8080, broker url=pulsar://127.0.0.1:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@4db60246
+08:18:30.970 [main] INFO org.apache.pulsar.broker.web.WebService - HTTP Service started at http://0.0.0.0:8080
+...
+07:53:37.322 [main] INFO org.apache.pulsar.broker.PulsarService - messaging service is ready, bootstrap service port = 8080, broker url= pulsar://localhost:6650, cluster=standalone, configs=org.apache.pulsar.broker.ServiceConfiguration@98b63c1
...
```
@@ -60,7 +53,7 @@ use one of these root URLs to interact with your cluster:
* `pulsar://localhost:6650`
* `http://localhost:8080`
-The following example will guide you get started with Pulsar quickly by using the [Python](client-libraries-python)
+The following example will guide you get started with Pulsar quickly by using the [Python client API](client-libraries-python)
client API.
Install the Pulsar Python client library directly from [PyPI](https://pypi.org/project/pulsar-client/):
@@ -128,51 +121,93 @@ The output is something like this:
```json
{
- "averageMsgSize": 0.0,
- "msgRateIn": 0.0,
- "msgRateOut": 0.0,
- "msgThroughputIn": 0.0,
- "msgThroughputOut": 0.0,
- "publishers": [
- {
- "address": "/172.17.0.1:35048",
- "averageMsgSize": 0.0,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 20:59:34.621+0000",
- "msgRateIn": 0.0,
- "msgThroughputIn": 0.0,
- "producerId": 0,
- "producerName": "standalone-0-1"
- }
- ],
- "replication": {},
- "storageSize": 16,
- "subscriptions": {
- "my-sub": {
- "blockedSubscriptionOnUnackedMsgs": false,
- "consumers": [
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesInCounter": 7097,
+ "msgInCounter": 143,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "averageMsgSize": 0.0,
+ "msgChunkPublished": false,
+ "storageSize": 7097,
+ "backlogSize": 0,
+ "offloadedStorageSize": 0,
+ "publishers": [
{
- "address": "/172.17.0.1:35064",
- "availablePermits": 996,
- "blockedConsumerOnUnackedMsgs": false,
- "clientVersion": "1.19.0-incubating",
- "connectedSince": "2017-08-09 21:05:39.222+0000",
- "consumerName": "166111",
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "unackedMessages": 0
+ "accessMode": "Shared",
+ "msgRateIn": 0.0,
+ "msgThroughputIn": 0.0,
+ "averageMsgSize": 0.0,
+ "chunkedMessageRate": 0.0,
+ "producerId": 0,
+ "metadata": {},
+ "address": "/127.0.0.1:35604",
+ "connectedSince": "2021-07-04T09:05:43.04788Z",
+ "clientVersion": "2.8.0",
+ "producerName": "standalone-2-5"
+ }
+ ],
+ "waitingPublishers": 0,
+ "subscriptions": {
+ "my-sub": {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0,
+ "msgBacklog": 0,
+ "backlogSize": 0,
+ "msgBacklogNoDelayed": 0,
+ "blockedSubscriptionOnUnackedMsgs": false,
+ "msgDelayed": 0,
+ "unackedMessages": 0,
+ "type": "Exclusive",
+ "activeConsumerName": "3c544f1daa",
+ "msgRateExpired": 0.0,
+ "totalMsgExpired": 0,
+ "lastExpireTimestamp": 0,
+ "lastConsumedFlowTimestamp": 1625389101290,
+ "lastConsumedTimestamp": 1625389546070,
+ "lastAckedTimestamp": 1625389546162,
+ "lastMarkDeleteAdvancedTimestamp": 1625389546163,
+ "consumers": [
+ {
+ "msgRateOut": 1.8332950480217471,
+ "msgThroughputOut": 91.33142602871978,
+ "bytesOutCounter": 6607,
+ "msgOutCounter": 133,
+ "msgRateRedeliver": 0.0,
+ "chunkedMessageRate": 0.0,
+ "consumerName": "3c544f1daa",
+ "availablePermits": 867,
+ "unackedMessages": 0,
+ "avgMessagesPerEntry": 6,
+ "blockedConsumerOnUnackedMsgs": false,
+ "lastAckedTimestamp": 1625389546162,
+ "lastConsumedTimestamp": 1625389546070,
+ "metadata": {},
+ "address": "/127.0.0.1:35472",
+ "connectedSince": "2021-07-04T08:58:21.287682Z",
+ "clientVersion": "2.8.0"
+ }
+ ],
+ "isDurable": true,
+ "isReplicated": false,
+ "allowOutOfOrderDelivery": false,
+ "consumersAfterMarkDeletePosition": {},
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0,
+ "durable": true,
+ "replicated": false
}
- ],
- "msgBacklog": 0,
- "msgRateExpired": 0.0,
- "msgRateOut": 0.0,
- "msgRateRedeliver": 0.0,
- "msgThroughputOut": 0.0,
- "type": "Exclusive",
- "unackedMessages": 0
- }
- }
+ },
+ "replication": {},
+ "deduplicationStatus": "Disabled",
+ "nonContiguousDeletedMessagesRanges": 0,
+ "nonContiguousDeletedMessagesRangesSerializedSize": 0
}
```
diff --git a/site2/website-next/versioned_docs/version-2.9.0/deploy-aws.md b/site2/website-next/versioned_docs/version-2.9.0/deploy-aws.md
index 7ae3bb0..071effef 100644
--- a/site2/website-next/versioned_docs/version-2.9.0/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.9.0/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.9.0/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.9.0/developing-binary-protocol.md
index b233f10..6a6b385 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.9.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.9.0/getting-started-docker.md
similarity index 95%
copy from site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.9.0/getting-started-docker.md
index f636d2d..d6c6bd7 100644
--- a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.9.0/getting-started-docker.md
@@ -1,7 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.9.0/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.9.0/getting-started-helm.md
index d435371..7bc9fa5 100644
--- a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.9.0/getting-started-helm.md
@@ -1,7 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website-next/versioned_docs/version-2.9.1/deploy-aws.md b/site2/website-next/versioned_docs/version-2.9.1/deploy-aws.md
index 7ae3bb0..071effef 100644
--- a/site2/website-next/versioned_docs/version-2.9.1/deploy-aws.md
+++ b/site2/website-next/versioned_docs/version-2.9.1/deploy-aws.md
@@ -35,7 +35,7 @@ $ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md b/site2/website-next/versioned_docs/version-2.9.1/developing-binary-protocol.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
copy to site2/website-next/versioned_docs/version-2.9.1/developing-binary-protocol.md
index b233f10..6a6b385 100644
--- a/site2/website-next/versioned_docs/version-2.7.0/develop-binary-protocol.md
+++ b/site2/website-next/versioned_docs/version-2.9.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: develop-binary-protocol
+id: developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: "Binary protocol"
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
@@ -61,7 +61,7 @@ Message metadata is stored alongside the application-specified payload as a seri
| `publish_time` | The publish timestamp in Unix time (i.e. as the number of milliseconds since January 1st, 1970 in UTC) |
| `properties` | A sequence of key/value pairs (using the [`KeyValue`](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/proto/PulsarApi.proto#L32) message). These are application-defined keys and values with no special meaning to Pulsar. |
| `replicated_from` *(optional)* | Indicates that the message has been replicated and specifies the name of the [cluster](reference-terminology.md#cluster) where the message was originally published |
-| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose |
+| `partition_key` *(optional)* | While publishing on a partition topic, if the key is present, the hash of the key is used to determine which partition to choose. Partition key is used as the message key. |
| `compression` *(optional)* | Signals that payload has been compressed and with which compression library |
| `uncompressed_size` *(optional)* | If compression is used, the producer must fill the uncompressed size field with the original payload size |
| `num_messages_in_batch` *(optional)* | If this message is really a [batch](#batch-messages) of multiple entries, this field must be set to the number of messages in the batch |
diff --git a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md b/site2/website-next/versioned_docs/version-2.9.1/getting-started-docker.md
similarity index 95%
copy from site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
copy to site2/website-next/versioned_docs/version-2.9.1/getting-started-docker.md
index f636d2d..d6c6bd7 100644
--- a/site2/website-next/versioned_docs/version-2.4.0/standalone-docker.md
+++ b/site2/website-next/versioned_docs/version-2.9.1/getting-started-docker.md
@@ -1,7 +1,8 @@
---
-id: standalone-docker
+id: getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: "Run Pulsar in Docker"
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
@@ -22,7 +23,6 @@ A few things to note about this command:
* The data, metadata, and configuration are persisted on Docker volumes in order to not start "fresh" every
time the container is restarted. For details on the volumes you can use `docker volume inspect <sourcename>`
* For Docker on Windows make sure to configure it to use Linux containers
- * The docker container will run as UID 10000 and GID 0, by default. You'll need to ensure the mounted volumes give write permission to either UID 10000 or GID 0. Note that UID 10000 is arbitrary, so it is recommended to make these mounts writable for the root group (GID 0).
If you start Pulsar successfully, you will see `INFO`-level log messages like this:
diff --git a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md b/site2/website-next/versioned_docs/version-2.9.1/getting-started-helm.md
similarity index 99%
copy from site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
copy to site2/website-next/versioned_docs/version-2.9.1/getting-started-helm.md
index d435371..7bc9fa5 100644
--- a/site2/website-next/versioned_docs/version-2.5.2/kubernetes-helm.md
+++ b/site2/website-next/versioned_docs/version-2.9.1/getting-started-helm.md
@@ -1,7 +1,8 @@
---
-id: kubernetes-helm
+id: getting-started-helm
title: Get started in Kubernetes
sidebar_label: "Run Pulsar in Kubernetes"
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.1.0-incubating/deploy-aws.md b/site2/website/versioned_docs/version-2.1.0-incubating/deploy-aws.md
index 01ef6d1..4010b12 100644
--- a/site2/website/versioned_docs/version-2.1.0-incubating/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.1.0-incubating/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You'll also need to have the Terraform and Ansible configurations for Pulsar locally on your machine. They're contained in Pulsar's [GitHub repository](https://github.com/apache/incubator-pulsar), which you can fetch using Git:
diff --git a/site2/website/versioned_docs/version-2.1.0-incubating/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.1.0-incubating/developing-binary-protocol.md
index ffcadbb..962c286 100644
--- a/site2/website/versioned_docs/version-2.1.0-incubating/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.1.0-incubating/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.1.0-incubating-develop-binary-protocol
+id: version-2.1.0-incubating-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-docker.md b/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-docker.md
index 1401df3..12edbe6 100644
--- a/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.1.0-incubating/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.1.0-incubating-standalone-docker
+id: version-2.1.0-incubating-getting-started-docker
title: Start a standalone cluster with Docker
sidebar_label: Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For the purposes of local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.2.0/deploy-aws.md b/site2/website/versioned_docs/version-2.2.0/deploy-aws.md
index eea3857..b4e87cc 100644
--- a/site2/website/versioned_docs/version-2.2.0/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.2.0/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You'll also need to have the Terraform and Ansible configurations for Pulsar locally on your machine. They're contained in Pulsar's [GitHub repository](https://github.com/apache/pulsar), which you can fetch using Git:
diff --git a/site2/website/versioned_docs/version-2.2.0/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.2.0/developing-binary-protocol.md
index 4b33be8..320504d 100644
--- a/site2/website/versioned_docs/version-2.2.0/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.2.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.2.0-develop-binary-protocol
+id: version-2.2.0-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.2.1/getting-started-docker.md b/site2/website/versioned_docs/version-2.2.1/getting-started-docker.md
index d1123a4..466cbee 100644
--- a/site2/website/versioned_docs/version-2.2.1/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.2.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.2.1-standalone-docker
+id: version-2.2.1-getting-started-docker
title: Start a standalone cluster with Docker
sidebar_label: Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For the purposes of local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.3.1/getting-started-docker.md b/site2/website/versioned_docs/version-2.3.1/getting-started-docker.md
index ba09309..6409d05 100644
--- a/site2/website/versioned_docs/version-2.3.1/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.3.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.3.1-standalone-docker
+id: version-2.3.1-getting-started-docker
title: Start a standalone cluster with Docker
sidebar_label: Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.3.2/getting-started-docker.md b/site2/website/versioned_docs/version-2.3.2/getting-started-docker.md
index d6de5ad..0f88a8f 100644
--- a/site2/website/versioned_docs/version-2.3.2/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.3.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.3.2-standalone-docker
+id: version-2.3.2-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.4.0/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.4.0/developing-binary-protocol.md
index 0510f78..dbc6575 100644
--- a/site2/website/versioned_docs/version-2.4.0/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.4.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.4.0-develop-binary-protocol
+id: version-2.4.0-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.5.0/deploy-aws.md b/site2/website/versioned_docs/version-2.5.0/deploy-aws.md
index 9ff9de4..80485d4 100644
--- a/site2/website/versioned_docs/version-2.5.0/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.5.0/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.5.0/getting-started-docker.md b/site2/website/versioned_docs/version-2.5.0/getting-started-docker.md
index 36707ef..7c2f91e 100644
--- a/site2/website/versioned_docs/version-2.5.0/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.5.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.5.0-standalone-docker
+id: version-2.5.0-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.5.0/getting-started-helm.md b/site2/website/versioned_docs/version-2.5.0/getting-started-helm.md
index 076c1b9..8831ffc 100644
--- a/site2/website/versioned_docs/version-2.5.0/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.5.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.5.0-kubernetes-helm
+id: version-2.5.0-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running
diff --git a/site2/website/versioned_docs/version-2.5.1/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.5.1/developing-binary-protocol.md
index 5fc09c2..b1ae445 100644
--- a/site2/website/versioned_docs/version-2.5.1/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.5.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.5.1-develop-binary-protocol
+id: version-2.5.1-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.5.1/kubernetes-helm.md b/site2/website/versioned_docs/version-2.5.1/kubernetes-helm.md
index 7889617..4d69a39 100644
--- a/site2/website/versioned_docs/version-2.5.1/kubernetes-helm.md
+++ b/site2/website/versioned_docs/version-2.5.1/kubernetes-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.5.1-kubernetes-helm
+id: version-2.5.1-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.5.1/standalone-docker.md b/site2/website/versioned_docs/version-2.5.1/standalone-docker.md
index d47a8dd..ce16776 100644
--- a/site2/website/versioned_docs/version-2.5.1/standalone-docker.md
+++ b/site2/website/versioned_docs/version-2.5.1/standalone-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.5.1-standalone-docker
+id: version-2.5.1-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.5.2/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.5.2/developing-binary-protocol.md
index 3b78e7b..f40c562 100644
--- a/site2/website/versioned_docs/version-2.5.2/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.5.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.5.2-develop-binary-protocol
+id: version-2.5.2-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.6.0/deploy-aws.md b/site2/website/versioned_docs/version-2.6.0/deploy-aws.md
index bcef265..bc406c5 100644
--- a/site2/website/versioned_docs/version-2.6.0/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.6.0/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.6.0/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.6.0/developing-binary-protocol.md
index 88fd9e1..eb001b3 100644
--- a/site2/website/versioned_docs/version-2.6.0/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.6.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.0-develop-binary-protocol
+id: version-2.6.0-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.6.0/getting-started-docker.md b/site2/website/versioned_docs/version-2.6.0/getting-started-docker.md
index ccb7a12..9cb280c 100644
--- a/site2/website/versioned_docs/version-2.6.0/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.6.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.0-standalone-docker
+id: version-2.6.0-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.6.0/getting-started-helm.md b/site2/website/versioned_docs/version-2.6.0/getting-started-helm.md
index 1350638..151cb2d 100644
--- a/site2/website/versioned_docs/version-2.6.0/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.6.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.0-kubernetes-helm
+id: version-2.6.0-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.6.1/deploy-aws.md b/site2/website/versioned_docs/version-2.6.1/deploy-aws.md
index 819ef52..edab0e4 100644
--- a/site2/website/versioned_docs/version-2.6.1/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.6.1/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.6.1/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.6.1/developing-binary-protocol.md
index 3be647d..98ce91a 100644
--- a/site2/website/versioned_docs/version-2.6.1/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.6.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.1-develop-binary-protocol
+id: version-2.6.1-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.6.1/getting-started-docker.md b/site2/website/versioned_docs/version-2.6.1/getting-started-docker.md
index aeb561d..e6142a7 100644
--- a/site2/website/versioned_docs/version-2.6.1/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.6.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.1-standalone-docker
+id: version-2.6.1-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.6.1/getting-started-helm.md b/site2/website/versioned_docs/version-2.6.1/getting-started-helm.md
index 8c19d5d..ca3f81c 100644
--- a/site2/website/versioned_docs/version-2.6.1/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.6.1/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.1-kubernetes-helm
+id: version-2.6.1-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.6.2/deploy-aws.md b/site2/website/versioned_docs/version-2.6.2/deploy-aws.md
index 8212048..3212c23 100644
--- a/site2/website/versioned_docs/version-2.6.2/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.6.2/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.6.2/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.6.2/developing-binary-protocol.md
index 4464309..8dfeaec 100644
--- a/site2/website/versioned_docs/version-2.6.2/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.6.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.2-develop-binary-protocol
+id: version-2.6.2-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.6.2/getting-started-docker.md b/site2/website/versioned_docs/version-2.6.2/getting-started-docker.md
index 8095c12..b3ea394 100644
--- a/site2/website/versioned_docs/version-2.6.2/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.6.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.2-standalone-docker
+id: version-2.6.2-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.6.2/getting-started-helm.md b/site2/website/versioned_docs/version-2.6.2/getting-started-helm.md
index e9d2d14..18e61d7 100644
--- a/site2/website/versioned_docs/version-2.6.2/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.6.2/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.2-kubernetes-helm
+id: version-2.6.2-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.6.3/deploy-aws.md b/site2/website/versioned_docs/version-2.6.3/deploy-aws.md
index 6559c65..926f881 100644
--- a/site2/website/versioned_docs/version-2.6.3/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.6.3/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.6.3/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.6.3/developing-binary-protocol.md
index 161ba8c..481b40c 100644
--- a/site2/website/versioned_docs/version-2.6.3/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.6.3/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.3-develop-binary-protocol
+id: version-2.6.3-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.6.3/getting-started-docker.md b/site2/website/versioned_docs/version-2.6.3/getting-started-docker.md
index 2121526..eaa2864 100644
--- a/site2/website/versioned_docs/version-2.6.3/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.6.3/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.3-standalone-docker
+id: version-2.6.3-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.6.3/getting-started-helm.md b/site2/website/versioned_docs/version-2.6.3/getting-started-helm.md
index 465017c..55ac150 100644
--- a/site2/website/versioned_docs/version-2.6.3/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.6.3/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.3-kubernetes-helm
+id: version-2.6.3-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.6.4/deploy-aws.md b/site2/website/versioned_docs/version-2.6.4/deploy-aws.md
index 6791794..251e2ae 100644
--- a/site2/website/versioned_docs/version-2.6.4/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.6.4/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.6.4/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.6.4/developing-binary-protocol.md
index 6cb7856..1e49dee 100644
--- a/site2/website/versioned_docs/version-2.6.4/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.6.4/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.4-develop-binary-protocol
+id: version-2.6.4-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.6.4/getting-started-docker.md b/site2/website/versioned_docs/version-2.6.4/getting-started-docker.md
index 7c08a5e..f9b2e99 100644
--- a/site2/website/versioned_docs/version-2.6.4/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.6.4/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.4-standalone-docker
+id: version-2.6.4-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.6.4/getting-started-helm.md b/site2/website/versioned_docs/version-2.6.4/getting-started-helm.md
index 0a7e6af..8450ae6 100644
--- a/site2/website/versioned_docs/version-2.6.4/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.6.4/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.6.4-kubernetes-helm
+id: version-2.6.4-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.7.0/deploy-aws.md b/site2/website/versioned_docs/version-2.7.0/deploy-aws.md
index 3b4d2c8..48806ea 100644
--- a/site2/website/versioned_docs/version-2.7.0/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.7.0/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.7.0/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.7.0/developing-binary-protocol.md
index 5f78db2..b8f9ba9 100644
--- a/site2/website/versioned_docs/version-2.7.0/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.7.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.0-develop-binary-protocol
+id: version-2.7.0-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.7.0/getting-started-docker.md b/site2/website/versioned_docs/version-2.7.0/getting-started-docker.md
index fe4f9a6..7fecb69 100644
--- a/site2/website/versioned_docs/version-2.7.0/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.7.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.0-standalone-docker
+id: version-2.7.0-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.7.0/getting-started-helm.md b/site2/website/versioned_docs/version-2.7.0/getting-started-helm.md
index e507682..a13c7af 100644
--- a/site2/website/versioned_docs/version-2.7.0/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.7.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.0-kubernetes-helm
+id: version-2.7.0-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.7.1/deploy-aws.md b/site2/website/versioned_docs/version-2.7.1/deploy-aws.md
index 14f8ac7..a149792 100644
--- a/site2/website/versioned_docs/version-2.7.1/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.7.1/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.7.1/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.7.1/developing-binary-protocol.md
index c6acb15..1ada2cf 100644
--- a/site2/website/versioned_docs/version-2.7.1/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.7.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.1-develop-binary-protocol
+id: version-2.7.1-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.7.1/getting-started-docker.md b/site2/website/versioned_docs/version-2.7.1/getting-started-docker.md
index e702664..66c07f3 100644
--- a/site2/website/versioned_docs/version-2.7.1/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.7.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.1-standalone-docker
+id: version-2.7.1-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.7.1/getting-started-helm.md b/site2/website/versioned_docs/version-2.7.1/getting-started-helm.md
index 1a8f90b..fc8286c 100644
--- a/site2/website/versioned_docs/version-2.7.1/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.7.1/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.1-kubernetes-helm
+id: version-2.7.1-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.7.2/deploy-aws.md b/site2/website/versioned_docs/version-2.7.2/deploy-aws.md
index 839addd..e12c51c 100644
--- a/site2/website/versioned_docs/version-2.7.2/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.7.2/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.7.2/develop-binary-protocol.md b/site2/website/versioned_docs/version-2.7.2/develop-binary-protocol.md
index 165fe826..89dbc1c 100644
--- a/site2/website/versioned_docs/version-2.7.2/develop-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.7.2/develop-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.2-develop-binary-protocol
+id: version-2.7.2-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.7.2/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.7.2/developing-binary-protocol.md
index 0792880..25caf1e 100644
--- a/site2/website/versioned_docs/version-2.7.2/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.7.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.2-develop-binary-protocol
+id: version-2.7.2-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.7.2/getting-started-docker.md b/site2/website/versioned_docs/version-2.7.2/getting-started-docker.md
index f63c047..03ad2c2 100644
--- a/site2/website/versioned_docs/version-2.7.2/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.7.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.2-standalone-docker
+id: version-2.7.2-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.7.2/getting-started-helm.md b/site2/website/versioned_docs/version-2.7.2/getting-started-helm.md
index 8f68671..741f79d 100644
--- a/site2/website/versioned_docs/version-2.7.2/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.7.2/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.2-kubernetes-helm
+id: version-2.7.2-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.7.3/deploy-aws.md b/site2/website/versioned_docs/version-2.7.3/deploy-aws.md
index 421e809..ba419b9 100644
--- a/site2/website/versioned_docs/version-2.7.3/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.7.3/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.7.3/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.7.3/developing-binary-protocol.md
index 9c79a57..9e6ab9c 100644
--- a/site2/website/versioned_docs/version-2.7.3/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.7.3/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.3-develop-binary-protocol
+id: version-2.7.3-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.7.3/getting-started-docker.md b/site2/website/versioned_docs/version-2.7.3/getting-started-docker.md
index ffb7409..69b3b91 100644
--- a/site2/website/versioned_docs/version-2.7.3/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.7.3/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.3-standalone-docker
+id: version-2.7.3-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.7.3/getting-started-helm.md b/site2/website/versioned_docs/version-2.7.3/getting-started-helm.md
index e574516..3621478 100644
--- a/site2/website/versioned_docs/version-2.7.3/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.7.3/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.3-kubernetes-helm
+id: version-2.7.3-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.7.4/deploy-aws.md b/site2/website/versioned_docs/version-2.7.4/deploy-aws.md
index f07e7b8..747fb51 100644
--- a/site2/website/versioned_docs/version-2.7.4/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.7.4/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.7.4/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.7.4/developing-binary-protocol.md
index 30cb728..50a5619 100644
--- a/site2/website/versioned_docs/version-2.7.4/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.7.4/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.4-develop-binary-protocol
+id: version-2.7.4-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.7.4/getting-started-docker.md b/site2/website/versioned_docs/version-2.7.4/getting-started-docker.md
index dfccb71..982aab5 100644
--- a/site2/website/versioned_docs/version-2.7.4/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.7.4/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.4-standalone-docker
+id: version-2.7.4-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.7.4/getting-started-helm.md b/site2/website/versioned_docs/version-2.7.4/getting-started-helm.md
index 56bd6c9..5d5ec05 100644
--- a/site2/website/versioned_docs/version-2.7.4/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.7.4/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.7.4-kubernetes-helm
+id: version-2.7.4-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.8.0/deploy-aws.md b/site2/website/versioned_docs/version-2.8.0/deploy-aws.md
index 68c5840..ec93c09 100644
--- a/site2/website/versioned_docs/version-2.8.0/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.8.0/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.8.0/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.8.0/developing-binary-protocol.md
index 8cd4139..1f59621 100644
--- a/site2/website/versioned_docs/version-2.8.0/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.8.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.0-develop-binary-protocol
+id: version-2.8.0-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.8.0/getting-started-docker.md b/site2/website/versioned_docs/version-2.8.0/getting-started-docker.md
index 12ca3e1..68939e5 100644
--- a/site2/website/versioned_docs/version-2.8.0/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.8.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.0-standalone-docker
+id: version-2.8.0-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.8.0/getting-started-helm.md b/site2/website/versioned_docs/version-2.8.0/getting-started-helm.md
index 95b10e0..02addd1 100644
--- a/site2/website/versioned_docs/version-2.8.0/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.8.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.0-kubernetes-helm
+id: version-2.8.0-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.8.1/deploy-aws.md b/site2/website/versioned_docs/version-2.8.1/deploy-aws.md
index 2ebe7df..4121be6 100644
--- a/site2/website/versioned_docs/version-2.8.1/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.8.1/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.8.1/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.8.1/developing-binary-protocol.md
index 2fc88f9..6857a7f 100644
--- a/site2/website/versioned_docs/version-2.8.1/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.8.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.1-develop-binary-protocol
+id: version-2.8.1-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.8.1/getting-started-docker.md b/site2/website/versioned_docs/version-2.8.1/getting-started-docker.md
index 092847a..d5bbb04 100644
--- a/site2/website/versioned_docs/version-2.8.1/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.8.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.1-standalone-docker
+id: version-2.8.1-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.8.1/getting-started-helm.md b/site2/website/versioned_docs/version-2.8.1/getting-started-helm.md
index 4494ccd..b18425b 100644
--- a/site2/website/versioned_docs/version-2.8.1/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.8.1/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.1-kubernetes-helm
+id: version-2.8.1-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.8.2/deploy-aws.md b/site2/website/versioned_docs/version-2.8.2/deploy-aws.md
index 0da6380..8884d75 100644
--- a/site2/website/versioned_docs/version-2.8.2/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.8.2/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.8.2/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.8.2/developing-binary-protocol.md
index 9f15273..2844185 100644
--- a/site2/website/versioned_docs/version-2.8.2/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.8.2/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.2-develop-binary-protocol
+id: version-2.8.2-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.8.2/getting-started-docker.md b/site2/website/versioned_docs/version-2.8.2/getting-started-docker.md
index 3654bb5..45b7ddd 100644
--- a/site2/website/versioned_docs/version-2.8.2/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.8.2/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.2-standalone-docker
+id: version-2.8.2-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.8.2/getting-started-helm.md b/site2/website/versioned_docs/version-2.8.2/getting-started-helm.md
index cba2e81..181209d 100644
--- a/site2/website/versioned_docs/version-2.8.2/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.8.2/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.2-kubernetes-helm
+id: version-2.8.2-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.8.3/deploy-aws.md b/site2/website/versioned_docs/version-2.8.3/deploy-aws.md
index 99f2c02..6083452 100644
--- a/site2/website/versioned_docs/version-2.8.3/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.8.3/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.8.3/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.8.3/developing-binary-protocol.md
index 245886b..242c724 100644
--- a/site2/website/versioned_docs/version-2.8.3/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.8.3/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.3-develop-binary-protocol
+id: version-2.8.3-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.8.3/getting-started-docker.md b/site2/website/versioned_docs/version-2.8.3/getting-started-docker.md
index 2dabce5..5a1aaaa 100644
--- a/site2/website/versioned_docs/version-2.8.3/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.8.3/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.3-standalone-docker
+id: version-2.8.3-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone
diff --git a/site2/website/versioned_docs/version-2.8.3/getting-started-helm.md b/site2/website/versioned_docs/version-2.8.3/getting-started-helm.md
index 181f8d0..5132bd3 100644
--- a/site2/website/versioned_docs/version-2.8.3/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.8.3/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.8.3-kubernetes-helm
+id: version-2.8.3-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.9.0/deploy-aws.md b/site2/website/versioned_docs/version-2.9.0/deploy-aws.md
index 2754726..b3d5e28 100644
--- a/site2/website/versioned_docs/version-2.9.0/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.9.0/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.9.0/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.9.0/developing-binary-protocol.md
index 82142ea..f7f20cb 100644
--- a/site2/website/versioned_docs/version-2.9.0/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.9.0/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.9.0-develop-binary-protocol
+id: version-2.9.0-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.9.0/getting-started-docker.md b/site2/website/versioned_docs/version-2.9.0/getting-started-docker.md
index eb84c38..87e2a89 100644
--- a/site2/website/versioned_docs/version-2.9.0/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.9.0/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.9.0-standalone-docker
+id: version-2.9.0-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
diff --git a/site2/website/versioned_docs/version-2.9.0/getting-started-helm.md b/site2/website/versioned_docs/version-2.9.0/getting-started-helm.md
index baa7565..f3dd661 100644
--- a/site2/website/versioned_docs/version-2.9.0/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.9.0/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.9.0-kubernetes-helm
+id: version-2.9.0-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections:
diff --git a/site2/website/versioned_docs/version-2.9.1/deploy-aws.md b/site2/website/versioned_docs/version-2.9.1/deploy-aws.md
index 854abc6..85ab544 100644
--- a/site2/website/versioned_docs/version-2.9.1/deploy-aws.md
+++ b/site2/website/versioned_docs/version-2.9.1/deploy-aws.md
@@ -31,7 +31,7 @@ You can install Ansible on Linux or macOS using pip.
$ pip install ansible
```
-You can install Terraform using the instructions [here](https://www.terraform.io/intro/getting-started/install.html).
+You can install Terraform using the instructions [here](https://learn.hashicorp.com/tutorials/terraform/install-cli).
You also need to have the Terraform and Ansible configuration for Pulsar locally on your machine. You can find them in the [GitHub repository](https://github.com/apache/pulsar) of Pulsar, which you can fetch using Git commands:
diff --git a/site2/website/versioned_docs/version-2.9.1/developing-binary-protocol.md b/site2/website/versioned_docs/version-2.9.1/developing-binary-protocol.md
index fb7bbe7..e6dd8f4 100644
--- a/site2/website/versioned_docs/version-2.9.1/developing-binary-protocol.md
+++ b/site2/website/versioned_docs/version-2.9.1/developing-binary-protocol.md
@@ -1,8 +1,8 @@
---
-id: version-2.9.1-develop-binary-protocol
+id: version-2.9.1-developing-binary-protocol
title: Pulsar binary protocol specification
sidebar_label: Binary protocol
-original_id: develop-binary-protocol
+original_id: developing-binary-protocol
---
Pulsar uses a custom binary protocol for communications between producers/consumers and brokers. This protocol is designed to support required features, such as acknowledgements and flow control, while ensuring maximum transport and implementation efficiency.
diff --git a/site2/website/versioned_docs/version-2.9.1/getting-started-docker.md b/site2/website/versioned_docs/version-2.9.1/getting-started-docker.md
index f8b76ff..b71405e 100644
--- a/site2/website/versioned_docs/version-2.9.1/getting-started-docker.md
+++ b/site2/website/versioned_docs/version-2.9.1/getting-started-docker.md
@@ -1,8 +1,8 @@
---
-id: version-2.9.1-standalone-docker
+id: version-2.9.1-getting-started-docker
title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
-original_id: standalone-docker
+original_id: getting-started-docker
---
For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
diff --git a/site2/website/versioned_docs/version-2.9.1/getting-started-helm.md b/site2/website/versioned_docs/version-2.9.1/getting-started-helm.md
index db1677e..35f766e 100644
--- a/site2/website/versioned_docs/version-2.9.1/getting-started-helm.md
+++ b/site2/website/versioned_docs/version-2.9.1/getting-started-helm.md
@@ -1,8 +1,8 @@
---
-id: version-2.9.1-kubernetes-helm
+id: version-2.9.1-getting-started-helm
title: Get started in Kubernetes
sidebar_label: Run Pulsar in Kubernetes
-original_id: kubernetes-helm
+original_id: getting-started-helm
---
This section guides you through every step of installing and running Apache Pulsar with Helm on Kubernetes quickly, including the following sections: