You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by te...@apache.org on 2022/06/03 01:16:07 UTC
[pulsar] branch master updated: [feature][doc] Add docs for the new S3 offloader (#15835)
This is an automated email from the ASF dual-hosted git repository.
technoboy pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 4abd864050d [feature][doc] Add docs for the new S3 offloader (#15835)
4abd864050d is described below
commit 4abd864050d5bff42a686cb40b0d2c2d47ffd052
Author: momo-jun <60...@users.noreply.github.com>
AuthorDate: Fri Jun 3 09:16:00 2022 +0800
[feature][doc] Add docs for the new S3 offloader (#15835)
---
site2/docs/tiered-storage-aliyun.md | 59 ++++----
site2/docs/tiered-storage-aws.md | 8 +-
site2/docs/tiered-storage-azure.md | 8 +-
site2/docs/tiered-storage-overview.md | 2 +
site2/docs/tiered-storage-s3.md | 247 ++++++++++++++++++++++++++++++++++
site2/website/sidebars.json | 3 +-
6 files changed, 285 insertions(+), 42 deletions(-)
diff --git a/site2/docs/tiered-storage-aliyun.md b/site2/docs/tiered-storage-aliyun.md
index e24dc9b3332..1fe6d7b5d44 100644
--- a/site2/docs/tiered-storage-aliyun.md
+++ b/site2/docs/tiered-storage-aliyun.md
@@ -18,9 +18,9 @@ Follow the steps below to install the Aliyun OSS offloader.
This example uses Pulsar 2.8.0.
-1. Download the Pulsar tarball, see [here](/docs/en/standalone/#install-pulsar-using-binary-release).
+1. [Download the Pulsar tarball](getting-started-standalone.md#install-pulsar-using-binary-release).
-2. Download and untar the Pulsar offloaders package, then copy the Pulsar offloaders as `offloaders` in the Pulsar directory, see [here](/docs/en/standalone/#install-tiered-storage-offloaders-optional).
+2. Download and untar the Pulsar offloaders package, then copy the Pulsar offloaders as `offloaders` in the Pulsar directory. See [Install tiered storage offloaders](getting-started-standalone.md#install-tiered-storage-offloaders-optional).
**Output**
@@ -42,12 +42,10 @@ This example uses Pulsar 2.8.0.
:::note
-Before offloading data from BookKeeper to Aliyun OSS, you need to configure some properties of the Aliyun OSS offload driver.
+Before offloading data from BookKeeper to Aliyun OSS, you need to configure some properties of the Aliyun OSS offload driver. Besides, you can also configure the Aliyun OSS offloader to run it automatically or trigger it manually.
:::
-Besides, you can also configure the Aliyun OSS offloader to run it automatically or trigger it manually.
-
### Configure Aliyun OSS offloader driver
You can configure the Aliyun OSS offloader driver in the configuration file `broker.conf` or `standalone.conf`.
@@ -58,17 +56,17 @@ You can configure the Aliyun OSS offloader driver in the configuration file `bro
| --- | --- |--- |
| `managedLedgerOffloadDriver` | Offloader driver name, which is case-insensitive. | aliyun-oss |
| `offloadersDirectory` | Offloader directory | offloaders |
- | `managedLedgerOffloadBucket` | Bucket | pulsar-topic-offload |
- | `managedLedgerOffloadServiceEndpoint` | Endpoint | http://oss-cn-hongkong.aliyuncs.com |
+ | `managedLedgerOffloadBucket` | [Bucket](#bucket-required) | pulsar-topic-offload |
+ | `managedLedgerOffloadServiceEndpoint` | [Endpoint](#endpoint-required) | http://oss-cn-hongkong.aliyuncs.com |
- **Optional** configurations are as below.
- | Optional | Description | Example value |
+ | Optional | Description | Default value |
| --- | --- | --- |
- | `managedLedgerOffloadReadBufferSizeInBytes` | Size of block read | 1 MB |
- | `managedLedgerOffloadMaxBlockSizeInBytes` | Size of block write | 64 MB |
- | `managedLedgerMinLedgerRolloverTimeMinutes` | Minimum time between ledger rollover for a topic<br /><br />**Note**: it is not recommended that you set this configuration in the production environment. | 2 |
- | `managedLedgerMaxEntriesPerLedger` | Maximum number of entries to append to a ledger before triggering a rollover.<br /><br />**Note**: it is not recommended that you set this configuration in the production environment. | 5000 |
+ | `managedLedgerOffloadReadBufferSizeInBytes` | Block size for each individual read when reading back data from S3-compatible storage. | 1 MB |
+ | `managedLedgerOffloadMaxBlockSizeInBytes` | Maximum block size sent during a multi-part upload to S3-compatible storage. It **cannot** be smaller than 5 MB. | 64 MB |
+ | `managedLedgerMinLedgerRolloverTimeMinutes` | Minimum time between ledger rollover for a topic.<br /><br />It's **not** recommended to change the default value in a production environment. | 2 |
+ | `managedLedgerMaxEntriesPerLedger` | Maximum number of entries to append to a ledger before triggering a rollover.<br /><br />**Note**: It's **not** recommended to change the default value in a production environment. | 5000 |
#### Bucket (required)
@@ -76,7 +74,7 @@ A bucket is a basic container that holds your data. Everything you store in Aliy
##### Example
-This example names the bucket as _pulsar-topic-offload_.
+This example names the bucket `pulsar-topic-offload`.
```conf
@@ -97,7 +95,7 @@ For more information about Aliyun OSS regions and endpoints, see [International
##### Example
-This example sets the endpoint as _oss-us-west-1-internal_.
+This example sets the endpoint as `oss-us-west-1-internal`.
```
@@ -111,8 +109,6 @@ To be able to access Aliyun OSS, you need to authenticate with Aliyun OSS.
Set the environment variables `ALIYUN_OSS_ACCESS_KEY_ID` and `ALIYUN_OSS_ACCESS_KEY_SECRET` in `conf/pulsar_env.sh`.
-"export" is important so that the variables are made available in the environment of spawned processes.
-
```bash
export ALIYUN_OSS_ACCESS_KEY_ID=ABC123456789
@@ -120,18 +116,15 @@ export ALIYUN_OSS_ACCESS_KEY_SECRET=ded7db27a4558e2ea8bbf0bf37ae0e8521618f366c
```
-#### Size of block read/write
+:::note
-You can configure the size of a request sent to or read from Aliyun OSS in the configuration file `broker.conf` or `standalone.conf`.
+Exporting these environment variables makes them available in the environment of spawned processes.
-| Configuration | Description | Default value |
-| --- | --- | --- |
-| `managedLedgerOffloadReadBufferSizeInBytes` | Block size for each individual read when reading back data from Aliyun OSS. | 1 MB |
-| `managedLedgerOffloadMaxBlockSizeInBytes` | Maximum size of a "part" sent during a multipart upload to Aliyun OSS. It **cannot** be smaller than 5 MB. | 64 MB |
+:::
### Run Aliyun OSS offloader automatically
-Namespace policy can be configured to offload data automatically once a threshold is reached. The threshold is based on the size of data that a topic has stored on a Pulsar cluster. Once the topic reaches the threshold, an offloading operation is triggered automatically.
+Namespace policy can be configured to offload data automatically once a threshold is reached. The threshold is based on the size of data that a topic has stored in a Pulsar cluster. Once the topic reaches the threshold, an offloading operation is triggered automatically.
| Threshold value | Action |
| --- | --- |
@@ -141,13 +134,13 @@ Namespace policy can be configured to offload data automatically once a threshol
Automatic offloading runs when a new segment is added to a topic log. If you set the threshold on a namespace, but few messages are being produced to the topic, the offloader does not work until the current segment is full.
-You can configure the threshold size using CLI tools, such as pulsar-admin.
+You can configure the threshold size using CLI tools, such as [`pulsar-admin`](/tools/pulsar-admin/).
-The offload configurations in `broker.conf` and `standalone.conf` are used for the namespaces that do not have namespace level offload policies. Each namespace can have its own offload policy. If you want to set offload policy for each namespace, use the command [`pulsar-admin namespaces set-offload-policies options`](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-set-offload-policies-em-) command.
+The offload configurations in `broker.conf` and `standalone.conf` are used for the namespaces that do not have namespace-level offload policies. Each namespace can have its offload policy. If you want to set an offload policy for a specific namespace, use the command [`pulsar-admin namespaces set-offload-policies options`](/tools/pulsar-admin/) command.
#### Example
-This example sets the Aliyun OSS offloader threshold size to 10 MB using pulsar-admin.
+This example sets the Aliyun OSS offloader threshold size to 10 MB using `pulsar-admin`.
```bash
@@ -157,7 +150,7 @@ bin/pulsar-admin namespaces set-offload-threshold --size 10M my-tenant/my-namesp
:::tip
-For more information about the `pulsar-admin namespaces set-offload-threshold options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-set-offload-threshold-em-).
+For more information about the `pulsar-admin namespaces set-offload-threshold options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
@@ -167,13 +160,13 @@ For individual topics, you can trigger the Aliyun OSS offloader manually using o
- Use REST endpoint.
-- Use CLI tools (such as pulsar-admin).
+- Use CLI tools, such as [`pulsar-admin`](/tools/pulsar-admin/).
- To trigger it via CLI tools, you need to specify the maximum amount of data (threshold) that should be retained on a Pulsar cluster for a topic. If the size of the topic data on the Pulsar cluster exceeds this threshold, segments from the topic are moved to Aliyun OSS until the threshold is no longer exceeded. Older segments are moved first.
+ To trigger it via CLI tools, you need to specify the maximum amount of data (threshold) that should be retained in a Pulsar cluster for a topic. If the size of the topic data on the Pulsar cluster exceeds this threshold, segments from the topic are moved to Aliyun OSS until the threshold is no longer exceeded. Older segments are moved first.
#### Example
-- This example triggers the Aliyun OSS offloader to run manually using pulsar-admin.
+- This example triggers the Aliyun OSS offloader to run manually using `pulsar-admin`.
```bash
@@ -191,11 +184,11 @@ For individual topics, you can trigger the Aliyun OSS offloader manually using o
:::tip
- For more information about the `pulsar-admin topics offload options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-offload-em-).
+ For more information about the `pulsar-admin topics offload options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
-- This example checks the Aliyun OSS offloader status using pulsar-admin.
+- This example checks the Aliyun OSS offloader status using `pulsar-admin`.
```bash
@@ -248,7 +241,7 @@ For individual topics, you can trigger the Aliyun OSS offloader manually using o
:::tip
- For more information about the `pulsar-admin topics offload-status options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-offload-status-em-).
+ For more information about the `pulsar-admin topics offload-status options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
diff --git a/site2/docs/tiered-storage-aws.md b/site2/docs/tiered-storage-aws.md
index 70ecc717f65..913e8129120 100644
--- a/site2/docs/tiered-storage-aws.md
+++ b/site2/docs/tiered-storage-aws.md
@@ -212,7 +212,7 @@ Automatic offloading runs when a new segment is added to a topic log. If you set
You can configure the threshold size using CLI tools, such as pulsar-admin.
-The offload configurations in `broker.conf` and `standalone.conf` are used for the namespaces that do not have namespace level offload policies. Each namespace can have its own offload policy. If you want to set offload policy for each namespace, use the command [`pulsar-admin namespaces set-offload-policies options`](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-set-offload-policies-em-) command.
+The offload configurations in `broker.conf` and `standalone.conf` are used for the namespaces that do not have namespace level offload policies. Each namespace can have its own offload policy. If you want to set offload policy for each namespace, use the command [`pulsar-admin namespaces set-offload-policies options`](/tools/pulsar-admin/) command.
#### Example
@@ -226,7 +226,7 @@ bin/pulsar-admin namespaces set-offload-threshold --size 10M my-tenant/my-namesp
:::tip
-For more information about the `pulsar-admin namespaces set-offload-threshold options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-set-offload-threshold-em-).
+For more information about the `pulsar-admin namespaces set-offload-threshold options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
@@ -260,7 +260,7 @@ For individual topics, you can trigger AWS S3 offloader manually using one of th
:::tip
- For more information about the `pulsar-admin topics offload options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-offload-em-).
+ For more information about the `pulsar-admin topics offload options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
@@ -317,7 +317,7 @@ For individual topics, you can trigger AWS S3 offloader manually using one of th
:::tip
- For more information about the `pulsar-admin topics offload-status options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-offload-status-em-).
+ For more information about the `pulsar-admin topics offload-status options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
diff --git a/site2/docs/tiered-storage-azure.md b/site2/docs/tiered-storage-azure.md
index b446346f283..c02d6a864ae 100644
--- a/site2/docs/tiered-storage-azure.md
+++ b/site2/docs/tiered-storage-azure.md
@@ -150,7 +150,7 @@ Automatic offloading runs when a new segment is added to a topic log. If you set
You can configure the threshold size using CLI tools, such as pulsar-admin.
-The offload configurations in `broker.conf` and `standalone.conf` are used for the namespaces that do not have namespace level offload policies. Each namespace can have its own offload policy. If you want to set offload policy for each namespace, use the command [`pulsar-admin namespaces set-offload-policies options`](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-set-offload-policies-em-) command.
+The offload configurations in `broker.conf` and `standalone.conf` are used for the namespaces that do not have namespace level offload policies. Each namespace can have its own offload policy. If you want to set offload policy for each namespace, use the command [`pulsar-admin namespaces set-offload-policies options`](/tools/pulsar-admin/)) command.
#### Example
@@ -164,7 +164,7 @@ bin/pulsar-admin namespaces set-offload-threshold --size 10M my-tenant/my-namesp
:::tip
-For more information about the `pulsar-admin namespaces set-offload-threshold options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-set-offload-threshold-em-).
+For more information about the `pulsar-admin namespaces set-offload-threshold options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
@@ -198,7 +198,7 @@ For individual topics, you can trigger Azure BlobStore offloader manually using
:::tip
- For more information about the `pulsar-admin topics offload options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-offload-em-).
+ For more information about the `pulsar-admin topics offload options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
@@ -255,7 +255,7 @@ For individual topics, you can trigger Azure BlobStore offloader manually using
:::tip
- For more information about the `pulsar-admin topics offload-status options` command, including flags, descriptions, and default values, see [here](/tools/pulsar-admin/2.6.0-SNAPSHOT/#-em-offload-status-em-).
+ For more information about the `pulsar-admin topics offload-status options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
:::
diff --git a/site2/docs/tiered-storage-overview.md b/site2/docs/tiered-storage-overview.md
index 1c58eafcba7..caed660527e 100644
--- a/site2/docs/tiered-storage-overview.md
+++ b/site2/docs/tiered-storage-overview.md
@@ -20,6 +20,8 @@ Pulsar's **Tiered Storage** feature allows older backlog data to be moved from B
- For more information about how to use the Aliyun OSS offloader with Pulsar, see [here](tiered-storage-aliyun).
+ - For more information about how to use the S3 offloader with Pulsar, see [here](tiered-storage-s3).
+
:::
* Tiered storage uses [Apache Hadoop](http://hadoop.apache.org/) to support filesystems for long term storage.
diff --git a/site2/docs/tiered-storage-s3.md b/site2/docs/tiered-storage-s3.md
new file mode 100644
index 00000000000..8d877c0a114
--- /dev/null
+++ b/site2/docs/tiered-storage-s3.md
@@ -0,0 +1,247 @@
+---
+id: tiered-storage-s3
+title: Use S3 offloader with Pulsar
+sidebar_label: "S3 offloader"
+---
+
+S3 offloader is introduced to serve S3-compatible storage, which means that the storage employs the S3 API as its “language" and applications that speak the S3 API are able to plug and play with S3-compatible storage.
+
+This chapter guides you through every step of installing and configuring the S3 offloader and using it with Pulsar.
+
+## Installation
+
+Follow the steps below to install the S3 offloader.
+
+### Prerequisite
+
+- Pulsar: 2.9.3 or later versions
+
+### Steps
+
+This example uses Pulsar 2.9.3.
+
+1. [Download the Pulsar tarball](getting-started-standalone.md#install-pulsar-using-binary-release).
+
+2. Download and untar the Pulsar offloaders package, then copy the Pulsar offloaders as `offloaders` in the Pulsar directory. See [Install tiered storage offloaders](getting-started-standalone.md#install-tiered-storage-offloaders-optional).
+
+ **Output**
+
+ As shown from the output, Pulsar uses [Apache jclouds](https://jclouds.apache.org) to support [AWS S3](https://aws.amazon.com/s3/), [GCS](https://cloud.google.com/storage/), [Azure](https://portal.azure.com/#home), and [Aliyun OSS](https://www.aliyun.com/product/oss) for long-term storage.
+
+ ```
+
+ tiered-storage-file-system-2.9.3.nar
+ tiered-storage-jcloud-2.9.3.nar
+
+ ```
+
+ :::note
+
+ * If you run Pulsar in a bare-metal cluster, ensure that `offloaders` tarball is unzipped in every broker's Pulsar directory.
+ * If you run Pulsar in Docker or deploy Pulsar using a Docker image (such as K8s and DCOS), you can use the `apachepulsar/pulsar-all` image. The `apachepulsar/pulsar-all` image has already bundled tiered storage offloaders.
+
+ :::
+
+## Configuration
+
+:::note
+
+Before offloading data from BookKeeper to S3-compatible storage, you need to configure some properties of the S3 offload driver. Besides, you can also configure the S3 offloader to run it automatically or trigger it manually.
+
+:::
+
+
+### Configure S3 offloader driver
+
+You can configure the S3 offloader driver in the configuration file `broker.conf` or `standalone.conf`.
+
+- **Required** configurations are as below.
+
+ | Required configuration | Description | Example value |
+ | --- | --- |--- |
+ | `managedLedgerOffloadDriver` | Offloader driver name, which is case-insensitive. | S3 |
+ | `offloadersDirectory` | Offloader directory | offloaders |
+ | `managedLedgerOffloadBucket` | [Bucket](#bucket-required) | pulsar-topic-offload |
+ | `managedLedgerOffloadServiceEndpoint` | [Endpoint](#endpoint-required) | http://localhost:9000 |
+
+- **Optional** configurations are as below.
+
+ | Optional | Description | Default value |
+ | --- | --- | --- |
+ | `managedLedgerOffloadReadBufferSizeInBytes` | Block size for each individual read when reading back data from S3-compatible storage. | 1 MB |
+ | `managedLedgerOffloadMaxBlockSizeInBytes` | Maximum block size sent during a multi-part upload to S3-compatible storage. It **cannot** be smaller than 5 MB. | 64 MB |
+ | `managedLedgerMinLedgerRolloverTimeMinutes` | Minimum time between ledger rollover for a topic.<br /><br />**Note**: It's **not** recommended to change the default value in a production environment. | 2 |
+ | `managedLedgerMaxEntriesPerLedger` | Maximum number of entries to append to a ledger before triggering a rollover.<br /><br />**Note**: It's **not** recommended to change the default value in a production environment. | 5000 |
+
+#### Bucket (required)
+
+A bucket is a basic container that holds your data. Everything you store in S3-compatible storage must be contained in a bucket. You can use a bucket to organize your data and control access to your data, but unlike directory and folder, you cannot nest a bucket.
+
+##### Example
+
+This example names the bucket `pulsar-topic-offload`.
+
+```conf
+
+managedLedgerOffloadBucket=pulsar-topic-offload
+
+```
+
+#### Endpoint (required)
+
+The endpoint is the region where a bucket is located.
+
+
+##### Example
+
+This example sets the endpoint as `localhost`.
+
+```
+
+managedLedgerOffloadServiceEndpoint=http://localhost:9000
+
+```
+
+#### Authentication (optional)
+
+To be able to access S3-compatible storage, you need to authenticate with S3-compatible storage.
+
+Set the environment variables `ACCESS_KEY_ID` and `ACCESS_KEY_SECRET` in `conf/pulsar_env.sh`.
+
+```bash
+
+export ACCESS_KEY_ID=ABC123456789
+export ACCESS_KEY_SECRET=ded7db27a4558e2ea8bbf0bf37ae0e8521618f366c
+
+```
+
+:::note
+
+Exporting these environment variables makes them available in the environment of spawned processes.
+
+:::
+
+### Run S3 offloader automatically
+
+Namespace policy can be configured to offload data automatically once a threshold is reached. The threshold is based on the size of data that a topic has stored in a Pulsar cluster. Once the topic reaches the threshold, an offloading operation is triggered automatically.
+
+| Threshold value | Action |
+| --- | --- |
+| > 0 | It triggers the offloading operation if the topic storage reaches its threshold. |
+| = 0 | It causes a broker to offload data as soon as possible. |
+| < 0 | It disables automatic offloading operation. |
+
+Automatic offloading runs when a new segment is added to a topic log. If you set the threshold for a namespace, but few messages are being produced to the topic, the offloader does not work until the current segment is full.
+
+You can configure the threshold size using CLI tools, such as [`pulsar-admin`](/tools/pulsar-admin/).
+
+The offload configurations in `broker.conf` and `standalone.conf` are used for the namespaces that do not have namespace-level offload policies. Each namespace can have its offload policy. If you want to set an offload policy for a specific namespace, use the command [`pulsar-admin namespaces set-offload-policies options`](/tools/pulsar-admin/) command.
+
+#### Example
+
+This example sets the S3 offloader threshold size to 10 MB using `pulsar-admin`.
+
+```bash
+
+bin/pulsar-admin namespaces set-offload-threshold --size 10M my-tenant/my-namespace
+
+```
+
+:::tip
+
+For more information about the `pulsar-admin namespaces set-offload-threshold options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
+
+:::
+
+### Run S3 offloader manually
+
+For individual topics, you can trigger the S3 offloader manually using one of the following methods:
+
+- Use REST endpoint.
+
+- Use CLI tools, such as [`pulsar-admin`](/tools/pulsar-admin/).
+
+ To trigger it via CLI tools, you need to specify the maximum amount of data (threshold) that should be retained in a Pulsar cluster for a topic. If the size of the topic data in the Pulsar cluster exceeds this threshold, segments from the topic are moved to S3-compatible storage until the threshold is no longer exceeded. Older segments are moved first.
+
+#### Example
+
+- This example triggers the S3 offloader to run manually using `pulsar-admin`.
+
+ ```bash
+
+ bin/pulsar-admin topics offload --size-threshold 10M my-tenant/my-namespace/topic1
+
+ ```
+
+ **Output**
+
+ ```bash
+
+ Offload triggered for persistent://my-tenant/my-namespace/topic1 for messages before 2:0:-1
+
+ ```
+
+ :::tip
+
+ For more information about the `pulsar-admin topics offload options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
+
+ :::
+
+- This example checks the S3 offloader status using `pulsar-admin`.
+
+ ```bash
+
+ bin/pulsar-admin topics offload-status persistent://my-tenant/my-namespace/topic1
+
+ ```
+
+ **Output**
+
+ ```bash
+
+ Offload is currently running
+
+ ```
+
+ To wait for the S3 offloader to complete the job, add the `-w` flag.
+
+ ```bash
+
+ bin/pulsar-admin topics offload-status -w persistent://my-tenant/my-namespace/topic1
+
+ ```
+
+ **Output**
+
+ ```
+
+ Offload was a success
+
+ ```
+
+ If there is an error in offloading, the error is propagated to the `pulsar-admin topics offload-status` command.
+
+ ```bash
+
+ bin/pulsar-admin topics offload-status persistent://my-tenant/my-namespace/topic1
+
+ ```
+
+ **Output**
+
+ ```
+
+ Error in offload
+ null
+
+ Reason: Error offloading: org.apache.bookkeeper.mledger.ManagedLedgerException: java.util.concurrent.CompletionException: com.amazonaws.services.s3.model.AmazonS3Exception: Anonymous users cannot initiate multipart uploads. Please authenticate. (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 798758DE3F1776DF; S3 Extended Request ID: dhBFz/lZm1oiG/oBEepeNlhrtsDlzoOhocuYMpKihQGXe6EG8puRGOkK6UwqzVrMXTWBxxHcS+g=), S3 Extended Request ID: dhBFz/lZm1oiG/oBEepeNl [...]
+
+ ```
+
+ :::tip
+
+ For more information about the `pulsar-admin topics offload-status options` command, including flags, descriptions, and default values, see [Pulsar admin docs](/tools/pulsar-admin/).
+
+ :::
+
+
diff --git a/site2/website/sidebars.json b/site2/website/sidebars.json
index 0fb5eb4c52e..2961629a2fa 100644
--- a/site2/website/sidebars.json
+++ b/site2/website/sidebars.json
@@ -86,7 +86,8 @@
"tiered-storage-gcs",
"tiered-storage-filesystem",
"tiered-storage-azure",
- "tiered-storage-aliyun"
+ "tiered-storage-aliyun",
+ "tiered-storage-s3"
]
},
{