You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by li...@apache.org on 2021/08/26 13:46:07 UTC
[pulsar] branch master updated: [docs] [ISSUE 8671] Add hardware
settings recommendations (#11712)
This is an automated email from the ASF dual-hosted git repository.
liuyu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 07aff70 [docs] [ISSUE 8671] Add hardware settings recommendations (#11712)
07aff70 is described below
commit 07aff70006e5875180ed9f4d1697024c0597a5ec
Author: sijia-w <53...@users.noreply.github.com>
AuthorDate: Thu Aug 26 15:45:01 2021 +0200
[docs] [ISSUE 8671] Add hardware settings recommendations (#11712)
* add hardware settings recommendations
* update
* update
---
site2/docs/deploy-bare-metal.md | 31 ++++++++++++++++++++---
site2/docs/getting-started-docker.md | 9 +++----
site2/docs/getting-started-helm.md | 16 ++++++------
site2/docs/getting-started-standalone.md | 42 ++++++++++++--------------------
4 files changed, 54 insertions(+), 44 deletions(-)
diff --git a/site2/docs/deploy-bare-metal.md b/site2/docs/deploy-bare-metal.md
index 0062284..7f56cbf 100644
--- a/site2/docs/deploy-bare-metal.md
+++ b/site2/docs/deploy-bare-metal.md
@@ -26,8 +26,7 @@ Deploying a Pulsar cluster consists of the following steps:
Currently, Pulsar is available for 64-bit **macOS**, **Linux**, and **Windows**. To use Pulsar, you need to install 64-bit JRE/JDK 8 or later versions.
->**Tips**
->
+>**Tips**
> You can reuse existing Zookeeper clusters.
To run Pulsar on bare metal, the following configuration is recommended:
@@ -37,8 +36,7 @@ To run Pulsar on bare metal, the following configuration is recommended:
* 3 for running a Pulsar broker, and a [BookKeeper](https://bookkeeper.apache.org) bookie
* A single [DNS](https://en.wikipedia.org/wiki/Domain_Name_System) name covering all of the Pulsar broker hosts
-> **Note**
->
+> **Note**
> * Broker is only supported on 64-bit JVM.
> * If you do not have enough machines, or you want to test Pulsar in cluster mode (and expand the cluster later), You can fully deploy Pulsar on a node on which ZooKeeper, bookie and broker run.
> * If you do not have a DNS server, you can use the multi-host format in the service URL instead.
@@ -65,6 +63,31 @@ For machines running a bookie and a Pulsar broker, more powerful machines are re
* Fast CPUs and 10Gbps [NIC](https://en.wikipedia.org/wiki/Network_interface_controller) (for Pulsar brokers)
* Small and fast [solid-state drives](https://en.wikipedia.org/wiki/Solid-state_drive) (SSDs) or [hard disk drives](https://en.wikipedia.org/wiki/Hard_disk_drive) (HDDs) with a [RAID](https://en.wikipedia.org/wiki/RAID) controller and a battery-backed write cache (for BookKeeper bookies)
+To start a Pulsar instance, below are the minimum and the recommended hardware settings.
+
+1. The minimum hardware settings (250 Pulsar topics)
+ - Broker
+ - CPU: 0.2
+ - Memory: 256MB
+ - Bookie
+ - CPU: 0.2
+ - Memory: 256MB
+ - Storage:
+ - Journal: 8GB, PD-SSD
+ - Ledger: 16GB, PD-STANDARD
+
+2. The recommended hardware settings (1000 Pulsar topics)
+
+ - Broker
+ - CPU: 8
+ - Memory: 8GB
+ - Bookie
+ - CPU: 4
+ - Memory: 8GB
+ - Storage:
+ - Journal: 256GB, PD-SSD
+ - Ledger: 2TB, PD-STANDARD
+
## Install the Pulsar binary package
> You need to install the Pulsar binary package on each machine in the cluster, including machines running ZooKeeper and BookKeeper.
diff --git a/site2/docs/getting-started-docker.md b/site2/docs/getting-started-docker.md
index c3a241a..16dfa8b 100644
--- a/site2/docs/getting-started-docker.md
+++ b/site2/docs/getting-started-docker.md
@@ -4,11 +4,9 @@ title: Set up a standalone Pulsar in Docker
sidebar_label: Run Pulsar in Docker
---
-For local development and testing, you can run Pulsar in standalone
-mode on your own machine within a Docker container.
+For local development and testing, you can run Pulsar in standalone mode on your own machine within a Docker container.
-If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition)
-and follow the instructions for your OS.
+If you have not installed Docker, download the [Community edition](https://www.docker.com/community-edition) and follow the instructions for your OS.
## Start Pulsar in Docker
@@ -32,8 +30,7 @@ If you start Pulsar successfully, you will see `INFO`-level log messages like th
...
```
-> #### Tip
->
+> **Tip**
> When you start a local standalone cluster, a `public/default`
namespace is created automatically. The namespace is used for development purposes. All Pulsar topics are managed within namespaces.
For more information, see [Topics](concepts-messaging.md#topics).
diff --git a/site2/docs/getting-started-helm.md b/site2/docs/getting-started-helm.md
index b1313039..b147f55 100644
--- a/site2/docs/getting-started-helm.md
+++ b/site2/docs/getting-started-helm.md
@@ -20,7 +20,7 @@ For deploying a Pulsar cluster for production usage, read the documentation on [
- kubectl 1.14.0+
- Helm 3.0+
-> #### Tip
+> **Tip**
> For the following steps, step 2 and step 3 are for **developers** and step 4 and step 5 are for **administrators**.
## Step 0: Prepare a Kubernetes cluster
@@ -52,7 +52,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
## Step 1: Install Pulsar Helm chart
-0. Add Pulsar charts repo.
+1. Add Pulsar charts repo.
```bash
helm repo add apache https://pulsar.apache.org/charts
@@ -62,14 +62,14 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
helm repo update
```
-1. Clone the Pulsar Helm chart repository.
+2. Clone the Pulsar Helm chart repository.
```bash
git clone https://github.com/apache/pulsar-helm-chart
cd pulsar-helm-chart
```
-2. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
+3. Run the script `prepare_helm_release.sh` to create secrets required for installing the Apache Pulsar Helm chart. The username `pulsar` and password `pulsar` are used for logging into the Grafana dashboard and Pulsar Manager.
```bash
./scripts/pulsar/prepare_helm_release.sh \
@@ -78,9 +78,9 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
-c
```
-3. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
+4. Use the Pulsar Helm chart to install a Pulsar cluster to Kubernetes.
- > **NOTE**
+ > **Note**
> You need to specify `--set initialize=true` when installing Pulsar the first time. This command installs and starts Apache Pulsar.
```bash
@@ -91,7 +91,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
pulsar-mini apache/pulsar
```
-4. Check the status of all pods.
+5. Check the status of all pods.
```bash
kubectl get pods -n pulsar
@@ -115,7 +115,7 @@ We use [Minikube](https://minikube.sigs.k8s.io/docs/start/) in this quick start
pulsar-mini-zookeeper-0 1/1 Running 0 9m27s
```
-5. Check the status of all services in the namespace `pulsar`.
+6. Check the status of all services in the namespace `pulsar`.
```bash
kubectl get services -n pulsar
diff --git a/site2/docs/getting-started-standalone.md b/site2/docs/getting-started-standalone.md
index 2449f52..657fd7d 100644
--- a/site2/docs/getting-started-standalone.md
+++ b/site2/docs/getting-started-standalone.md
@@ -6,22 +6,21 @@ sidebar_label: Run Pulsar locally
For local development and testing, you can run Pulsar in standalone mode on your machine. The standalone mode includes a Pulsar broker, the necessary ZooKeeper and BookKeeper components running inside of a single Java Virtual Machine (JVM) process.
-> #### Pulsar in production?
+> **Pulsar in production?**
> If you're looking to run a full production Pulsar installation, see the [Deploying a Pulsar instance](deploy-bare-metal.md) guide.
## Install Pulsar standalone
-This tutorial guides you through every step of the installation process.
+This tutorial guides you through every step of installing Pulsar locally.
### System requirements
Currently, Pulsar is available for 64-bit **macOS**, **Linux**, and **Windows**. To use Pulsar, you need to install 64-bit JRE/JDK 8 or later versions.
-> #### Tip
+> **Tip**
> By default, Pulsar allocates 2G JVM heap memory to start. It can be changed in `conf/pulsar_env.sh` file under `PULSAR_MEM`. This is extra options passed into JVM.
-> **Note**
->
+> **Note**
> Broker is only supported on 64-bit JVM.
### Install Pulsar using binary release
@@ -67,7 +66,7 @@ Directory | Contains
`instances` | Artifacts created for [Pulsar Functions](functions-overview.md).
`logs` | Logs created by the installation.
-> #### Tip
+> **Tip**
> If you want to use builtin connectors and tiered storage offloaders, you can install them according to the following instructions：
>
> * [Install builtin connectors (optional)](#install-builtin-connectors-optional)
@@ -104,20 +103,16 @@ pulsar-io-aerospike-{{pulsar:version}}.nar
...
```
-> #### Note
->
-> * If you are running Pulsar in a bare metal cluster, make sure `connectors` tarball is unzipped in every pulsar directory of the broker
-> (or in every pulsar directory of function-worker if you are running a separate worker cluster for Pulsar Functions).
+> **Note**
+> * If you are running Pulsar in a bare metal cluster, make sure `connectors` tarball is unzipped in every pulsar directory of the broker (or in every pulsar directory of function-worker if you are running a separate worker cluster for Pulsar Functions).
>
-> * If you are [running Pulsar in Docker](getting-started-docker.md) or deploying Pulsar using a docker image (e.g. [K8S](deploy-kubernetes.md) or [DCOS](deploy-dcos.md)),
-> you can use the `apachepulsar/pulsar-all` image instead of the `apachepulsar/pulsar` image. `apachepulsar/pulsar-all` image has already bundled [all builtin connectors](io-overview.md#working-with-connectors).
+> * If you are [running Pulsar in Docker](getting-started-docker.md) or deploying Pulsar using a docker image (e.g. [K8S](deploy-kubernetes.md) or [DCOS](deploy-dcos.md)), you can use the `apachepulsar/pulsar-all` image instead of the `apachepulsar/pulsar` image. `apachepulsar/pulsar-all` image has already bundled [all builtin connectors](io-overview.md#working-with-connectors).
### Install tiered storage offloaders (optional)
-> #### Tip
->
-> Since `2.2.0` release, Pulsar releases a separate binary distribution, containing the tiered storage offloaders.
-> To enable tiered storage feature, follow the instructions below; otherwise skip this section.
+> **Tip**
+> - Since `2.2.0` release, Pulsar releases a separate binary distribution, containing the tiered storage offloaders.
+> - To enable tiered storage feature, follow the instructions below; otherwise skip this section.
To get started with [tiered storage offloaders](concepts-tiered-storage.md), you need to download the offloaders tarball release on every broker node in one of the following ways:
@@ -150,12 +145,10 @@ tiered-storage-jcloud-{{pulsar:version}}.nar
For more information on how to configure tiered storage, see [Tiered storage cookbook](cookbooks-tiered-storage.md).
-> #### Note
->
+> **Note**
> * If you are running Pulsar in a bare metal cluster, make sure that `offloaders` tarball is unzipped in every broker's pulsar directory.
>
-> * If you are [running Pulsar in Docker](getting-started-docker.md) or deploying Pulsar using a docker image (e.g. [K8S](deploy-kubernetes.md) or [DCOS](deploy-dcos.md)),
-> you can use the `apachepulsar/pulsar-all` image instead of the `apachepulsar/pulsar` image. `apachepulsar/pulsar-all` image has already bundled tiered storage offloaders.
+> * If you are [running Pulsar in Docker](getting-started-docker.md) or deploying Pulsar using a docker image (e.g. [K8S](deploy-kubernetes.md) or [DCOS](deploy-dcos.md)), you can use the `apachepulsar/pulsar-all` image instead of the `apachepulsar/pulsar` image. `apachepulsar/pulsar-all` image has already bundled tiered storage offloaders.
## Start Pulsar standalone
@@ -173,8 +166,7 @@ If you have started Pulsar successfully, you will see `INFO`-level log messages
21:59:34.576 [main] INFO org.apache.pulsar.websocket.WebSocketService - Pulsar WebSocket Service started
```
-> #### Tip
->
+> **Tip**
> * The service is running on your terminal, which is under your direct control. If you need to run other commands, open a new terminal window.
You can also run the service as a background process using the `pulsar-daemon start standalone` command. For more information, see [pulsar-daemon](https://pulsar.apache.org/docs/en/reference-cli-tools/#pulsar-daemon).
>
@@ -200,8 +192,7 @@ If the message has been successfully consumed, you will see a confirmation like
22:17:16.781 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully consumed
```
-> #### Tip
->
+> **Tip**
> As you have noticed that we do not explicitly create the `my-topic` topic, to which we consume the message. When you consume a message to a topic that does not yet exist, Pulsar creates that topic for you automatically. Producing a message to a topic that does not exist will automatically create that topic for you as well.
### Produce a message
@@ -222,8 +213,7 @@ If the message has been successfully published to the topic, you will see a conf
Press `Ctrl+C` to stop a local standalone Pulsar.
-> #### Tip
->
+> **Tip**
> If the service runs as a background process using the `pulsar-daemon start standalone` command, then use the `pulsar-daemon stop standalone` command to stop the service.
>
> For more information, see [pulsar-daemon](https://pulsar.apache.org/docs/en/reference-cli-tools/#pulsar-daemon).