You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by mm...@apache.org on 2018/02/01 02:17:29 UTC
[incubator-pulsar] branch master updated: Pulsar proxy
documentation (#1086)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 3191db4 Pulsar proxy documentation (#1086)
3191db4 is described below
commit 3191db460fb2aa4612056677e0e39d421fae34b4
Author: Luc Perkins <lu...@gmail.com>
AuthorDate: Wed Jan 31 18:17:27 2018 -0800
Pulsar proxy documentation (#1086)
* add pulsar CLI docs to YAML config
* add new pulsar proxy doc and add doc to sidebar menu
* add section on concepts and architecture
* finish draft of concepts/arch documentation
* add basic admin instructions for proxy
---
site/Gemfile | 2 +-
site/_data/cli/pulsar.yaml | 13 ++++++
site/_data/sidebar.yaml | 2 +
site/docs/latest/admin/Proxy.md | 53 ++++++++++++++++++++++
.../getting-started/ConceptsAndArchitecture.md | 25 ++++++++++
5 files changed, 94 insertions(+), 1 deletion(-)
diff --git a/site/Gemfile b/site/Gemfile
index 242f760..94e2a87 100644
--- a/site/Gemfile
+++ b/site/Gemfile
@@ -20,4 +20,4 @@
source 'https://rubygems.org'
ruby '2.4.1'
-gem 'jekyll', '3.7.0'
\ No newline at end of file
+gem 'jekyll', '3.7.0'
diff --git a/site/_data/cli/pulsar.yaml b/site/_data/cli/pulsar.yaml
index 50d9488..f5ae17e 100644
--- a/site/_data/cli/pulsar.yaml
+++ b/site/_data/cli/pulsar.yaml
@@ -79,6 +79,19 @@ commands:
description: The web service URL for the new cluster with TLS encryption
- flags: -zk, --zookeeper
description: The local ZooKeeper quorum connection string
+- name: proxy
+ description: Manages the Pulsar proxy
+ example: |
+ pulsar proxy \
+ --zookeeper-servers zk-0,zk-1,zk2 \
+ --global-zookeeper-servers zk-0,zk-1,zk-2
+ options:
+ - flags: -c, --config
+ description: Path to a Pulsar proxy configuration file
+ - flags: -gzk, --global-zookeeper-servers
+ description: Global ZooKeeper connection string
+ - flags: -zk, --zookeeper-servers
+ description: Local ZooKeeper connection string
- name: standalone
description: Run a broker service with local bookies and local ZooKeeper
options:
diff --git a/site/_data/sidebar.yaml b/site/_data/sidebar.yaml
index 4a543e2..c026438 100644
--- a/site/_data/sidebar.yaml
+++ b/site/_data/sidebar.yaml
@@ -65,6 +65,8 @@ groups:
endpoint: Stats
- title: Modular load manager
endpoint: ModularLoadManager
+ - title: Pulsar proxy
+ endpoint: Proxy
- title: Client libraries
dir: clients
diff --git a/site/docs/latest/admin/Proxy.md b/site/docs/latest/admin/Proxy.md
new file mode 100644
index 0000000..07f1bc0
--- /dev/null
+++ b/site/docs/latest/admin/Proxy.md
@@ -0,0 +1,53 @@
+---
+title: The Pulsar proxy
+---
+
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+The [Pulsar proxy](../../getting-started/ConceptsAndArchitecture#pulsar-proxy) is an optional gateway that you can run over the {% popover brokers %} in a Pulsar {% popover cluster %}. We recommend running a Pulsar proxy in cases when direction connections between clients and Pulsar brokers are either infeasible, undesirable, or both, for example when running Pulsar in a cloud environment or on [Kubernetes](https://kubernetes.io) or an analogous platform.
+
+## Running the proxy
+
+In order to run the Pulsar proxy, you need to have both a local and global [ZooKeeper](https://zookeeper.apache.org) quorum set up for use by your Pulsar cluster. For instructions, see [this document](../../deployment/cluster). Once you have ZooKeeper set up and have connection strings for both ZooKeeper quorums, you can use the [`proxy`](../../reference/CliTools#pulsar-proxy) command of the [`pulsar`](../../reference/CliTools#pulsar) CLI tool to start up the proxy (preferably on its own [...]
+
+To start the proxy:
+
+```bash
+$ cd /path/to/pulsar/directory
+$ bin/pulsar proxy \
+ --zookeeper-servers zk-0,zk-1,zk-2 \
+ --global-zookeeper-servers zk-0,zk-1,zk-2
+```
+
+{% include admonition.html type="success" content="You can run as many instances of the Pulsar proxy in a cluster as you would like." %}
+
+## Stopping the proxy
+
+The Pulsar proxy runs by default in the foreground. To stop the proxy, simply stop the process in which it's running.
+
+## Proxy frontends
+
+We recommend running the Pulsar proxy behind some kind of load-distributing frontend, such as an [HAProxy](https://www.digitalocean.com/community/tutorials/an-introduction-to-haproxy-and-load-balancing-concepts) load balancer.
+
+## Using Pulsar clients with the proxy
+
+Once your Pulsar proxy is up and running, preferably behind a load-distributing [frontend](#proxy-frontends), clients can connect to the proxy via whichever address is used by the frontend. If the address were the DNS address `pulsar.cluster.default`, for example, then the connection URL for clients would be `pulsar://pulsar.cluster.default:6650`.
\ No newline at end of file
diff --git a/site/docs/latest/getting-started/ConceptsAndArchitecture.md b/site/docs/latest/getting-started/ConceptsAndArchitecture.md
index 22a0440..f69fd78 100644
--- a/site/docs/latest/getting-started/ConceptsAndArchitecture.md
+++ b/site/docs/latest/getting-started/ConceptsAndArchitecture.md
@@ -307,6 +307,31 @@ When an application wants to create a producer/consumer, the Pulsar client libra
Whenever the TCP connection breaks, the client will immediately re-initiate this setup phase and will keep trying with exponential backoff to re-establish the producer or consumer until the operation succeeds.
+## Pulsar proxy
+
+One way for Pulsar clients to interact with a Pulsar [cluster](#clusters) is by connecting to Pulsar message [brokers](#brokers) directly. In some cases, however, this kind of direct connection is either infeasible or undesirable because the client doesn't have direct access to broker addresses. If you're running Pulsar in a cloud environment or on [Kubernetes](https://kubernetes.io) or an analogous platform, for example, then direct client connections to brokers are likely not possible.
+
+The **Pulsar proxy** provides a solution to this problem by acting as a single gateway for all of the brokers in a cluster. If you run the Pulsar proxy (which, again, is optional), all client connections with the Pulsar {% popover cluster %} will flow through the proxy rather than communicating with brokers.
+
+{% include admonition.html type="success" content="For the sake of performance and fault tolerance, you can run as many instances of the Pulsar proxy as you'd like." %}
+
+Architecturally, the Pulsar proxy gets all the information it requires from ZooKeeper. When starting the proxy on a machine, you only need to provide ZooKeeper connection strings for the cluster-specific and {% popover global ZooKeeper %} clusters. Here's an example:
+
+```bash
+$ bin/pulsar proxy \
+ --zookeeper-servers zk-0,zk-1,zk-2 \
+ --global-zookeeper-servers zk-0,zk-1,zk-2
+```
+
+{% include admonition.html type="info" title="Pulsar proxy docs" content='
+For documentation on using the Pulsar proxy, see the [Pulsar proxy admin documentation](../../admin/Proxy).
+' %}
+
+Some important things to know about the Pulsar proxy:
+
+* Connecting clients don't need to provide *any* specific configuration to use the Pulsar proxy. You won't need to update the client configuration for existing applications beyond updating the IP used for the service URL (for example if you're running a load balancer over the Pulsar proxy).
+* [TLS encryption and authentication](../../admin/Authz/#tls-client-auth) is supported by the Pulsar proxy
+
## Service discovery
[Clients](../../getting-started/Clients) connecting to Pulsar {% popover brokers %} need to be able to communicate with an entire Pulsar {% popover instance %} using a single URL. Pulsar provides a built-in service discovery mechanism that you can set up using the instructions in the [Deploying a Pulsar instance](../../deployment/InstanceSetup#service-discovery-setup) guide.
--
To stop receiving notification emails like this one, please contact
mmerli@apache.org.