You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by ur...@apache.org on 2022/09/20 06:21:21 UTC
[pulsar-site] branch main updated: Docs sync done from apache/pulsar(#3d35448)
This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new 11caad3de44 Docs sync done from apache/pulsar(#3d35448)
11caad3de44 is described below
commit 11caad3de44eb9cc7e777e49d746e95cc35d38c6
Author: Pulsar Site Updater <de...@pulsar.apache.org>
AuthorDate: Tue Sep 20 06:21:15 2022 +0000
Docs sync done from apache/pulsar(#3d35448)
---
site2/website-next/docs/client-libraries-java.md | 90 +-----
site2/website-next/docs/security-athenz.md | 42 ++-
site2/website-next/docs/security-basic-auth.md | 31 ++-
site2/website-next/docs/security-jwt.md | 242 ++++++++--------
site2/website-next/docs/security-kerberos.md | 30 +-
site2/website-next/docs/security-oauth2.md | 97 ++++---
.../docs/security-tls-authentication.md | 164 +++++++++--
site2/website-next/docs/security-tls-transport.md | 305 +++++++++++++++++++--
site2/website-next/sidebars.json | 42 ++-
9 files changed, 707 insertions(+), 336 deletions(-)
diff --git a/site2/website-next/docs/client-libraries-java.md b/site2/website-next/docs/client-libraries-java.md
index b0dc9068dcc..cbe3156b545 100644
--- a/site2/website-next/docs/client-libraries-java.md
+++ b/site2/website-next/docs/client-libraries-java.md
@@ -1258,92 +1258,18 @@ For examples of ProtobufNativeSchema, see [`SchemaDefinition` in `Complex type`]
## Authentication
-Pulsar currently supports three authentication schemes: [TLS](security-tls-authentication.md), [Athenz](security-athenz.md), and [Oauth2](security-oauth2.md). You can use the Pulsar Java client with all of them.
+Pulsar currently supports the following authentication mechansims:
+* [TLS](security-tls-authentication.md#configure-tls-authentication-in-pulsar-clients)
+* [JWT](security-jwt.md#configure-jwt-authentication-in-pulsar-clients)
+* [Athenz](security-athenz.md#configure-athenz-authentication-in-pulsar-clients)
+* [Kerberos](security-kerberos.md#java-client-and-java-admin-client)
+* [OAuth2](security-oauth2.md#configure-oauth2-authentication-in-pulsar-clients)
+* [HTTP basic](security-basic-auth.md#configure-basic-authentication-in-pulsar-clients)
-### TLS Authentication
-
-To use [TLS](security-tls-authentication.md), `enableTls` method is deprecated and you need to use "pulsar+ssl://" in serviceUrl to enable, point your Pulsar client to a TLS cert path, and provide paths to cert and key files.
-
-The following is an example.
-
-```java
-Map<String, String> authParams = new HashMap();
-authParams.put("tlsCertFile", "/path/to/client-cert.pem");
-authParams.put("tlsKeyFile", "/path/to/client-key.pem");
-
-Authentication tlsAuth = AuthenticationFactory
- .create(AuthenticationTls.class.getName(), authParams);
-
-PulsarClient client = PulsarClient.builder()
- .serviceUrl("pulsar+ssl://my-broker.com:6651")
- .tlsTrustCertsFilePath("/path/to/cacert.pem")
- .authentication(tlsAuth)
- .build();
-```
-
-### Athenz
-
-To use [Athenz](security-athenz.md) as an authentication provider, you need to [use TLS](#tls-authentication.md) and provide values for four parameters in a hash:
-
-* `tenantDomain`
-* `tenantService`
-* `providerDomain`
-* `privateKey`
-
-You can also set an optional `keyId`. The following is an example.
-
-```java
-Map<String, String> authParams = new HashMap();
-authParams.put("tenantDomain", "shopping"); // Tenant domain name
-authParams.put("tenantService", "some_app"); // Tenant service name
-authParams.put("providerDomain", "pulsar"); // Provider domain name
-authParams.put("privateKey", "file:///path/to/private.pem"); // Tenant private key path
-authParams.put("keyId", "v1"); // Key id for the tenant private key (optional, default: "0")
-
-Authentication athenzAuth = AuthenticationFactory
- .create(AuthenticationAthenz.class.getName(), authParams);
-
-PulsarClient client = PulsarClient.builder()
- .serviceUrl("pulsar+ssl://my-broker.com:6651")
- .tlsTrustCertsFilePath("/path/to/cacert.pem")
- .authentication(athenzAuth)
- .build();
-```
-
-#### Supported pattern formats
-The `privateKey` parameter supports the following three pattern formats:
-* `file:///path/to/file`
-* `file:/path/to/file`
-* `data:application/x-pem-file;base64,<base64-encoded value>`
-
-### Oauth2
-
-The following example shows how to use [Oauth2](security-oauth2.md) as an authentication provider for the Pulsar Java client.
-
-You can use the factory method to configure authentication for Pulsar Java client.
-
-```java
-PulsarClient client = PulsarClient.builder()
- .serviceUrl("pulsar://broker.example.com:6650/")
- .authentication(
- AuthenticationFactoryOAuth2.clientCredentials(this.issuerUrl, this.credentialsUrl, this.audience))
- .build();
-```
-
-In addition, you can also use the encoded parameters to configure authentication for Pulsar Java client.
-
-```java
-Authentication auth = AuthenticationFactory
- .create(AuthenticationOAuth2.class.getName(), "{"type":"client_credentials","privateKey":"...","issuerUrl":"...","audience":"..."}");
-PulsarClient client = PulsarClient.builder()
- .serviceUrl("pulsar://broker.example.com:6650/")
- .authentication(auth)
- .build();
-```
## Cluster-level failover
-For more concepts and reference information about cluster-level failover, including concept, benefits, use cases, constraints, usage and working principles, see [Cluster-level failover](concepts-cluster-level-failover.md).
+For more concepts and reference information about cluster-level failover, including concepts, benefits, use cases, constraints, usage and working principles, see [Cluster-level failover](concepts-cluster-level-failover.md).
:::tip
diff --git a/site2/website-next/docs/security-athenz.md b/site2/website-next/docs/security-athenz.md
index d46b8a76ff5..86c67d3180c 100644
--- a/site2/website-next/docs/security-athenz.md
+++ b/site2/website-next/docs/security-athenz.md
@@ -4,9 +4,9 @@ title: Authentication using Athenz
sidebar_label: "Authentication using Athenz"
---
-[Athenz](https://github.com/AthenZ/athenz) is a role-based authentication/authorization system. In Pulsar, you can use Athenz role tokens (also known as *z-tokens*) to establish the identify of the client.
+[Athenz](https://github.com/AthenZ/athenz) is a role-based authentication/authorization system. In Pulsar, you can use Athenz role tokens (also known as *z-tokens*) to establish the identity of the client.
-## Athenz authentication settings
+## Enable Athenz authentication
A [decentralized Athenz system](https://github.com/AthenZ/athenz/blob/master/docs/decent_authz_flow.md) contains an [authori**Z**ation **M**anagement **S**ystem](https://github.com/AthenZ/athenz/blob/master/docs/setup_zms.md) (ZMS) server and an [authori**Z**ation **T**oken **S**ystem](https://github.com/AthenZ/athenz/blob/master/docs/setup_zts) (ZTS) server.
@@ -36,7 +36,7 @@ Note that you can specify any action and resource in step 2 since they are not u
For more specific steps involving UI, refer to [Example Service Access Control Setup](https://github.com/AthenZ/athenz/blob/master/docs/example_service_athenz_setup.md#server-provider-domain).
-## Configure the broker for Athenz
+## Enable Athenz authentication on brokers
> ### TLS encryption
>
@@ -65,13 +65,41 @@ brokerClientAuthenticationParameters={"tenantDomain":"shopping","tenantService":
> A full listing of parameters is available in the `conf/broker.conf` file, you can also find the default
> values for those parameters in [Broker Configuration](reference-configuration.md#broker).
-## Configure clients for Athenz
+## Configure Athenz authentication in Pulsar clients
-For more information on Pulsar client authentication using Athenz, see the following language-specific docs:
+To use Athenz as an authentication provider, you need to [use TLS](#tls-authentication.md) and provide values for four parameters in a hash:
+* `tenantDomain`
+* `tenantService`
+* `providerDomain`
+* `privateKey`
-* [Java client](client-libraries-java.md#athenz)
+You can also set an optional `keyId`. The following is an example.
-## Configure CLI tools for Athenz
+```java
+Map<String, String> authParams = new HashMap();
+authParams.put("tenantDomain", "shopping"); // Tenant domain name
+authParams.put("tenantService", "some_app"); // Tenant service name
+authParams.put("providerDomain", "pulsar"); // Provider domain name
+authParams.put("privateKey", "file:///path/to/private.pem"); // Tenant private key path
+authParams.put("keyId", "v1"); // Key id for the tenant private key (optional, default: "0")
+
+Authentication athenzAuth = AuthenticationFactory
+ .create(AuthenticationAthenz.class.getName(), authParams);
+
+PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar+ssl://my-broker.com:6651")
+ .tlsTrustCertsFilePath("/path/to/cacert.pem")
+ .authentication(athenzAuth)
+ .build();
+```
+
+#### Supported pattern formats
+The `privateKey` parameter supports the following three pattern formats:
+* `file:///path/to/file`
+* `file:/path/to/file`
+* `data:application/x-pem-file;base64,<base64-encoded value>`
+
+## Configure Athenz authentication in CLI tools
[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
diff --git a/site2/website-next/docs/security-basic-auth.md b/site2/website-next/docs/security-basic-auth.md
index 105a35cbbea..608a1281bef 100644
--- a/site2/website-next/docs/security-basic-auth.md
+++ b/site2/website-next/docs/security-basic-auth.md
@@ -129,8 +129,11 @@ authParams={"userId":"superuser","password":"admin"}
The following example shows how to configure basic authentication when using Pulsar clients.
-<Tabs>
- <TabItem value="Java" label="Java" default>
+````mdx-code-block
+<Tabs groupId="lang-choice"
+ defaultValue="Java"
+ values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"C++","value":"C++"},{"label":"Go","value":"Go"}]}>
+<TabItem value="Java">
```java
AuthenticationBasic auth = new AuthenticationBasic();
@@ -142,7 +145,7 @@ The following example shows how to configure basic authentication when using Pul
```
</TabItem>
- <TabItem value="C++" label="C++" default>
+ <TabItem value="C++">
```cpp
#include <pulsar/Client.h>
@@ -158,4 +161,26 @@ The following example shows how to configure basic authentication when using Pul
```
</TabItem>
+ <TabItem value="Python">
+
+ ```python
+ if __name__ == "__main__":
+ client = Client("pulsar://broker.example.com:6650", authentication=AuthenticationBasic("admin", "123456"))
+ ```
+
+ </TabItem>
+ <TabItem value="Go">
+
+ ```go
+ provider, err := pulsar.NewAuthenticationBasic("admin", "123456")
+ if err != nil {
+ log.Fatal(err)
+ }
+ client, err := pulsar.NewClient(pulsar.ClientOptions{
+ URL: "pulsar://broker.example.com:6650",
+ Authentication: provider,
+ })
+ ```
+
+ </TabItem>
</Tabs>
diff --git a/site2/website-next/docs/security-jwt.md b/site2/website-next/docs/security-jwt.md
index b2064695388..374324a8d90 100644
--- a/site2/website-next/docs/security-jwt.md
+++ b/site2/website-next/docs/security-jwt.md
@@ -33,125 +33,6 @@ Always use TLS transport encryption when you connect to the Pulsar service, beca
:::
-### CLI Tools
-
-[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
-
-You need to add the following parameters to that file to use the token authentication with CLI tools of Pulsar:
-
-```conf
-webServiceUrl=http://broker.example.com:8080/
-brokerServiceUrl=pulsar://broker.example.com:6650/
-authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
-authParams=token:eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY
-```
-
-The token string can also be read from a file, for example:
-
-```conf
-authParams=file:///path/to/token/file
-```
-
-### Pulsar client
-
-You can use tokens to authenticate the following Pulsar clients.
-
-````mdx-code-block
-<Tabs groupId="lang-choice"
- defaultValue="Java"
- values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"Go","value":"Go"},{"label":"C++","value":"C++"},{"label":"C#","value":"C#"}]}>
-<TabItem value="Java">
-
-```java
-PulsarClient client = PulsarClient.builder()
- .serviceUrl("pulsar://broker.example.com:6650/")
- .authentication(
- AuthenticationFactory.token("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY")）
- .build();
-```
-
-Similarly, you can also pass a `Supplier`:
-
-```java
-PulsarClient client = PulsarClient.builder()
- .serviceUrl("pulsar://broker.example.com:6650/")
- .authentication(
- AuthenticationFactory.token(() -> {
- // Read token from custom source
- return readToken();
- }))
- .build();
-```
-
-</TabItem>
-<TabItem value="Python">
-
-```python
-from pulsar import Client, AuthenticationToken
-
-client = Client('pulsar://broker.example.com:6650/'
- authentication=AuthenticationToken('eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY'))
-```
-
-Alternatively, you can also pass a `Supplier`:
-
-```python
-def read_token():
- with open('/path/to/token.txt') as tf:
- return tf.read().strip()
-
-client = Client('pulsar://broker.example.com:6650/'
- authentication=AuthenticationToken(read_token))
-```
-
-</TabItem>
-<TabItem value="Go">
-
-```go
-client, err := NewClient(ClientOptions{
- URL: "pulsar://localhost:6650",
- Authentication: NewAuthenticationToken("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY"),
-})
-```
-
-Similarly, you can also pass a `Supplier`:
-
-```go
-client, err := NewClient(ClientOptions{
- URL: "pulsar://localhost:6650",
- Authentication: NewAuthenticationTokenSupplier(func () string {
- // Read token from custom source
- return readToken()
- }),
-})
-```
-
-</TabItem>
-<TabItem value="C++">
-
-```cpp
-#include <pulsar/Client.h>
-
-pulsar::ClientConfiguration config;
-config.setAuth(pulsar::AuthToken::createWithToken("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY"));
-
-pulsar::Client client("pulsar://broker.example.com:6650/", config);
-```
-
-</TabItem>
-<TabItem value="C#">
-
-```csharp
-var client = PulsarClient.Builder()
- .AuthenticateUsingToken("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY")
- .Build();
-```
-
-</TabItem>
-
-</Tabs>
-````
-
## Enable token authentication
JWT supports two different kinds of keys to generate and validate the tokens:
@@ -225,7 +106,7 @@ The token itself does not have any permission associated. The authorization engi
:::
-### Enable token authentication on Brokers
+### Enable token authentication on brokers
To configure brokers to authenticate clients, add the following parameters to the `conf/broker.conf` or `conf/standalone.conf` file.
@@ -262,7 +143,7 @@ Equivalent to `brokerClientAuthenticationParameters`, you need to configure `aut
:::
-### Enable token authentication on Proxies
+### Enable token authentication on proxies
To configure proxies to authenticate clients, add the following parameters to the `conf/proxy.conf` file.
@@ -282,3 +163,122 @@ brokerClientAuthenticationParameters={"token":"eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ0
```
The proxy uses its own token when connecting to brokers. You need to configure the role token for this key pair in the `proxyRoles` of the brokers. For more details, see [authorization](security-authorization.md).
+
+### Configure JWT authentication in CLI Tools
+
+[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
+
+You need to add the following parameters to that file to use the token authentication with CLI tools of Pulsar:
+
+```conf
+webServiceUrl=http://broker.example.com:8080/
+brokerServiceUrl=pulsar://broker.example.com:6650/
+authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken
+authParams=token:eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY
+```
+
+The token string can also be read from a file, for example:
+
+```conf
+authParams=file:///path/to/token/file
+```
+
+### Configure JWT authentication in Pulsar clients
+
+You can use tokens to authenticate the following Pulsar clients.
+
+````mdx-code-block
+<Tabs groupId="lang-choice"
+ defaultValue="Java"
+ values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"Go","value":"Go"},{"label":"C++","value":"C++"},{"label":"C#","value":"C#"}]}>
+<TabItem value="Java">
+
+```java
+PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar://broker.example.com:6650/")
+ .authentication(
+ AuthenticationFactory.token("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY")）
+ .build();
+```
+
+Similarly, you can also pass a `Supplier`:
+
+```java
+PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar://broker.example.com:6650/")
+ .authentication(
+ AuthenticationFactory.token(() -> {
+ // Read token from custom source
+ return readToken();
+ }))
+ .build();
+```
+
+</TabItem>
+<TabItem value="Python">
+
+```python
+from pulsar import Client, AuthenticationToken
+
+client = Client('pulsar://broker.example.com:6650/'
+ authentication=AuthenticationToken('eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY'))
+```
+
+Alternatively, you can also pass a `Supplier`:
+
+```python
+def read_token():
+ with open('/path/to/token.txt') as tf:
+ return tf.read().strip()
+
+client = Client('pulsar://broker.example.com:6650/'
+ authentication=AuthenticationToken(read_token))
+```
+
+</TabItem>
+<TabItem value="Go">
+
+```go
+client, err := pulsar.NewClient(pulsar.ClientOptions{
+ URL: "pulsar://localhost:6650",
+ Authentication: NewAuthenticationToken("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY"),
+})
+```
+
+Similarly, you can also pass a `Supplier`:
+
+```go
+client, err := pulsar.NewClient(pulsar.ClientOptions{
+ URL: "pulsar://localhost:6650",
+ Authentication: pulsar.NewAuthenticationTokenSupplier(func () string {
+ // Read token from custom source
+ return readToken()
+ }),
+})
+```
+
+</TabItem>
+<TabItem value="C++">
+
+```cpp
+#include <pulsar/Client.h>
+
+pulsar::ClientConfiguration config;
+config.setAuth(pulsar::AuthToken::createWithToken("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY"));
+
+pulsar::Client client("pulsar://broker.example.com:6650/", config);
+```
+
+</TabItem>
+<TabItem value="C#">
+
+```csharp
+var client = PulsarClient.Builder()
+ .AuthenticateUsingToken("eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJKb2UifQ.ipevRNuRP6HflG8cFKnmUPtypruRC4fb1DWtoLL62SY")
+ .Build();
+```
+
+</TabItem>
+
+</Tabs>
+````
\ No newline at end of file
diff --git a/site2/website-next/docs/security-kerberos.md b/site2/website-next/docs/security-kerberos.md
index 03b1a320fe6..506dc1573c6 100644
--- a/site2/website-next/docs/security-kerberos.md
+++ b/site2/website-next/docs/security-kerberos.md
@@ -10,7 +10,7 @@ In Pulsar, you can use Kerberos with [SASL](https://en.wikipedia.org/wiki/Simple
This document introduces how to configure `Kerberos` with `SASL` between Pulsar clients and brokers and how to configure Kerberos for Pulsar proxy in detail.
-## Configuration for Kerberos between Client and Broker
+## Configure Kerberos between client and broker
### Prerequisites
@@ -40,7 +40,7 @@ Note that *Kerberos* requires that all your hosts can be resolved with their FQD
The first part of broker principal (for example, `broker` in `broker/{hostname}@{REALM}`) is the `serverType` of each host. The suggested values of `serverType` are `broker` (host machine runs service Pulsar brokers) and `proxy` (host machine runs service Pulsar Proxy).
-#### Configure how to connect to KDC
+#### Connect to KDC
You need to enter the command below to specify the path to the `krb5.conf` file for the client side and the broker side. The content of `krb5.conf` file indicates the default Realm and KDC information. See [JDK’s Kerberos Requirements](https://docs.oracle.com/javase/8/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html) for more details.
@@ -62,7 +62,7 @@ Here is an example of the krb5.conf file. `EXAMPLE.COM` is the default realm; `k
Usually machines configured with kerberos already have a system-wide configuration and this configuration is optional.
-#### JAAS configuration file
+#### Configure JAAS configuration file
You need the JAAS configuration file for the client side and the broker side. JAAS configuration file provides the section of information that is used to connect KDC. Here is an example named `pulsar_jaas.conf`:
@@ -103,7 +103,7 @@ You can have 2 separate JAAS configuration files:
* the file for a client that only has a `PulsarClient` section.
-### Kerberos configuration for Brokers
+### Configure Kerberos authentication for brokers
#### Configure the `broker.conf` file
@@ -131,7 +131,7 @@ brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.Authenticati
brokerClientAuthenticationParameters={"saslJaasClientSectionName":"PulsarClient", "serverType":"broker"}
```
-#### Set Broker JVM parameter
+#### Set broker JVM parameters
Set JVM parameters for the JAAS configuration file and krb5 configuration file with additional options.
@@ -143,7 +143,7 @@ You can add this at the end of `PULSAR_EXTRA_OPTS` in the file [`pulsar_env.sh`]
You must ensure that the operating system user who starts broker can reach the keytabs configured in the `pulsar_jaas.conf` file and kdc server in the `krb5.conf` file.
-### Kerberos configuration for clients
+### Configure Kerberos authentication for clients
#### Java Client and Java Admin Client
@@ -214,7 +214,7 @@ or add this line `OPTS="$OPTS -Djava.security.auth.login.config=/etc/pulsar/puls
The meaning of configurations is the same as the meaning of configurations in Java client section.
-## Kerberos configuration for working with Pulsar Proxy
+## Configure Kerberos authentication for proxies
With the above configuration, clients and brokers can do authentication using Kerberos.
@@ -222,7 +222,7 @@ A client that connects to Pulsar Proxy is a little different. Pulsar Proxy (as a
Now in comparison with the above configuration between client and broker, we show you how to configure Pulsar Proxy as follows.
-### Create principal for Pulsar Proxy in Kerberos
+### Create principals in Kerberos
You need to add new principals for Pulsar proxy compared with the above configuration. If you already have principals for client and broker, you only need to add the proxy principal here.
@@ -238,7 +238,7 @@ sudo /usr/sbin/kadmin.local -q 'addprinc -randkey client/{hostname}@{REALM}'
sudo /usr/sbin/kadmin.local -q "ktadd -k /etc/security/keytabs/{client-keytabname}.keytab client/{hostname}@{REALM}"
```
-### Add a section in JAAS configuration file for Pulsar Proxy
+### Add a section in JAAS configuration file
In comparison with the above configuration, add a new section for Pulsar Proxy in the JAAS configuration file.
@@ -273,7 +273,7 @@ Here is an example named `pulsar_jaas.conf`:
};
```
-### Proxy client configuration
+### Configure proxy clients
Pulsar client configuration is similar to client and broker configuration, except that you need to set `serverType` to `proxy` instead of `broker`, for the reason that you need to do the Kerberos authentication between client and proxy.
@@ -300,7 +300,7 @@ Pulsar client configuration is similar to client and broker configuration, excep
java -cp -Djava.security.auth.login.config=/etc/pulsar/pulsar_jaas.conf -Djava.security.krb5.conf=/etc/pulsar/krb5.conf $APP-jar-with-dependencies.jar $CLASSNAME
```
-### Kerberos configuration for Pulsar proxy service
+### Configure proxy service
In the `proxy.conf` file, set Kerberos-related configuration. Here is an example:
@@ -321,7 +321,7 @@ The first part relates to authenticating between clients and Pulsar proxies. In
The second part relates to authenticating between Pulsar proxies and Pulsar brokers. In this phase, Pulsar Proxy works as SASL client, while Pulsar broker works as SASL server.
-### Broker side configuration.
+### Configure brokers
The broker-side configuration file is the same as the above `broker.conf`, you do not need special configurations for Pulsar Proxy.
@@ -332,7 +332,7 @@ saslJaasClientAllowedIds=.*client.*
saslJaasServerSectionName=PulsarBroker
```
-## Regarding authorization and role token
+## Authorization and role token
For Kerberos authentication, we usually use the authenticated principal as the role token for Pulsar authorization. For more information on authorization in Pulsar, see [security authorization](security-authorization.md).
@@ -344,7 +344,7 @@ For example:
superUserRoles=client/{clientIp}@EXAMPLE.COM
```
-## Regarding authentication between ZooKeeper and Broker
+## Configure Kerberos authentication between ZooKeeper and broker
Pulsar broker acts as a Kerberos client when you authenticate with Zookeeper. According to [ZooKeeper document](https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication), you need these settings in `conf/zookeeper.conf`:
@@ -368,7 +368,7 @@ Enter the following commands to add a section of `Client` configurations in the
In this setting, the principal of Pulsar broker and keyTab file indicates the role of brokers when you authenticate with ZooKeeper.
-## Regarding authentication between BookKeeper and Broker
+## Configure Kerberos authentication between BookKeeper and broker
Pulsar broker acts as a Kerberos client when you authenticate with Bookie. According to [BookKeeper document](https://bookkeeper.apache.org/docs/next/security/sasl/), you need to add `bookkeeperClientAuthenticationPlugin` parameter in `broker.conf`:
diff --git a/site2/website-next/docs/security-oauth2.md b/site2/website-next/docs/security-oauth2.md
index 5ce2adaa11a..649bac85af9 100644
--- a/site2/website-next/docs/security-oauth2.md
+++ b/site2/website-next/docs/security-oauth2.md
@@ -69,13 +69,15 @@ In the above example, the mapping relationship is shown as below.
- The `privateKey` file parameter in this plugin should at least contains the `client_id` and `client_secret` fields.
- The `audience` parameter in this plugin is mapped to `"audience":"https://dev-kt-aa9ne.us.auth0.com/api/v2/"`. This field is only used by some identity providers.
-## Client Configuration
+## Configure OAuth2 authentication in Pulsar clients
You can use the OAuth2 authentication provider with the following Pulsar clients.
-### Java client
-
-You can use the factory method to configure authentication for Pulsar Java client.
+````mdx-code-block
+<Tabs groupId="lang-choice"
+ defaultValue="Java"
+ values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"C++","value":"C++"},{"label":"Node.js","value":"Node.js"},{"label":"Go","value":"Go"}]}>
+<TabItem value="Java">
```java
import org.apache.pulsar.client.impl.auth.oauth2.AuthenticationFactoryOAuth2;
@@ -102,47 +104,8 @@ PulsarClient client = PulsarClient.builder()
.build();
```
-### C++ client
-
-The C++ client is similar to the Java client. You need to provide the parameters of `issuerUrl`, `private_key` (the credentials file path), and `audience`.
-
-```cpp
-#include <pulsar/Client.h>
-
-pulsar::ClientConfiguration config;
-std::string params = R"({
- "issuer_url": "https://dev-kt-aa9ne.us.auth0.com",
- "private_key": "../../pulsar-broker/src/test/resources/authentication/token/cpp_credentials_file.json",
- "audience": "https://dev-kt-aa9ne.us.auth0.com/api/v2/"})";
-
-config.setAuth(pulsar::AuthOauth2::create(params));
-
-pulsar::Client client("pulsar://broker.example.com:6650/", config);
-```
-
-### Go client
-
-To enable OAuth2 authentication in Go client, you need to configure OAuth2 authentication.
-This example shows how to configure OAuth2 authentication in Go client.
-
-```go
-oauth := pulsar.NewAuthenticationOAuth2(map[string]string{
- "type": "client_credentials",
- "issuerUrl": "https://dev-kt-aa9ne.us.auth0.com",
- "audience": "https://dev-kt-aa9ne.us.auth0.com/api/v2/",
- "privateKey": "/path/to/privateKey",
- "clientId": "0Xx...Yyxeny",
- })
-client, err := pulsar.NewClient(pulsar.ClientOptions{
- URL: "pulsar://my-cluster:6650",
- Authentication: oauth,
-})
-```
-
-### Python client
-
-To enable OAuth2 authentication in Python client, you need to configure OAuth2 authentication.
-This example shows how to configure OAuth2 authentication in Python client.
+</TabItem>
+<TabItem value="Python">
```python
from pulsar import Client, AuthenticationOauth2
@@ -158,10 +121,25 @@ params = '''
client = Client("pulsar://my-cluster:6650", authentication=AuthenticationOauth2(params))
```
-### Node.js client
+</TabItem>
+<TabItem value="C++">
+
+```cpp
+#include <pulsar/Client.h>
+
+pulsar::ClientConfiguration config;
+std::string params = R"({
+ "issuer_url": "https://dev-kt-aa9ne.us.auth0.com",
+ "private_key": "../../pulsar-broker/src/test/resources/authentication/token/cpp_credentials_file.json",
+ "audience": "https://dev-kt-aa9ne.us.auth0.com/api/v2/"})";
+
+config.setAuth(pulsar::AuthOauth2::create(params));
+
+pulsar::Client client("pulsar://broker.example.com:6650/", config);
+```
-To enable OAuth2 authentication in Node.js client, you need to configure OAuth2 authentication.
-This example shows how to configure OAuth2 authentication in Node.js client.
+</TabItem>
+<TabItem value="Node.js">
```javascript
const Pulsar = require('pulsar-client');
@@ -205,6 +183,27 @@ The support for OAuth2 authentication is only available in Node.js client 1.6.2
:::
+</TabItem>
+<TabItem value="Go">
+
+```go
+oauth := pulsar.NewAuthenticationOAuth2(map[string]string{
+ "type": "client_credentials",
+ "issuerUrl": "https://dev-kt-aa9ne.us.auth0.com",
+ "audience": "https://dev-kt-aa9ne.us.auth0.com/api/v2/",
+ "privateKey": "/path/to/privateKey",
+ "clientId": "0Xx...Yyxeny",
+ })
+client, err := pulsar.NewClient(pulsar.ClientOptions{
+ URL: "pulsar://my-cluster:6650",
+ Authentication: oauth,
+})
+```
+
+</TabItem>
+</Tabs>
+````
+
## Broker configuration
To enable OAuth2 authentication in brokers, add the following parameters to the `broker.conf` or `standalone.conf` file.
@@ -220,7 +219,7 @@ brokerClientAuthenticationParameters={"privateKey":"/path/to/privateKey",\
"audience":"https://dev-kt-aa9ne.us.auth0.com/api/v2/","issuerUrl":"https://dev-kt-aa9ne.us.auth0.com"}
```
-## CLI configuration
+## Configure OAuth2 authentication in CLI tools
This section describes how to use Pulsar CLI tools to connect a cluster through OAuth2 authentication plugin.
diff --git a/site2/website-next/docs/security-tls-authentication.md b/site2/website-next/docs/security-tls-authentication.md
index dfa6b7cdb37..f2e393abd74 100644
--- a/site2/website-next/docs/security-tls-authentication.md
+++ b/site2/website-next/docs/security-tls-authentication.md
@@ -4,17 +4,22 @@ title: Authentication using TLS
sidebar_label: "Authentication using TLS"
---
+````mdx-code-block
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+````
+
## TLS authentication overview
TLS authentication is an extension of [TLS transport encryption](security-tls-transport.md). Not only servers have keys and certs that the client uses to verify the identity of servers, clients also have keys and certs that the server uses to verify the identity of clients. You must have TLS transport encryption configured on your cluster before you can use TLS authentication. This guide assumes you already have TLS transport encryption configured.
-### Create client certificates
+## Create client certificates
Client certificates are generated using the certificate authority. Server certificates are also generated with the same certificate authority.
The biggest difference between client certs and server certs is that the **common name** for the client certificate is the **role token** that the client is authenticated as.
-To use client certificates, you need to set `tlsRequireTrustedClientCertOnConnect=true` at the broker side. For details, refer to [TLS broker configuration](security-tls-transport.md#configure-broker).
+To use client certificates, you need to set `tlsRequireTrustedClientCertOnConnect=true` at the broker side. For details, refer to [TLS broker configuration](security-tls-transport.md#configure-brokers).
First, you need to enter the following command to generate the key :
@@ -65,7 +70,7 @@ cd /etc/pki/tls/misc/CA
## Enable TLS authentication on brokers
-To configure brokers to authenticate clients, add the following parameters to `broker.conf`, alongside [the configuration to enable TLS transport](security-tls-transport.md#broker-configuration):
+To configure brokers to authenticate clients, add the following parameters to `broker.conf`, alongside [the configuration to enable TLS transport](security-tls-transport.md#configure-brokers):
```properties
# Configuration to enable authentication
@@ -80,7 +85,7 @@ brokerClientTrustCertsFilePath=/path/my-ca/certs/ca.cert.pem
## Enable TLS authentication on proxies
-To configure proxies to authenticate clients, add the following parameters to `proxy.conf`, alongside [the configuration to enable TLS transport](security-tls-transport.md#proxy-configuration):
+To configure proxies to authenticate clients, add the following parameters to `proxy.conf`, alongside [the configuration to enable TLS transport](security-tls-transport.md#configure-proxies):
The proxy should have its own client key pair for connecting to brokers. You need to configure the role token for this key pair in the `proxyRoles` of the brokers. See the [authorization guide](security-authorization.md) for more details.
@@ -95,22 +100,15 @@ brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.Authenticati
brokerClientAuthenticationParameters=tlsCertFile:/path/to/proxy.cert.pem,tlsKeyFile:/path/to/proxy.key-pk8.pem
```
-## Client configuration
+## Configure TLS authentication in Pulsar clients
When you use TLS authentication, client connects via TLS transport. You need to configure the client to use `https://` and 8443 port for the web service URL, `pulsar+ssl://` and 6651 port for the broker service URL.
-### CLI tools
-
-[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
-
-To use TLS authentication with the CLI tools of Pulsar, you need to add the following parameters to the `conf/client.conf` file, alongside [the configuration to enable TLS transport](security-tls-transport.md#cli-tools):
-
-```properties
-authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
-authParams=tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem
-```
-
-### Java client
+````mdx-code-block
+<Tabs groupId="lang-choice"
+ defaultValue="Java"
+ values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"C++","value":"C++"},{"label":"Node.js","value":"Node.js"},{"label":"C#","value":"C#"}]}>
+<TabItem value="Java">
```java
import org.apache.pulsar.client.api.PulsarClient;
@@ -123,7 +121,8 @@ PulsarClient client = PulsarClient.builder()
.build();
```
-### Python client
+</TabItem>
+<TabItem value="Python">
```python
from pulsar import Client, AuthenticationTLS
@@ -135,7 +134,8 @@ client = Client("pulsar+ssl://broker.example.com:6651/",
authentication=auth)
```
-### C++ client
+</TabItem>
+<TabItem value="C++">
```cpp
#include <pulsar/Client.h>
@@ -152,7 +152,8 @@ config.setAuth(auth);
pulsar::Client client("pulsar+ssl://broker.example.com:6651/", config);
```
-### Node.js client
+</TabItem>
+<TabItem value="Node.js">
```javascript
const Pulsar = require('pulsar-client');
@@ -171,7 +172,8 @@ const Pulsar = require('pulsar-client');
})();
```
-### C# client
+</TabItem>
+<TabItem value="C#">
```csharp
var clientCertificate = new X509Certificate2("admin.pfx");
@@ -180,3 +182,123 @@ var client = PulsarClient.Builder()
.Build();
```
+</TabItem>
+</Tabs>
+````
+
+## Configure TLS authentication in CLI tools
+
+[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
+
+To use TLS authentication with the CLI tools of Pulsar, you need to add the following parameters to the `conf/client.conf` file, alongside [the configuration to enable TLS encryption](security-tls-transport.md#configure-tls-encryption-in-cli-tools):
+
+```properties
+authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
+authParams=tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem
+```
+
+## Configure TLS authentication with KeyStore
+
+Apache Pulsar supports [TLS encryption](security-tls-transport.md) and [TLS authentication](security-tls-authentication.md) between clients and Apache Pulsar service. By default, it uses PEM format file configuration. This section tries to describe how to use [KeyStore](https://en.wikipedia.org/wiki/Java_KeyStore) type to configure TLS.
+
+### Configure brokers
+
+Configure the `broker.conf` file as follows.
+
+```properties
+# Configuration to enable authentication
+authenticationEnabled=true
+authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
+
+# Enable KeyStore type
+tlsEnabledWithKeyStore=true
+tlsRequireTrustedClientCertOnConnect=true
+
+# key store
+tlsKeyStoreType=JKS
+tlsKeyStore=/var/private/tls/broker.keystore.jks
+tlsKeyStorePassword=brokerpw
+
+# trust store
+tlsTrustStoreType=JKS
+tlsTrustStore=/var/private/tls/broker.truststore.jks
+tlsTrustStorePassword=brokerpw
+
+# internal client/admin-client config
+brokerClientTlsEnabled=true
+brokerClientTlsEnabledWithKeyStore=true
+brokerClientTlsTrustStoreType=JKS
+brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks
+brokerClientTlsTrustStorePassword=clientpw
+# internal auth config
+brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
+brokerClientAuthenticationParameters={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"}
+```
+
+### Configure clients
+
+Besides configuring [TLS encryption](security-tls-transport.md), you need to configure the KeyStore, which contains a valid CN as client role, for clients.
+
+For example:
+
+1. for [Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](reference-cli-tools#pulsar-admin), [`pulsar-perf`](reference-cli-tools#pulsar-perf), and [`pulsar-client`](reference-cli-tools#pulsar-client), set the `conf/client.conf` file in a Pulsar installation.
+
+ ```properties
+ webServiceUrl=https://broker.example.com:8443/
+ brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
+ useKeyStoreTls=true
+ tlsTrustStoreType=JKS
+ tlsTrustStorePath=/var/private/tls/client.truststore.jks
+ tlsTrustStorePassword=clientpw
+ authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
+ authParams={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"}
+ ```
+
+1. for Java client
+
+ ```java
+ import org.apache.pulsar.client.api.PulsarClient;
+
+ PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar+ssl://broker.example.com:6651/")
+ .useKeyStoreTls(true)
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .allowTlsInsecureConnection(false)
+ .enableTlsHostnameVerification(false)
+ .authentication(
+ "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
+ "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
+ .build();
+ ```
+
+1. for Java admin client
+
+ ```java
+ PulsarAdmin amdin = PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443")
+ .useKeyStoreTls(true)
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .allowTlsInsecureConnection(false)
+ .enableTlsHostnameVerification(false)
+ .authentication(
+ "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
+ "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
+ .build();
+ ```
+
+:::note
+
+Configure `tlsTrustStorePath` when you set `useKeyStoreTls` to `true`.
+
+:::
+
+## Enable TLS Logging
+
+You can enable TLS debug logging at the JVM level by starting the brokers and/or clients with `javax.net.debug` system property. For example:
+
+```shell
+-Djavax.net.debug=all
+```
+
+You can find more details on this in [Oracle documentation](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html) on [debugging SSL/TLS connections](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html).
diff --git a/site2/website-next/docs/security-tls-transport.md b/site2/website-next/docs/security-tls-transport.md
index 21d88dc5e18..453b6ce608a 100644
--- a/site2/website-next/docs/security-tls-transport.md
+++ b/site2/website-next/docs/security-tls-transport.md
@@ -4,6 +4,12 @@ title: Transport Encryption using TLS
sidebar_label: "Transport Encryption using TLS"
---
+
+````mdx-code-block
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+````
+
## TLS overview
By default, Apache Pulsar clients communicate with the Apache Pulsar service in plain text. This means that all data is sent in the clear. You can use TLS to encrypt this traffic to protect the traffic from the snooping of a man-in-the-middle attacker.
@@ -164,7 +170,7 @@ openssl ca -config openssl.cnf -extensions client_cert \
At this point, you have a cert `client.cert.pem` and a key `client.key-pk8.pem`, which you can use along with `ca.cert.pem` to configure TLS encryption for your client.
-## Configure broker
+## Configure brokers
To configure a Pulsar [broker](reference-terminology.md#broker) to use TLS transport encryption, you need to make some changes to `broker.conf`, which locates in the `conf` directory of your [Pulsar installation](getting-started-standalone.md).
@@ -186,7 +192,7 @@ brokerClientKeyFilePath=/path/to/client.key-pk8.pem
> You can find a full list of parameters available in the `conf/broker.conf` file,
> as well as the default values for those parameters, in [Broker Configuration](reference-configuration.md#broker)
->
+
### TLS Protocol Version and Cipher
You can configure the broker (and proxy) to require specific TLS protocol versions and ciphers for TLS negiotation. You can use the TLS protocol versions and ciphers to stop clients from requesting downgraded TLS protocol versions or ciphers that may have weaknesses.
@@ -204,7 +210,7 @@ For JDK 11, you can obtain a list of supported values from the documentation:
- [TLS protocol](https://docs.oracle.com/en/java/javase/11/security/oracle-providers.html#GUID-7093246A-31A3-4304-AC5F-5FB6400405E2__SUNJSSEPROVIDERPROTOCOLPARAMETERS-BBF75009)
- [Ciphers](https://docs.oracle.com/en/java/javase/11/security/oracle-providers.html#GUID-7093246A-31A3-4304-AC5F-5FB6400405E2__SUNJSSE_CIPHER_SUITES)
-## Proxy Configuration
+## Configure proxies
Proxies need to configure TLS in two directions, for clients connecting to the proxy, and for the proxy connecting to brokers.
@@ -225,7 +231,7 @@ brokerClientCertificateFilePath=/path/to/client.cert.pem
brokerClientKeyFilePath=/path/to/client.key-pk8.pem
```
-## Client configuration
+## Configure clients
When you enable the TLS transport encryption, you need to configure the client to use `https://` and port 8443 for the web service URL, and `pulsar+ssl://` and port 6651 for the broker service URL.
@@ -239,25 +245,13 @@ Moreover, as the administrator has full control of the certificate authority, a
One scenario where you may want to enable hostname verification is where you have multiple proxy nodes behind a VIP, and the VIP has a DNS record, for example, pulsar.mycompany.com. In this case, you can generate a TLS cert with pulsar.mycompany.com as the "CommonName," and then enable hostname verification on the client.
-The examples below show that hostname verification is disabled for the CLI tools/Java/Python/C++/Node.js/C# clients by default.
-
-### CLI tools
-
-[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](reference-cli-tools.md#pulsar-admin), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
-
-You need to add the following parameters to that file to use TLS transport with the CLI tools of Pulsar:
-
-```properties
-webServiceUrl=https://broker.example.com:8443/
-brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
-tlsAllowInsecureConnection=false
-tlsTrustCertsFilePath=/path/to/ca.cert.pem
-tlsKeyFilePath=/path/to/client.key-pk8.pem
-tlsCertFile=/path/to/client-cert.pem
-tlsEnableHostnameVerification=false
-```
+The examples below show that hostname verification is disabled for Java/Python/C++/Node.js/C# clients by default.
-#### Java client
+````mdx-code-block
+<Tabs groupId="lang-choice"
+ defaultValue="Java"
+ values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"C++","value":"C++"},{"label":"Node.js","value":"Node.js"},{"label":"C#","value":"C#"}]}>
+<TabItem value="Java">
```java
import org.apache.pulsar.client.api.PulsarClient;
@@ -272,7 +266,8 @@ PulsarClient client = PulsarClient.builder()
.build();
```
-#### Python client
+</TabItem>
+<TabItem value="Python">
```python
from pulsar import Client
@@ -283,7 +278,8 @@ client = Client("pulsar+ssl://broker.example.com:6651/",
tls_allow_insecure_connection=False) // defaults to false from v2.2.0 onwards
```
-#### C++ client
+</TabItem>
+<TabItem value="C++">
```cpp
#include <pulsar/Client.h>
@@ -296,7 +292,8 @@ config.setAuth(pulsar::AuthTls::create(clientPublicKeyPath, clientPrivateKeyPath
config.setValidateHostName(false);
```
-#### Node.js client
+</TabItem>
+<TabItem value="Node.js">
```javascript
const Pulsar = require('pulsar-client');
@@ -312,7 +309,8 @@ const Pulsar = require('pulsar-client');
})();
```
-#### C# client
+</TabItem>
+<TabItem value="C#">
```csharp
var certificate = new X509Certificate2("ca.cert.pem");
@@ -324,3 +322,258 @@ var client = PulsarClient.Builder()
```
> Note that `VerifyCertificateName` refers to the configuration of hostname verification in the C# client.
+
+</TabItem>
+</Tabs>
+````
+
+## Configure TLS encryption in CLI tools
+
+[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](reference-cli-tools.md#pulsar-admin), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
+
+To use TLS encryption with the CLI tools of Pulsar, you need to add the following parameters to the `conf/client.conf` file.
+
+```properties
+webServiceUrl=https://broker.example.com:8443/
+brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
+tlsAllowInsecureConnection=false
+tlsTrustCertsFilePath=/path/to/ca.cert.pem
+tlsKeyFilePath=/path/to/client.key-pk8.pem
+tlsCertFile=/path/to/client-cert.pem
+tlsEnableHostnameVerification=false
+```
+
+## Configure TLS encryption with KeyStore
+
+Apache Pulsar supports [TLS encryption](security-tls-transport.md) and [TLS authentication](security-tls-authentication.md) between clients and Apache Pulsar service. By default, it uses PEM format file configuration. This section tries to describe how to use [KeyStore](https://en.wikipedia.org/wiki/Java_KeyStore) type to configure TLS.
+
+### Generate TLS key and certificate
+
+The first step of deploying TLS is to generate the key and the certificate for each machine in the cluster. You can use Java’s `keytool` utility to accomplish this task. We will generate the key into a temporary keystore initially for broker, so that we can export and sign it later with CA.
+
+```shell
+keytool -keystore broker.keystore.jks -alias localhost -validity {validity} -genkeypair -keyalg RSA
+```
+
+You need to specify two parameters in the above command:
+
+1. `keystore`: the keystore file that stores the certificate. The *keystore* file contains the private key of
+ the certificate; hence, it needs to be kept safely.
+2. `validity`: the valid time of the certificate in days.
+
+> Ensure that common name (CN) matches exactly with the fully qualified domain name (FQDN) of the server.
+The client compares the CN with the DNS domain name to ensure that it is indeed connecting to the desired server, not a malicious one.
+
+### Create your own CA
+
+After the first step, each broker in the cluster has a public-private key pair, and a certificate to identify the machine.
+The certificate, however, is unsigned, which means that an attacker can create such a certificate to pretend to be any machine.
+
+Therefore, it is important to prevent forged certificates by signing them for each machine in the cluster.
+A `certificate authority (CA)` is responsible for signing certificates. CA works like a government that issues passports —
+the government stamps (signs) each passport so that the passport becomes difficult to forge. Other governments verify the stamps
+to ensure the passport is authentic. Similarly, the CA signs the certificates, and the cryptography guarantees that a signed
+certificate is computationally difficult to forge. Thus, as long as the CA is a genuine and trusted authority, the clients have
+high assurance that they are connecting to authentic machines.
+
+```shell
+openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
+```
+
+The generated CA is simply a *public-private* key pair and certificate, and it is intended to sign other certificates.
+
+The next step is to add the generated CA to the clients' truststore so that the clients can trust this CA:
+
+```shell
+keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
+```
+
+:::note
+
+If you configure the brokers to require client authentication by setting `tlsRequireTrustedClientCertOnConnect` to `true` on the broker configuration, then you must also provide a truststore for the brokers and it should have all the CA certificates that clients keys were signed by.
+
+:::
+
+```shell
+keytool -keystore broker.truststore.jks -alias CARoot -import -file ca-cert
+```
+
+In contrast to the keystore, which stores each machine’s own identity, the truststore of a client stores all the certificates
+that the client should trust. Importing a certificate into one’s truststore also means trusting all certificates that are signed
+by that certificate. As the analogy above, trusting the government (CA) also means trusting all passports (certificates) that
+it has issued. This attribute is called the chain of trust, and it is particularly useful when deploying TLS on a large BookKeeper cluster.
+You can sign all certificates in the cluster with a single CA, and have all machines share the same truststore that trusts the CA.
+That way all machines can authenticate all other machines.
+
+
+### Sign the certificate
+
+The next step is to sign all certificates in the keystore with the CA we generated.
+
+1. Export the certificate from the keystore:
+
+```shell
+keytool -keystore broker.keystore.jks -alias localhost -certreq -file cert-file
+keytool -keystore client.keystore.jks -alias localhost -certreq -file cert-file
+```
+
+2. Sign it with the CA:
+
+```shell
+openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days {validity} -CAcreateserial -passin pass:{ca-password}
+```
+
+3. Import both the certificate of the CA and the signed certificate into the keystore:
+
+```shell
+keytool -keystore broker.keystore.jks -alias CARoot -import -file ca-cert
+keytool -keystore broker.keystore.jks -alias localhost -import -file cert-signed
+
+keytool -keystore client.keystore.jks -alias CARoot -import -file ca-cert
+keytool -keystore client.keystore.jks -alias localhost -import -file cert-signed
+```
+
+The definitions of the parameters are the following:
+
+1. `keystore`: the location of the keystore
+2. `ca-cert`: the certificate of the CA
+3. `ca-key`: the private key of the CA
+4. `ca-password`: the passphrase of the CA
+5. `cert-file`: the exported, unsigned certificate of the broker
+6. `cert-signed`: the signed certificate of the broker
+
+### Configure brokers
+
+Brokers enable TLS by providing valid `brokerServicePortTls` and `webServicePortTls`, and also set `tlsEnabledWithKeyStore` to `true` for using KeyStore type configuration. Besides this, KeyStore path, KeyStore password, TrustStore path, and TrustStore password need to be provided. And since brokers create internal client/admin client to communicate with other brokers, users also need to provide config for them, this is similar to how users configure the outside client/admin-client. If [...]
+
+The following TLS configs are needed on the broker side:
+
+```properties
+brokerServicePortTls=6651
+webServicePortTls=8081
+
+tlsRequireTrustedClientCertOnConnect=true
+tlsEnabledWithKeyStore=true
+# key store
+tlsKeyStoreType=JKS
+tlsKeyStore=/var/private/tls/broker.keystore.jks
+tlsKeyStorePassword=brokerpw
+
+# trust store
+tlsTrustStoreType=JKS
+tlsTrustStore=/var/private/tls/broker.truststore.jks
+tlsTrustStorePassword=brokerpw
+
+# internal client/admin-client config
+brokerClientTlsEnabled=true
+brokerClientTlsEnabledWithKeyStore=true
+brokerClientTlsTrustStoreType=JKS
+brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks
+brokerClientTlsTrustStorePassword=clientpw
+brokerClientTlsKeyStoreType=JKS
+brokerClientTlsKeyStore=/var/private/tls/client.keystore.jks
+brokerClientTlsKeyStorePassword=clientpw
+```
+
+:::note
+
+It is important to restrict access to the store files via filesystem permissions.
+
+:::
+
+If you have configured TLS on the broker, to disable non-TLS ports, you can set the values of the following configurations to empty as below.
+
+```conf
+brokerServicePort=
+webServicePort=
+```
+
+In this case, you need to set the following configurations.
+
+```properties
+brokerClientTlsEnabled=true // Set this to true
+brokerClientTlsEnabledWithKeyStore=true // Set this to true
+brokerClientTlsTrustStore= // Set this to your desired value
+brokerClientTlsTrustStorePassword= // Set this to your desired value
+```
+
+Optional settings that may worth considering:
+
+1. tlsClientAuthentication=false: Enable/Disable using TLS for authentication. This config when enabled will authenticate the other end
+ of the communication channel. It should be enabled on both brokers and clients for mutual TLS.
+2. tlsCiphers=[TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256], A cipher suite is a named combination of authentication, encryption, MAC and key exchange
+ algorithm used to negotiate the security settings for a network connection using TLS network protocol. By default,
+ it is null. [OpenSSL Ciphers](https://www.openssl.org/docs/man1.0.2/apps/ciphers.html)
+ [JDK Ciphers](http://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#ciphersuites)
+3. tlsProtocols=[TLSv1.3,TLSv1.2] (list out the TLS protocols that you are going to accept from clients).
+ By default, it is not set.
+
+### Configure Clients
+
+This is similar to [TLS encryption configurations for clients with PEM type](security-tls-transport.md#configure-clients).
+For a minimal configuration, you need to provide the TrustStore information.
+
+For example:
+1. for [Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](reference-cli-tools#pulsar-admin), [`pulsar-perf`](reference-cli-tools#pulsar-perf), and [`pulsar-client`](reference-cli-tools#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
+
+ ```properties
+ webServiceUrl=https://broker.example.com:8443/
+ brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
+ useKeyStoreTls=true
+ tlsTrustStoreType=JKS
+ tlsTrustStorePath=/var/private/tls/client.truststore.jks
+ tlsTrustStorePassword=clientpw
+ tlsKeyStoreType=JKS
+ tlsKeyStorePath=/var/private/tls/client.keystore.jks
+ keyStorePassword=clientpw
+ ```
+
+2. for Java client
+
+ ```java
+ import org.apache.pulsar.client.api.PulsarClient;
+
+ PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar+ssl://broker.example.com:6651/")
+ .useKeyStoreTls(true)
+ .tlsTrustStoreType("JKS")
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .tlsKeyStoreType("JKS")
+ .tlsKeyStorePath("/var/private/tls/client.keystore.jks")
+ .tlsKeyStorePassword("clientpw")
+ .enableTlsHostnameVerification(false) // false by default, in any case
+ .allowTlsInsecureConnection(false) // false by default, in any case
+ .build();
+ ```
+
+3. for Java admin client
+
+ ```java
+ PulsarAdmin amdin = PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443")
+ .tlsTrustStoreType("JKS")
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .tlsKeyStoreType("JKS")
+ .tlsKeyStorePath("/var/private/tls/client.keystore.jks")
+ .tlsKeyStorePassword("clientpw")
+ .enableTlsHostnameVerification(false) // false by default, in any case
+ .allowTlsInsecureConnection(false) // false by default, in any case
+ .build();
+ ```
+
+:::note
+
+Configure `tlsTrustStorePath` when you set `useKeyStoreTls` to `true`.
+
+:::
+
+## Enable TLS Logging
+
+You can enable TLS debug logging at the JVM level by starting the brokers and/or clients with `javax.net.debug` system property. For example:
+
+```shell
+-Djavax.net.debug=all
+```
+
+You can find more details on this in [Oracle documentation](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html) on [debugging SSL/TLS connections](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html).
diff --git a/site2/website-next/sidebars.json b/site2/website-next/sidebars.json
index 0da4f386bea..beb8078a181 100644
--- a/site2/website-next/sidebars.json
+++ b/site2/website-next/sidebars.json
@@ -263,19 +263,33 @@
"label": "Security",
"items": [
"security-overview",
- "security-policy-and-supported-versions",
- "security-tls-transport",
- "security-tls-authentication",
- "security-tls-keystore",
- "security-jwt",
- "security-athenz",
- "security-kerberos",
- "security-oauth2",
- "security-basic-auth",
+ {
+ "type": "category",
+ "label": "Encryption",
+ "link": {
+ "type": "doc",
+ "id": "security-encryption"
+ },
+ "items": [
+ "security-tls-transport",
+ "security-bouncy-castle"
+ ]
+ },
+ {
+ "type": "category",
+ "label": "Authentication",
+ "items": [
+ "security-tls-authentication",
+ "security-jwt",
+ "security-athenz",
+ "security-kerberos",
+ "security-oauth2",
+ "security-basic-auth"
+ ]
+ },
"security-authorization",
- "security-encryption",
"security-extending",
- "security-bouncy-castle"
+ "security-policy-and-supported-versions"
]
},
{
@@ -354,7 +368,11 @@
"items": [
"reference-terminology",
"reference-cli-tools",
- "reference-configuration",
+ {
+ "type": "link",
+ "href": "https://pulsar.apache.org/reference",
+ "label": "Pulsar configuration"
+ },
"reference-metrics",
"reference-rest-api-overview",
{