You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by zi...@apache.org on 2022/08/09 02:06:07 UTC
[pulsar] branch master updated: [improve][doc] Improve TLS transport encryption and authentication (#16924)
This is an automated email from the ASF dual-hosted git repository.
zixuan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 21634edd30b [improve][doc] Improve TLS transport encryption and authentication (#16924)
21634edd30b is described below
commit 21634edd30bbe33d231ec73e8045d7666a8ad366
Author: Zixuan Liu <no...@gmail.com>
AuthorDate: Tue Aug 9 10:06:00 2022 +0800
[improve][doc] Improve TLS transport encryption and authentication (#16924)
Signed-off-by: Zixuan Liu <no...@gmail.com>
---
site2/docs/security-tls-authentication.md | 15 +-------
site2/docs/security-tls-keystore.md | 42 ++++++++++++++-------
site2/docs/security-tls-transport.md | 62 ++++++++++++++++++++++++++++---
3 files changed, 87 insertions(+), 32 deletions(-)
diff --git a/site2/docs/security-tls-authentication.md b/site2/docs/security-tls-authentication.md
index 295c1620dba..d82122dd7f1 100644
--- a/site2/docs/security-tls-authentication.md
+++ b/site2/docs/security-tls-authentication.md
@@ -8,8 +8,6 @@ sidebar_label: "Authentication using TLS"
TLS authentication is an extension of [TLS transport encryption](security-tls-transport.md). Not only servers have keys and certs that the client uses to verify the identity of servers, clients also have keys and certs that the server uses to verify the identity of clients. You must have TLS transport encryption configured on your cluster before you can use TLS authentication. This guide assumes you already have TLS transport encryption configured.
-`Bouncy Castle Provider` provides TLS related cipher suites and algorithms in Pulsar. If you need [FIPS](https://www.bouncycastle.org/fips_faq.html) version of `Bouncy Castle Provider`, please reference [Bouncy Castle page](security-bouncy-castle.md).
-
### Create client certificates
Client certificates are generated using the certificate authority. Server certificates are also generated with the same certificate authority.
@@ -87,11 +85,6 @@ To configure brokers to authenticate clients, add the following parameters to `b
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
-# operations and publish/consume from all topics
-superUserRoles=admin
-
-# Authentication settings of the broker itself. Used when the broker connects to other brokers, either in same or other clusters
-brokerClientTlsEnabled=true
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
brokerClientAuthenticationParameters={"tlsCertFile":"/path/my-ca/admin.cert.pem","tlsKeyFile":"/path/my-ca/admin.key-pk8.pem"}
brokerClientTrustCertsFilePath=/path/my-ca/certs/ca.cert.pem
@@ -124,15 +117,10 @@ When you use TLS authentication, client connects via TLS transport. You need to
[Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](/tools/pulsar-admin/), [`pulsar-perf`](reference-cli-tools.md#pulsar-perf), and [`pulsar-client`](reference-cli-tools.md#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
-You need to add the following parameters to that file to use TLS authentication with the CLI tools of Pulsar:
+To use TLS authentication with the CLI tools of Pulsar, you need to add the following parameters to the `conf/client.conf` file, alongside [the configuration to enable TLS transport](security-tls-transport.md#cli-tools):
```properties
-webServiceUrl=https://broker.example.com:8443/
-brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
-useTls=true
-tlsAllowInsecureConnection=false
-tlsTrustCertsFilePath=/path/to/ca.cert.pem
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationTls
authParams=tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem
@@ -146,7 +134,6 @@ import org.apache.pulsar.client.api.PulsarClient;
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar+ssl://broker.example.com:6651/")
- .enableTls(true)
.tlsTrustCertsFilePath("/path/to/ca.cert.pem")
.authentication("org.apache.pulsar.client.impl.auth.AuthenticationTls",
"tlsCertFile:/path/to/my-role.cert.pem,tlsKeyFile:/path/to/my-role.key-pk8.pem")
diff --git a/site2/docs/security-tls-keystore.md b/site2/docs/security-tls-keystore.md
index d655290a2cb..775415aab4b 100644
--- a/site2/docs/security-tls-keystore.md
+++ b/site2/docs/security-tls-keystore.md
@@ -85,6 +85,7 @@ The next step is to sign all certificates in the keystore with the CA we generat
```shell
keytool -keystore broker.keystore.jks -alias localhost -certreq -file cert-file
+keytool -keystore client.keystore.jks -alias localhost -certreq -file cert-file
```
@@ -103,6 +104,8 @@ Finally, you need to import both the certificate of the CA and the signed certif
keytool -keystore broker.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore broker.keystore.jks -alias localhost -import -file cert-signed
+keytool -keystore client.keystore.jks -alias CARoot -import -file ca-cert
+keytool -keystore client.keystore.jks -alias localhost -import -file cert-signed
```
The definitions of the parameters are the following:
@@ -124,7 +127,10 @@ If `tlsRequireTrustedClientCertOnConnect` is `true`, broker will reject the Conn
The following TLS configs are needed on the broker side:
```properties
+brokerServicePortTls=6651
+webServicePortTls=8081
+tlsRequireTrustedClientCertOnConnect=true
tlsEnabledWithKeyStore=true
# key store
tlsKeyStoreType=JKS
@@ -142,6 +148,9 @@ brokerClientTlsEnabledWithKeyStore=true
brokerClientTlsTrustStoreType=JKS
brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks
brokerClientTlsTrustStorePassword=clientpw
+brokerClientTlsKeyStoreType=JKS
+brokerClientTlsKeyStore=/var/private/tls/client.keystore.jks
+brokerClientTlsKeyStorePassword=clientpw
```
@@ -193,7 +202,10 @@ For example:
tlsTrustStoreType=JKS
tlsTrustStorePath=/var/private/tls/client.truststore.jks
tlsTrustStorePassword=clientpw
-
+ tlsKeyStoreType=JKS
+ tlsKeyStorePath=/var/private/tls/client.keystore.jks
+ keyStorePassword=clientpw
+
```
1. for java client
@@ -204,11 +216,15 @@ For example:
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar+ssl://broker.example.com:6651/")
- .enableTls(true)
.useKeyStoreTls(true)
+ .tlsTrustStoreType("JKS")
.tlsTrustStorePath("/var/private/tls/client.truststore.jks")
.tlsTrustStorePassword("clientpw")
- .allowTlsInsecureConnection(false)
+ .tlsKeyStoreType("JKS")
+ .tlsKeyStorePath("/var/private/tls/client.keystore.jks")
+ .tlsKeyStorePassword("clientpw")
+ .enableTlsHostnameVerification(false) // false by default, in any case
+ .allowTlsInsecureConnection(false) // false by default, in any case
.build();
```
@@ -218,10 +234,14 @@ For example:
```java
PulsarAdmin amdin = PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443")
- .useKeyStoreTls(true)
+ .tlsTrustStoreType("JKS")
.tlsTrustStorePath("/var/private/tls/client.truststore.jks")
.tlsTrustStorePassword("clientpw")
- .allowTlsInsecureConnection(false)
+ .tlsKeyStoreType("JKS")
+ .tlsKeyStorePath("/var/private/tls/client.keystore.jks")
+ .tlsKeyStorePassword("clientpw")
+ .enableTlsHostnameVerification(false) // false by default, in any case
+ .allowTlsInsecureConnection(false) // false by default, in any case
.build();
```
@@ -242,12 +262,9 @@ This similar to [TLS authentication with PEM type](security-tls-authentication.m
authenticationEnabled=true
authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
-# this should be the CN for one of client keystore.
-superUserRoles=admin
-
# Enable KeyStore type
tlsEnabledWithKeyStore=true
-requireTrustedClientCertOnConnect=true
+tlsRequireTrustedClientCertOnConnect=true
# key store
tlsKeyStoreType=JKS
@@ -268,8 +285,6 @@ brokerClientTlsTrustStorePassword=clientpw
# internal auth config
brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
brokerClientAuthenticationParameters={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"}
-# currently websocket not support keystore type
-webSocketServiceEnabled=false
```
@@ -289,7 +304,7 @@ For example:
tlsTrustStorePath=/var/private/tls/client.truststore.jks
tlsTrustStorePassword=clientpw
authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
- authParams={"keyStoreType":"JKS","keyStorePath":"/path/to/keystorefile","keyStorePassword":"keystorepw"}
+ authParams={"keyStoreType":"JKS","keyStorePath":"/var/private/tls/client.keystore.jks","keyStorePassword":"clientpw"}
```
@@ -301,11 +316,11 @@ For example:
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar+ssl://broker.example.com:6651/")
- .enableTls(true)
.useKeyStoreTls(true)
.tlsTrustStorePath("/var/private/tls/client.truststore.jks")
.tlsTrustStorePassword("clientpw")
.allowTlsInsecureConnection(false)
+ .enableTlsHostnameVerification(false)
.authentication(
"org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
"keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
@@ -322,6 +337,7 @@ For example:
.tlsTrustStorePath("/var/private/tls/client.truststore.jks")
.tlsTrustStorePassword("clientpw")
.allowTlsInsecureConnection(false)
+ .enableTlsHostnameVerification(false)
.authentication(
"org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
"keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
diff --git a/site2/docs/security-tls-transport.md b/site2/docs/security-tls-transport.md
index f25898c47e0..881563a31e5 100644
--- a/site2/docs/security-tls-transport.md
+++ b/site2/docs/security-tls-transport.md
@@ -28,7 +28,9 @@ For TLS transport encryption, the clients can use the **trust cert** to verify t
For TLS authentication, the server uses the **trust cert** to verify that the client has a key pair that the certificate authority signed. The common name of the **client cert** is then used as the client's role token (see [Overview](security-overview.md)).
-`Bouncy Castle Provider` provides cipher suites and algorithms in Pulsar. If you need [FIPS](https://www.bouncycastle.org/fips_faq.html) version of `Bouncy Castle Provider`, please reference [Bouncy Castle page](security-bouncy-castle.md).
+Pulsar uses [netty-tcnative](https://github.com/netty/netty-tcnative) and [Conscrypt](https://github.com/google/conscrypt) as security providers. There are two certificate formats:
+* Java KeyStore(JKS): Pulsar uses Conscrypt by default for both broker service and Web service.
+* CAcerts: Pulsar uses netty-tcnative by default, which includes two implementations, OpenSSL (default) and JDK. When OpenSSL is unavailable, JDK is used.
## Create TLS certificates
@@ -144,6 +146,46 @@ openssl ca -config openssl.cnf -extensions server_cert \
At this point, you have a cert, `broker.cert.pem`, and a key, `broker.key-pk8.pem`, which you can use along with `ca.cert.pem` to configure TLS transport encryption for your broker and proxy nodes.
+### Client certificate
+
+1. Enter the command below to generate the key.
+
+```bash
+
+openssl genrsa -out client.key.pem 2048
+
+```
+
+The client expects the key to be in [PKCS 8](https://en.wikipedia.org/wiki/PKCS_8) format, so enter the following command to convert it.
+
+```bash
+
+openssl pkcs8 -topk8 -inform PEM -outform PEM \
+ -in client.key.pem -out client.key-pk8.pem -nocrypt
+
+```
+
+2. Enter the following command to generate the certificate request.
+
+```bash
+
+openssl req -config openssl.cnf \
+ -key client.key.pem -new -sha256 -out client.csr.pem
+
+```
+
+3. Sign it with the certificate authority by entering the command below.
+
+```bash
+
+openssl ca -config openssl.cnf -extensions client_cert \
+ -days 1000 -notext -md sha256 \
+ -in client.csr.pem -out client.cert.pem
+
+```
+
+At this point, you have a cert `client.cert.pem` and a key `client.key-pk8.pem`, which you can use along with `ca.cert.pem` to configure TLS encryption for your client.
+
## Configure broker
To configure a Pulsar [broker](reference-terminology.md#broker) to use TLS transport encryption, you need to make some changes to `broker.conf`, which locates in the `conf` directory of your [Pulsar installation](getting-started-standalone.md).
@@ -159,6 +201,10 @@ tlsCertificateFilePath=/path/to/broker.cert.pem
tlsKeyFilePath=/path/to/broker.key-pk8.pem
tlsTrustCertsFilePath=/path/to/ca.cert.pem
+brokerClientTlsEnabled=true
+brokerClientTrustCertsFilePath=/path/to/ca.cert.pem
+brokerClientCertificateFilePath=/path/to/client.cert.pem
+brokerClientKeyFilePath=/path/to/client.key-pk8.pem
```
> You can find a full list of parameters available in the `conf/broker.conf` file,
@@ -189,8 +235,11 @@ Proxies need to configure TLS in two directions, for clients connecting to the p
```properties
+servicePortTls=6651
+webServicePortTls=8081
+
# For clients connecting to the proxy
-tlsEnabledInProxy=true
+tlsRequireTrustedClientCertOnConnect=true
tlsCertificateFilePath=/path/to/broker.cert.pem
tlsKeyFilePath=/path/to/broker.key-pk8.pem
tlsTrustCertsFilePath=/path/to/ca.cert.pem
@@ -198,7 +247,8 @@ tlsTrustCertsFilePath=/path/to/ca.cert.pem
# For the proxy to connect to brokers
tlsEnabledWithBroker=true
brokerClientTrustCertsFilePath=/path/to/ca.cert.pem
-
+brokerClientCertificateFilePath=/path/to/client.cert.pem
+brokerClientKeyFilePath=/path/to/client.key-pk8.pem
```
## Client configuration
@@ -227,9 +277,10 @@ You need to add the following parameters to that file to use TLS transport with
webServiceUrl=https://broker.example.com:8443/
brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
-useTls=true
tlsAllowInsecureConnection=false
tlsTrustCertsFilePath=/path/to/ca.cert.pem
+tlsKeyFilePath=/path/to/client.key-pk8.pem
+tlsCertFile=/path/to/client-cert.pem
tlsEnableHostnameVerification=false
```
@@ -242,7 +293,8 @@ import org.apache.pulsar.client.api.PulsarClient;
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar+ssl://broker.example.com:6651/")
- .enableTls(true)
+ .tlsKeyFilePath("/path/to/client.key-pk8.pem")
+ .tlsCertificateFilePath("/path/to/client.cert.pem")
.tlsTrustCertsFilePath("/path/to/ca.cert.pem")
.enableTlsHostnameVerification(false) // false by default, in any case
.allowTlsInsecureConnection(false) // false by default, in any case