You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by zh...@apache.org on 2020/05/12 13:45:02 UTC
[pulsar] 11/17: add keystore doc (#6922)
This is an automated email from the ASF dual-hosted git repository.
zhaijia pushed a commit to branch branch-2.5
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 9d9af98161ff421a82e8b8dad852890912ec2c58
Author: Jia Zhai <zh...@apache.org>
AuthorDate: Tue May 12 13:58:26 2020 +0800
add keystore doc (#6922)
related with #6853
add keystore tls config doc
(cherry picked from commit fd6f772d934725e4a38f019da90016e30166ada7)
---
conf/client.conf | 2 +-
site2/docs/security-tls-keystore.md | 286 ++++++++++++++++++++++++++++++++++++
site2/website/sidebars.json | 1 +
3 files changed, 288 insertions(+), 1 deletion(-)
diff --git a/conf/client.conf b/conf/client.conf
index 597478e..8e93e1f 100644
--- a/conf/client.conf
+++ b/conf/client.conf
@@ -57,7 +57,7 @@ tlsEnableHostnameVerification=false
tlsTrustCertsFilePath=
# Enable TLS with KeyStore type configuration in broker.
-useKeyStoreTls=false;
+useKeyStoreTls=false
# TLS KeyStore type configuration: JKS, PKCS12
tlsTrustStoreType=JKS
diff --git a/site2/docs/security-tls-keystore.md b/site2/docs/security-tls-keystore.md
new file mode 100644
index 0000000..befe23c
--- /dev/null
+++ b/site2/docs/security-tls-keystore.md
@@ -0,0 +1,286 @@
+---
+id: security-tls-keystore
+title: Using TLS with KeyStore configure
+sidebar_label: Using TLS with KeyStore configure
+---
+
+## Overview
+
+Apache Pulsar supports [TLS encryption](security-tls-transport.md) and [TLS authentication](security-tls-authentication.md) between clients and Apache Pulsar service.
+By default it uses PEM format file configuration. This page tries to describe use [KeyStore](https://en.wikipedia.org/wiki/Java_KeyStore) type configure for TLS.
+
+
+## TLS encryption with KeyStore configure
+
+### Generate TLS key and certificate
+
+The first step of deploying TLS is to generate the key and the certificate for each machine in the cluster.
+You can use Java’s `keytool` utility to accomplish this task. We will generate the key into a temporary keystore
+initially for broker, so that we can export and sign it later with CA.
+
+```shell
+keytool -keystore broker.keystore.jks -alias localhost -validity {validity} -genkey
+```
+
+You need to specify two parameters in the above command:
+
+1. `keystore`: the keystore file that stores the certificate. The *keystore* file contains the private key of
+ the certificate; hence, it needs to be kept safely.
+2. `validity`: the valid time of the certificate in days.
+
+> Ensure that common name (CN) matches exactly with the fully qualified domain name (FQDN) of the server.
+The client compares the CN with the DNS domain name to ensure that it is indeed connecting to the desired server, not a malicious one.
+
+### Creating your own CA
+
+After the first step, each broker in the cluster has a public-private key pair, and a certificate to identify the machine.
+The certificate, however, is unsigned, which means that an attacker can create such a certificate to pretend to be any machine.
+
+Therefore, it is important to prevent forged certificates by signing them for each machine in the cluster.
+A `certificate authority (CA)` is responsible for signing certificates. CA works likes a government that issues passports —
+the government stamps (signs) each passport so that the passport becomes difficult to forge. Other governments verify the stamps
+to ensure the passport is authentic. Similarly, the CA signs the certificates, and the cryptography guarantees that a signed
+certificate is computationally difficult to forge. Thus, as long as the CA is a genuine and trusted authority, the clients have
+high assurance that they are connecting to the authentic machines.
+
+```shell
+openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
+```
+
+The generated CA is simply a *public-private* key pair and certificate, and it is intended to sign other certificates.
+
+The next step is to add the generated CA to the clients' truststore so that the clients can trust this CA:
+
+```shell
+keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
+```
+
+NOTE: If you configure the brokers to require client authentication by setting `tlsRequireTrustedClientCertOnConnect` to `true` on the
+broker configuration, then you must also provide a truststore for the brokers and it should have all the CA certificates that clients keys were signed by.
+
+```shell
+keytool -keystore broker.truststore.jks -alias CARoot -import -file ca-cert
+```
+
+In contrast to the keystore, which stores each machine’s own identity, the truststore of a client stores all the certificates
+that the client should trust. Importing a certificate into one’s truststore also means trusting all certificates that are signed
+by that certificate. As the analogy above, trusting the government (CA) also means trusting all passports (certificates) that
+it has issued. This attribute is called the chain of trust, and it is particularly useful when deploying TLS on a large BookKeeper cluster.
+You can sign all certificates in the cluster with a single CA, and have all machines share the same truststore that trusts the CA.
+That way all machines can authenticate all other machines.
+
+
+### Signing the certificate
+
+The next step is to sign all certificates in the keystore with the CA we generated. First, you need to export the certificate from the keystore:
+
+```shell
+keytool -keystore broker.keystore.jks -alias localhost -certreq -file cert-file
+```
+
+Then sign it with the CA:
+
+```shell
+openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days {validity} -CAcreateserial -passin pass:{ca-password}
+```
+
+Finally, you need to import both the certificate of the CA and the signed certificate into the keystore:
+
+```shell
+keytool -keystore broker.keystore.jks -alias CARoot -import -file ca-cert
+keytool -keystore broker.keystore.jks -alias localhost -import -file cert-signed
+```
+
+The definitions of the parameters are the following:
+
+1. `keystore`: the location of the keystore
+2. `ca-cert`: the certificate of the CA
+3. `ca-key`: the private key of the CA
+4. `ca-password`: the passphrase of the CA
+5. `cert-file`: the exported, unsigned certificate of the broker
+6. `cert-signed`: the signed certificate of the broker
+
+### Configuring brokers
+
+Brokers enable TLS by provide valid `brokerServicePortTls` and `webServicePortTls`, and also need set `tlsEnabledWithKeyStore` to `true` for using KeyStore type configuration.
+Besides this, KeyStore path, KeyStore password, TrustStore path, and TrustStore password need to provided.
+And since broker will create internal client/admin client to communicate with other brokers, user also need to provide config for them, this is similar to how user config the outside client/admin-client.
+If `tlsRequireTrustedClientCertOnConnect` is `true`, broker will reject the Connection if the Client Certificate is not trusted.
+
+The following TLS configs are needed on the broker side:
+
+```properties
+tlsEnabledWithKeyStore=true
+# key store
+tlsKeyStoreType=JKS
+tlsKeyStore=/var/private/tls/broker.keystore.jks
+tlsKeyStorePassword=brokerpw
+
+# trust store
+tlsTrustStoreType=JKS
+tlsTrustStore=/var/private/tls/broker.truststore.jks
+tlsTrustStorePassword=brokerpw
+
+# interal client/admin-client config
+brokerClientTlsEnabled=true
+brokerClientTlsEnabledWithKeyStore=true
+brokerClientTlsTrustStoreType=JKS
+brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks
+brokerClientTlsTrustStorePassword=clientpw
+```
+
+NOTE: it is important to restrict access to the store files via filesystem permissions.
+
+Optional settings that may worth consider:
+
+1. tlsClientAuthentication=false: Enable/Disable using TLS for authentication. This config when enabled will authenticate the other end
+ of the communication channel. It should be enabled on both brokers and clients for mutual TLS.
+2. tlsCiphers=[TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256], A cipher suite is a named combination of authentication, encryption, MAC and key exchange
+ algorithm used to negotiate the security settings for a network connection using TLS network protocol. By default,
+ it is null. [OpenSSL Ciphers](https://www.openssl.org/docs/man1.0.2/apps/ciphers.html)
+ [JDK Ciphers](http://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#ciphersuites)
+3. tlsProtocols=[TLSv1.2,TLSv1.1,TLSv1] (list out the TLS protocols that you are going to accept from clients).
+ By default, it is not set.
+
+### Configuring Clients
+
+This is similar to [TLS encryption configuing for client with PEM type](security-tls-transport.md#Client configuration).
+For a a minimal configuration, user need to provide the TrustStore information.
+
+e.g.
+1. for [Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](reference-cli-tools#pulsar-admin), [`pulsar-perf`](reference-cli-tools#pulsar-perf), and [`pulsar-client`](reference-cli-tools#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
+
+ ```properties
+ webServiceUrl=https://broker.example.com:8443/
+ brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
+ useKeyStoreTls=true
+ tlsTrustStoreType=JKS
+ tlsTrustStorePath=/var/private/tls/client.truststore.jks
+ tlsTrustStorePassword=clientpw
+ ```
+
+1. for java client
+ ```java
+ import org.apache.pulsar.client.api.PulsarClient;
+
+ PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar+ssl://broker.example.com:6651/")
+ .enableTls(true)
+ .useKeyStoreTls(true)
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .allowTlsInsecureConnection(false)
+ .build();
+ ```
+
+1. for java admin client
+```java
+ PulsarAdmin amdin = PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443")
+ .useKeyStoreTls(true)
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .allowTlsInsecureConnection(false)
+ .build();
+```
+
+## TLS authentication with KeyStore configure
+
+This similar to [TLS authentication with PEM type](security-tls-authentication.md)
+
+### broker authentication config
+
+`broker.conf`
+
+```properties
+# Configuration to enable authentication
+authenticationEnabled=true
+authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderTls
+
+# this should be the CN for one of client keystore.
+superUserRoles=admin
+
+# Enable KeyStore type
+tlsEnabledWithKeyStore=true
+requireTrustedClientCertOnConnect=true
+
+# key store
+tlsKeyStoreType=JKS
+tlsKeyStore=/var/private/tls/broker.keystore.jks
+tlsKeyStorePassword=brokerpw
+
+# trust store
+tlsTrustStoreType=JKS
+tlsTrustStore=/var/private/tls/broker.truststore.jks
+tlsTrustStorePassword=brokerpw
+
+# interal client/admin-client config
+brokerClientTlsEnabled=true
+brokerClientTlsEnabledWithKeyStore=true
+brokerClientTlsTrustStoreType=JKS
+brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks
+brokerClientTlsTrustStorePassword=clientpw
+# internal auth config
+brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
+brokerClientAuthenticationParameters=keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw
+# currently websocket not support keystore type
+webSocketServiceEnabled=false
+```
+
+### client authentication configuring
+
+Besides the TLS encryption configuring. The main work is configuring the KeyStore, which contains a valid CN as client role, for client.
+
+e.g.
+1. for [Command-line tools](reference-cli-tools.md) like [`pulsar-admin`](reference-cli-tools#pulsar-admin), [`pulsar-perf`](reference-cli-tools#pulsar-perf), and [`pulsar-client`](reference-cli-tools#pulsar-client) use the `conf/client.conf` config file in a Pulsar installation.
+
+ ```properties
+ webServiceUrl=https://broker.example.com:8443/
+ brokerServiceUrl=pulsar+ssl://broker.example.com:6651/
+ useKeyStoreTls=true
+ tlsTrustStoreType=JKS
+ tlsTrustStorePath=/var/private/tls/client.truststore.jks
+ tlsTrustStorePassword=clientpw
+ authPlugin=org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls
+ authParams=keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw
+ ```
+
+1. for java client
+ ```java
+ import org.apache.pulsar.client.api.PulsarClient;
+
+ PulsarClient client = PulsarClient.builder()
+ .serviceUrl("pulsar+ssl://broker.example.com:6651/")
+ .enableTls(true)
+ .useKeyStoreTls(true)
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .allowTlsInsecureConnection(false)
+ .authentication(
+ "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
+ "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
+ .build();
+ ```
+
+1. for java admin client
+ ```java
+ PulsarAdmin amdin = PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443")
+ .useKeyStoreTls(true)
+ .tlsTrustStorePath("/var/private/tls/client.truststore.jks")
+ .tlsTrustStorePassword("clientpw")
+ .allowTlsInsecureConnection(false)
+ .authentication(
+ "org.apache.pulsar.client.impl.auth.AuthenticationKeyStoreTls",
+ "keyStoreType:JKS,keyStorePath:/var/private/tls/client.keystore.jks,keyStorePassword:clientpw")
+ .build();
+ ```
+
+## Enabling TLS Logging
+
+You can enable TLS debug logging at the JVM level by starting the brokers and/or clients with `javax.net.debug` system property. For example:
+
+```shell
+-Djavax.net.debug=all
+```
+
+You can find more details on this in [Oracle documentation](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html) on
+[debugging SSL/TLS connections](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html).
diff --git a/site2/website/sidebars.json b/site2/website/sidebars.json
index b3b6473..8fc9d49 100644
--- a/site2/website/sidebars.json
+++ b/site2/website/sidebars.json
@@ -75,6 +75,7 @@
"security-overview",
"security-tls-transport",
"security-tls-authentication",
+ "security-tls-keystore",
"security-jwt",
"security-athenz",
"security-kerberos",