You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/10/13 08:51:40 UTC
[GitHub] [pulsar] momo-jun opened a new pull request, #18035: [Improve][doc] Improve the authentication enablement workflow across multiple providers
momo-jun opened a new pull request, #18035:
URL: https://github.com/apache/pulsar/pull/18035
### Motivation
Improve the doc set of the Security chapter to provide Pulsar users with an easy-to-use and easy-to-understand information experience.
This PR attempts to align the workflow to enable Athenz/JWT/OAuth2/Keberos authentication with TLS/Basic Auth. Currently, the described workflow lacks clear logic and seems to be very different with each other. Outdated information will be refreshed as well.
Previously related work: https://github.com/apache/pulsar/pull/17615, https://github.com/apache/pulsar/pull/17666 and https://github.com/apache/pulsar/pull/17808.
### Modifications
1. Align the workflow to enable authentication across multiple providers
- [x] Athenz
- [ ] JWT
- [ ] OAuth2
- [ ] Keberos
2. Add code snippets to show the supportability of clients.
3. ...
### Preview
WIP
### Documentation
- [x] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun merged pull request #18035: [improve][doc] Improve the authentication enablement workflow across multiple providers
Posted by GitBox <gi...@apache.org>.
momo-jun merged PR #18035:
URL: https://github.com/apache/pulsar/pull/18035
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #18035: [Improve][doc] Improve the authentication enablement workflow across multiple providers
Posted by GitBox <gi...@apache.org>.
nodece commented on code in PR #18035:
URL: https://github.com/apache/pulsar/pull/18035#discussion_r996597100
##########
site2/docs/security-overview.md:
##########
@@ -21,28 +21,41 @@ Encryption ensures that if an attacker gets access to your data, the attacker ca
**What's next?**
-* To configure end-to-end encryption, see [End-to-end encryption](security-encryption.md) for more details.
-* To configure transport layer encryption, see [TLS encryption](security-tls-transport.md) for more details.
+- To configure end-to-end encryption, see [End-to-end encryption](security-encryption.md) for more details.
+- To configure transport layer encryption, see [TLS encryption](security-tls-transport.md) for more details.
## Authentication
-Authentication is the process of verifying the identity of clients. In Pulsar, the authentication provider is responsible for properly identifying clients and associating the clients with role tokens. If you only enable authentication, an authenticated role token can access all resources in the cluster.
+Authentication is the process of verifying the identity of clients. In Pulsar, the authentication provider is responsible for properly identifying clients and associating them with role tokens. Note that if you only enable authentication, an authenticated role token can access all resources in the cluster.
-Pulsar supports a pluggable authentication mechanism, and Pulsar clients use this mechanism to authenticate with brokers and proxies.
+**How it works in Pulsar**
-Pulsar broker validates the authentication credentials when a connection is established. After the initial connection is authenticated, the "principal" token is stored for authorization though the connection is not re-authenticated. The broker periodically checks the expiration status of every `ServerCnx` object. By default, the `authenticationRefreshCheckSeconds` is set to 60s. When the authentication is expired, the broker re-authenticates the connection. If the re-authentication fails, the broker disconnects the client.
+Pulsar provides a pluggable authentication framework, and Pulsar brokers/proxies use this mechanism to authenticate clients.
-Pulsar broker supports learning whether a particular client supports authentication refreshing. If a client supports authentication refreshing and the credential is expired, the authentication provider calls the `refreshAuthentication` method to initiate the refreshing process. If a client does not support authentication refreshing and the credential is expired, the broker disconnects the client.
+The way how each client passes its authentication data to brokers varies depending on the protocols it uses. Brokers validate the authentication credentials when a connection is established and check whether the authentication data is expired.
+- When using HTTP/HTTPS protocol for cluster management, each client passes the authentication data based on the HTTP/HTTPS header, and brokers check the data upon request.
+- When using [Pulsar protocol](developing-binary-protocol.md) for productions/consumptions, each client passes the authentication data by sending the `CommandConnect` command. Brokers cache the data and periodically check whether the data has expired. If it has expired, brokers send a `CommandAuthChallenge` command to exchange the authentication data with the client. Without the previous authentication challenge returned, brokers disconnect the client.
+
+As soon as the initial connection is authenticated, the "principal" token is stored for authorization though the connection has not been re-authenticated. Brokers periodically check the expiration status of every `ServerCnx` object and learn whether a particular client supports authentication refreshing.
+- If a client supports authentication refreshing and the credential is expired, the authentication provider calls the `refreshAuthentication` method to initiate the refreshing process and re-authenticate the connection. By default, the `authenticationRefreshCheckSeconds` is set to 60s.
+- If a client does not support authentication refreshing and the credential is expired, brokers disconnect the client.
+
+:::note
+
+When using proxies between clients and brokers, you only get proxies authenticated (named **self-authentication**) by default. To forward the authentication data from clients to brokers for client authentication (named **original authentication**), you need to set `forwardAuthorizationCredentials` to `true`.
Review Comment:
```suggestion
When using proxies between clients and brokers, you can only get proxies authentication data (named **self-authentication**) by default on the broker. To forward the authentication data from clients to brokers for client authentication (named **original authentication**), you need to set `forwardAuthorizationCredentials =true` in the `proxy.conf`, and set `authenticateOriginalAuthData=true` in the `broker.conf`, this means the brokers recheck the client authentication.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on pull request #18035: [Improve][doc] Improve the authentication enablement workflow across multiple providers
Posted by GitBox <gi...@apache.org>.
momo-jun commented on PR #18035:
URL: https://github.com/apache/pulsar/pull/18035#issuecomment-1281996941
@nodece @futeng can you pls review these improvements from a tech perspective?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nodece commented on a diff in pull request #18035: [Improve][doc] Improve the authentication enablement workflow across multiple providers
Posted by GitBox <gi...@apache.org>.
nodece commented on code in PR #18035:
URL: https://github.com/apache/pulsar/pull/18035#discussion_r996597100
##########
site2/docs/security-overview.md:
##########
@@ -21,28 +21,41 @@ Encryption ensures that if an attacker gets access to your data, the attacker ca
**What's next?**
-* To configure end-to-end encryption, see [End-to-end encryption](security-encryption.md) for more details.
-* To configure transport layer encryption, see [TLS encryption](security-tls-transport.md) for more details.
+- To configure end-to-end encryption, see [End-to-end encryption](security-encryption.md) for more details.
+- To configure transport layer encryption, see [TLS encryption](security-tls-transport.md) for more details.
## Authentication
-Authentication is the process of verifying the identity of clients. In Pulsar, the authentication provider is responsible for properly identifying clients and associating the clients with role tokens. If you only enable authentication, an authenticated role token can access all resources in the cluster.
+Authentication is the process of verifying the identity of clients. In Pulsar, the authentication provider is responsible for properly identifying clients and associating them with role tokens. Note that if you only enable authentication, an authenticated role token can access all resources in the cluster.
-Pulsar supports a pluggable authentication mechanism, and Pulsar clients use this mechanism to authenticate with brokers and proxies.
+**How it works in Pulsar**
-Pulsar broker validates the authentication credentials when a connection is established. After the initial connection is authenticated, the "principal" token is stored for authorization though the connection is not re-authenticated. The broker periodically checks the expiration status of every `ServerCnx` object. By default, the `authenticationRefreshCheckSeconds` is set to 60s. When the authentication is expired, the broker re-authenticates the connection. If the re-authentication fails, the broker disconnects the client.
+Pulsar provides a pluggable authentication framework, and Pulsar brokers/proxies use this mechanism to authenticate clients.
-Pulsar broker supports learning whether a particular client supports authentication refreshing. If a client supports authentication refreshing and the credential is expired, the authentication provider calls the `refreshAuthentication` method to initiate the refreshing process. If a client does not support authentication refreshing and the credential is expired, the broker disconnects the client.
+The way how each client passes its authentication data to brokers varies depending on the protocols it uses. Brokers validate the authentication credentials when a connection is established and check whether the authentication data is expired.
+- When using HTTP/HTTPS protocol for cluster management, each client passes the authentication data based on the HTTP/HTTPS header, and brokers check the data upon request.
+- When using [Pulsar protocol](developing-binary-protocol.md) for productions/consumptions, each client passes the authentication data by sending the `CommandConnect` command. Brokers cache the data and periodically check whether the data has expired. If it has expired, brokers send a `CommandAuthChallenge` command to exchange the authentication data with the client. Without the previous authentication challenge returned, brokers disconnect the client.
+
+As soon as the initial connection is authenticated, the "principal" token is stored for authorization though the connection has not been re-authenticated. Brokers periodically check the expiration status of every `ServerCnx` object and learn whether a particular client supports authentication refreshing.
+- If a client supports authentication refreshing and the credential is expired, the authentication provider calls the `refreshAuthentication` method to initiate the refreshing process and re-authenticate the connection. By default, the `authenticationRefreshCheckSeconds` is set to 60s.
+- If a client does not support authentication refreshing and the credential is expired, brokers disconnect the client.
+
+:::note
+
+When using proxies between clients and brokers, you only get proxies authenticated (named **self-authentication**) by default. To forward the authentication data from clients to brokers for client authentication (named **original authentication**), you need to set `forwardAuthorizationCredentials` to `true`.
Review Comment:
```suggestion
When using proxies between clients and brokers, you can only get proxies authentication data (named **self-authentication**) by default on the broker. To forward the authentication data from clients to brokers for client authentication (named **original authentication**), you need to set `forwardAuthorizationCredentials` to `true` in the `proxy.conf`, this means the brokers recheck the client authentication.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on a diff in pull request #18035: [improve][doc] Improve the authentication enablement workflow across multiple providers
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on code in PR #18035:
URL: https://github.com/apache/pulsar/pull/18035#discussion_r999055839
##########
site2/docs/security-athenz.md:
##########
@@ -65,16 +61,46 @@ brokerClientAuthenticationParameters={"tenantDomain":"shopping","tenantService":
> A full listing of parameters is available in the `conf/broker.conf` file, you can also find the default
> values for those parameters in [Broker Configuration](reference-configuration.md#broker).
+## Enable Athenz authentication on proxies
+
+Configure the required parameters in the `conf/proxy.conf` file in your Pulsar installation.
+
+```properties
+# Add the Athenz auth provider
+authenticationEnabled=true
+authorizationEnabled=true
+authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderAthenz
+athenzDomainNames=pulsar
+
+brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationAthenz
+brokerClientAuthenticationParameters={"tenantDomain":"shopping","tenantService":"some_app","providerDomain":"pulsar","privateKey":"file:///path/to/private.pem","keyId":"v1"}
+```
+
## Configure Athenz authentication in Pulsar clients
-To use Athenz as an authentication provider, you need to [use TLS](#tls-authentication.md) and provide values for four parameters in a hash:
+To use Athenz as an authentication provider, you need to provide values for four parameters in a hash:
* `tenantDomain`
* `tenantService`
* `providerDomain`
* `privateKey`
+:::tip
+
+The `privateKey` parameter supports the following three pattern formats:
+* `file:///path/to/file`
+* `file:/path/to/file`
+* `data:application/x-pem-file;base64,<base64-encoded value>`
+
+:::
+
You can also set an optional `keyId`. The following is an example.
+````mdx-code-block
+<Tabs groupId="lang-choice"
+ defaultValue="Java"
+ values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"C++","value":"C++"},{"label":"Node.js","value":"Node.js"},{"label":"Go","value":"Go"}]}>
+<TabItem value="Java">
+
```java
Map<String, String> authParams = new HashMap();
authParams.put("tenantDomain", "shopping"); // Tenant domain name
Review Comment:
Let's also add the ztsUrl setting for the java client to make it consistent with other client.
```
authParams.put("ztsUrl", "http://localhost:9998");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #18035: [improve][doc] Improve the authentication enablement workflow across multiple providers
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #18035:
URL: https://github.com/apache/pulsar/pull/18035#discussion_r1000067645
##########
site2/docs/security-athenz.md:
##########
@@ -65,16 +61,46 @@ brokerClientAuthenticationParameters={"tenantDomain":"shopping","tenantService":
> A full listing of parameters is available in the `conf/broker.conf` file, you can also find the default
> values for those parameters in [Broker Configuration](reference-configuration.md#broker).
+## Enable Athenz authentication on proxies
+
+Configure the required parameters in the `conf/proxy.conf` file in your Pulsar installation.
+
+```properties
+# Add the Athenz auth provider
+authenticationEnabled=true
+authorizationEnabled=true
+authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderAthenz
+athenzDomainNames=pulsar
+
+brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationAthenz
+brokerClientAuthenticationParameters={"tenantDomain":"shopping","tenantService":"some_app","providerDomain":"pulsar","privateKey":"file:///path/to/private.pem","keyId":"v1"}
+```
+
## Configure Athenz authentication in Pulsar clients
-To use Athenz as an authentication provider, you need to [use TLS](#tls-authentication.md) and provide values for four parameters in a hash:
+To use Athenz as an authentication provider, you need to provide values for four parameters in a hash:
* `tenantDomain`
* `tenantService`
* `providerDomain`
* `privateKey`
+:::tip
+
+The `privateKey` parameter supports the following three pattern formats:
+* `file:///path/to/file`
+* `file:/path/to/file`
+* `data:application/x-pem-file;base64,<base64-encoded value>`
+
+:::
+
You can also set an optional `keyId`. The following is an example.
+````mdx-code-block
+<Tabs groupId="lang-choice"
+ defaultValue="Java"
+ values={[{"label":"Java","value":"Java"},{"label":"Python","value":"Python"},{"label":"C++","value":"C++"},{"label":"Node.js","value":"Node.js"},{"label":"Go","value":"Go"}]}>
+<TabItem value="Java">
+
```java
Map<String, String> authParams = new HashMap();
authParams.put("tenantDomain", "shopping"); // Tenant domain name
Review Comment:
Done. Thanks for your comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org