You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "vtlim (via GitHub)" <gi...@apache.org> on 2023/04/10 23:28:40 UTC
[GitHub] [druid] vtlim commented on a diff in pull request #14055: Add basic security functions to druidapi (follow up to #14009)
vtlim commented on code in PR #14055:
URL: https://github.com/apache/druid/pull/14055#discussion_r1162082085
##########
docs/operations/security-overview.md:
##########
@@ -157,10 +148,12 @@ An example configuration:
# Default password for internal 'druid_system' user, should be changed for production.
druid.auth.authenticator.MyBasicMetadataAuthenticator.initialInternalClientPassword=password2
- # Uses the metadata store for storing users, you can use authentication API to create new users and grant permissions
+ # Uses the metadata store for storing users.
+ # You can use authentication API to create new users and grant permissions
Review Comment:
```suggestion
# You can use the authentication API to create new users and grant permissions
```
##########
docs/operations/security-overview.md:
##########
@@ -176,70 +169,83 @@ An example configuration:
druid.auth.authorizer.MyBasicMetadataAuthorizer.type=basic
```
-3. Restart the cluster.
+3. Restart the cluster.
-See [Authentication and Authorization](../design/auth.md) for more information about the Authenticator, Escalator, and Authorizer concepts. See [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about the extension used in the examples above, and [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+See the following topics for more information:
+* [Authentication and Authorization](../design/auth.md) for more information about the Authenticator,
+Escalator, and Authorizer.
+* [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about
+the extension used in the examples above.
+* [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+* [User authentication and authorization](security-user-auth.md) for details about permissions.
+* [SQL permissions](security-user-auth.md#sql-permissions) for permissions on SQL system tables.
+* [The `druidapi` Python library](../tutorials/tutorial-jupyter-index.md),
+ provided as part of the Druid tutorials, for functions you can
+ use in a Jupyter notebook to learn how security works, and to set up users and roles for testing.
## Enable authorizers
-After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
+After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
The following diagram depicts the authorization model, and the relationship between users, roles, permissions, and resources.
-
-
+
-The following steps walk through a sample setup procedure:
+
+The following steps walk through a sample setup procedure:
> The default Coordinator API port is 8081 for non-TLS connections and 8281 for secured connections.
-1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`, replacing USERNAME with the *new* username you are trying to create. For example:
- ```
+1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`,
+ replacing USERNAME with the *new* username you are trying to create. For example:
+ ```bash
curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/myname
- ```
- > If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates, you may choose to include the `insecure` curl option to forgo certificate checking for the curl command.
+ ```
+ > If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates,
+ you may choose to include the `insecure` curl option to forgo certificate checking for the curl command.
+
2. Add a credential for the user by issuing a POST to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>/credentials`. For example:
Review Comment:
```suggestion
2. Add a credential for the user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>/credentials`. For example:
```
##########
docs/operations/security-overview.md:
##########
@@ -176,70 +169,83 @@ An example configuration:
druid.auth.authorizer.MyBasicMetadataAuthorizer.type=basic
```
-3. Restart the cluster.
+3. Restart the cluster.
-See [Authentication and Authorization](../design/auth.md) for more information about the Authenticator, Escalator, and Authorizer concepts. See [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about the extension used in the examples above, and [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+See the following topics for more information:
+* [Authentication and Authorization](../design/auth.md) for more information about the Authenticator,
+Escalator, and Authorizer.
+* [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about
+the extension used in the examples above.
+* [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+* [User authentication and authorization](security-user-auth.md) for details about permissions.
+* [SQL permissions](security-user-auth.md#sql-permissions) for permissions on SQL system tables.
+* [The `druidapi` Python library](../tutorials/tutorial-jupyter-index.md),
+ provided as part of the Druid tutorials, for functions you can
+ use in a Jupyter notebook to learn how security works, and to set up users and roles for testing.
## Enable authorizers
-After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
+After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
The following diagram depicts the authorization model, and the relationship between users, roles, permissions, and resources.
-
-
+
-The following steps walk through a sample setup procedure:
+
+The following steps walk through a sample setup procedure:
> The default Coordinator API port is 8081 for non-TLS connections and 8281 for secured connections.
-1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`, replacing USERNAME with the *new* username you are trying to create. For example:
- ```
+1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`,
+ replacing USERNAME with the *new* username you are trying to create. For example:
Review Comment:
```suggestion
1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`.
Replace `<USERNAME>` with the *new* username you are trying to create. For example:
```
##########
docs/operations/security-overview.md:
##########
@@ -157,10 +148,12 @@ An example configuration:
# Default password for internal 'druid_system' user, should be changed for production.
druid.auth.authenticator.MyBasicMetadataAuthenticator.initialInternalClientPassword=password2
- # Uses the metadata store for storing users, you can use authentication API to create new users and grant permissions
+ # Uses the metadata store for storing users.
+ # You can use authentication API to create new users and grant permissions
druid.auth.authenticator.MyBasicMetadataAuthenticator.credentialsValidator.type=metadata
- # If true and the request credential doesn't exists in this credentials store, the request will proceed to next Authenticator in the chain.
+ # If true and the request credential doesn't exist in this credentials store,
Review Comment:
```suggestion
# If true and if the request credential doesn't exist in this credentials store,
```
The config is set to false here, so I wonder if this comment would be better rewritten to describe what's actually done. Something about not proceeding to the next authenticator, perhaps?
##########
docs/operations/security-overview.md:
##########
@@ -176,70 +169,83 @@ An example configuration:
druid.auth.authorizer.MyBasicMetadataAuthorizer.type=basic
```
-3. Restart the cluster.
+3. Restart the cluster.
-See [Authentication and Authorization](../design/auth.md) for more information about the Authenticator, Escalator, and Authorizer concepts. See [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about the extension used in the examples above, and [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+See the following topics for more information:
+* [Authentication and Authorization](../design/auth.md) for more information about the Authenticator,
+Escalator, and Authorizer.
+* [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about
+the extension used in the examples above.
+* [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+* [User authentication and authorization](security-user-auth.md) for details about permissions.
+* [SQL permissions](security-user-auth.md#sql-permissions) for permissions on SQL system tables.
+* [The `druidapi` Python library](../tutorials/tutorial-jupyter-index.md),
+ provided as part of the Druid tutorials, for functions you can
+ use in a Jupyter notebook to learn how security works, and to set up users and roles for testing.
## Enable authorizers
-After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
+After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
The following diagram depicts the authorization model, and the relationship between users, roles, permissions, and resources.
-
-
+
-The following steps walk through a sample setup procedure:
+
+The following steps walk through a sample setup procedure:
> The default Coordinator API port is 8081 for non-TLS connections and 8281 for secured connections.
-1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`, replacing USERNAME with the *new* username you are trying to create. For example:
- ```
+1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`,
+ replacing USERNAME with the *new* username you are trying to create. For example:
+ ```bash
curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/myname
- ```
- > If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates, you may choose to include the `insecure` curl option to forgo certificate checking for the curl command.
+ ```
+ > If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates,
Review Comment:
```suggestion
> If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates,
```
##########
docs/operations/security-overview.md:
##########
@@ -176,70 +169,83 @@ An example configuration:
druid.auth.authorizer.MyBasicMetadataAuthorizer.type=basic
```
-3. Restart the cluster.
+3. Restart the cluster.
-See [Authentication and Authorization](../design/auth.md) for more information about the Authenticator, Escalator, and Authorizer concepts. See [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about the extension used in the examples above, and [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+See the following topics for more information:
+* [Authentication and Authorization](../design/auth.md) for more information about the Authenticator,
+Escalator, and Authorizer.
+* [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about
+the extension used in the examples above.
+* [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+* [User authentication and authorization](security-user-auth.md) for details about permissions.
+* [SQL permissions](security-user-auth.md#sql-permissions) for permissions on SQL system tables.
+* [The `druidapi` Python library](../tutorials/tutorial-jupyter-index.md),
+ provided as part of the Druid tutorials, for functions you can
+ use in a Jupyter notebook to learn how security works, and to set up users and roles for testing.
Review Comment:
```suggestion
provided as part of the Druid tutorials, to set up users and roles for learning how security works.
```
##########
examples/quickstart/jupyter-notebooks/druidapi/druidapi/basic_auth.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+BASIC_AUTH_BASE = '/druid-ext/basic-security'
+
+AUTHENTICATION_BASE = BASIC_AUTH_BASE + '/authentication'
+REQ_AUTHENTICATION_LOAD_STATUS = AUTHENTICATION_BASE + '/loadStatus'
+REQ_AUTHENTICATION_REFRESH_ALL = AUTHENTICATION_BASE + '/refreshAll'
+AUTHENTICATOR_BASE = AUTHENTICATION_BASE + '/db/{}'
+REQ_AUTHENTICATION_USERS = AUTHENTICATOR_BASE + '/users'
+REQ_AUTHENTICATION_USER = REQ_AUTHENTICATION_USERS + '/{}'
+REQ_AUTHENTICATION_CREDENTIALS = REQ_AUTHENTICATION_USER + '/credentials'
+
+AUTHORIZATION_BASE = BASIC_AUTH_BASE + '/authorization'
+REQ_AUTHORIZATION_LOAD_STATUS = AUTHORIZATION_BASE + '/loadStatus'
+REQ_AUTHORIZATION_REFRESH_ALL = AUTHORIZATION_BASE + '/refreshAll'
+AUTHORIZATION_BASE = AUTHORIZATION_BASE + '/db/{}'
+REQ_AUTHORIZATION_USERS = AUTHORIZATION_BASE + '/users'
+REQ_AUTHORIZATION_USER = REQ_AUTHORIZATION_USERS + '/{}'
+REQ_AUTHORIZATION_USER_ROLES = REQ_AUTHORIZATION_USER + '/roles'
+REQ_AUTHORIZATION_USER_ROLE = REQ_AUTHORIZATION_USER_ROLES + '/{}'
+REQ_AUTHORIZATION_GROUP_MAPPINGS = AUTHORIZATION_BASE + '/groupMappings'
+REQ_AUTHORIZATION_GROUP_MAPPING = AUTHORIZATION_BASE + '/groupMappings/{}'
+REQ_AUTHORIZATION_GROUP_ROLES = REQ_AUTHORIZATION_GROUP_MAPPING + '/roles'
+REQ_AUTHORIZATION_GROUP_ROLE = REQ_AUTHORIZATION_GROUP_ROLES + '/{}'
+REQ_AUTHORIZATION_ROLES = AUTHORIZATION_BASE + '/roles'
+REQ_AUTHORIZATION_ROLE = REQ_AUTHORIZATION_ROLES + '/{}'
+REQ_AUTHORIZATION_ROLE_PERMISSIONS = REQ_AUTHORIZATION_ROLE + '/permissions'
+REQ_USER_MAP = AUTHORIZATION_BASE + '/cachedSerializedUserMap'
+
+class BasicAuthClient:
+ '''
+ Manage basic security. The Druid session must be logged in with the super
+ user, or some other user who has permission to modify user credentials.
+
+ Each client works with one authorizer/authenticator pair. Create multiple clients if you have to
+ work with multiple authenticators on a single server.
+
+ The basic pattern to add users and permissions is:
+
+ ```
+ # Create a client for your coordinator (Basic auth is not proxied through the router)
+ coord = druidapi.jupyter_client('http://localhost:8081', auth=('admin', 'password'))
+
+ # Get a client for your authenticator and authorizer:
+ ac = coord.basic_security('yourAuthorizer', 'yourAuthenticator')
+
+ # Create a user in both the authenticator and authorizer
+ ac.add_user('bob', 'secret')
+
+ # Define a role
+ ac.add_role('myRole')
+
+ # Assign the role to the user
+ ac.assign_role_to_user('myRole', 'bob')
+
+ # Give the role some permissions
+ ac.grant_permissions('myRole', [[consts.DATASOURCE_RESOURCE, 'foo', consts.READ_ACTION]])
+ ```
+
+ Then use the various other methods to list users, roles and permissions to verify the
Review Comment:
```suggestion
Then use the various other methods to list users, roles, and permissions to verify the
```
##########
docs/operations/security-overview.md:
##########
@@ -176,70 +169,83 @@ An example configuration:
druid.auth.authorizer.MyBasicMetadataAuthorizer.type=basic
```
-3. Restart the cluster.
+3. Restart the cluster.
-See [Authentication and Authorization](../design/auth.md) for more information about the Authenticator, Escalator, and Authorizer concepts. See [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about the extension used in the examples above, and [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+See the following topics for more information:
+* [Authentication and Authorization](../design/auth.md) for more information about the Authenticator,
+Escalator, and Authorizer.
+* [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about
+the extension used in the examples above.
+* [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
+* [User authentication and authorization](security-user-auth.md) for details about permissions.
+* [SQL permissions](security-user-auth.md#sql-permissions) for permissions on SQL system tables.
+* [The `druidapi` Python library](../tutorials/tutorial-jupyter-index.md),
+ provided as part of the Druid tutorials, for functions you can
+ use in a Jupyter notebook to learn how security works, and to set up users and roles for testing.
## Enable authorizers
-After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
+After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
The following diagram depicts the authorization model, and the relationship between users, roles, permissions, and resources.
-
-
+
-The following steps walk through a sample setup procedure:
+
+The following steps walk through a sample setup procedure:
> The default Coordinator API port is 8081 for non-TLS connections and 8281 for secured connections.
-1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`, replacing USERNAME with the *new* username you are trying to create. For example:
- ```
+1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`,
+ replacing USERNAME with the *new* username you are trying to create. For example:
+ ```bash
curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/myname
- ```
- > If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates, you may choose to include the `insecure` curl option to forgo certificate checking for the curl command.
+ ```
+ > If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates,
+ you may choose to include the `insecure` curl option to forgo certificate checking for the curl command.
+
2. Add a credential for the user by issuing a POST to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>/credentials`. For example:
- ```
- curl -u admin:password1 -H'Content-Type: application/json' -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/myname/credentials --data-raw '{"password": "my_password"}'
- ```
-2. For each authenticator user you create, create a corresponding authorizer user by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>`. For example:
- ```
- curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/myname
- ```
-3. Create authorizer roles to control permissions by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>`. For example:
- ```
+ ```bash
+ curl -u admin:password1 -H'Content-Type: application/json' -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/myname/credentials --data-raw '{"password": "my_password"}'
+ ```
+3. For each authenticator user you create, create a corresponding authorizer user by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>`. For example:
+ ```bash
+ curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/myname
+ ```
+4. Create authorizer roles to control permissions by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>`. For example:
+ ```bash
curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/myrole
```
-4. Assign roles to users by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>/roles/<ROLENAME>`. For example:
- ```
- curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/myname/roles/myrole | jq
- ```
-5. Finally, attach permissions to the roles to control how they can interact with Druid at `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>/permissions`.
- For example:
- ```
- curl -u admin:password1 -H'Content-Type: application/json' -XPOST --data-binary @perms.json https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/myrole/permissions
- ```
- The payload of `perms.json` should be in the form:
- ```
- [
- {
- "resource": {
- "name": "<PATTERN>",
- "type": "DATASOURCE"
+5. Assign roles to users by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>/roles/<ROLENAME>`. For example:
+ ```bash
+ curl -u admin:password1 -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/myname/roles/myrole | jq
+ ```
+
+6. Finally, attach permissions to the roles to control how they can interact with Druid at `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>/permissions`. For example:
+ ```bash
+ curl -u admin:password1 -H'Content-Type: application/json' -XPOST --data-binary @perms.json https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/myrole/permissions
+ ```
+ payload of `perms.json` should be in the form:
Review Comment:
```suggestion
The payload of `perms.json` should be in the following form:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org