You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ozone.apache.org by si...@apache.org on 2022/03/07 20:10:08 UTC
[ozone] branch HDDS-4944 updated: HDDS-6275. [Multi-Tenant] Add feature documentation and CLI quick start guide (#3095)
This is an automated email from the ASF dual-hosted git repository.
siyao pushed a commit to branch HDDS-4944
in repository https://gitbox.apache.org/repos/asf/ozone.git
The following commit(s) were added to refs/heads/HDDS-4944 by this push:
new ecb685a HDDS-6275. [Multi-Tenant] Add feature documentation and CLI quick start guide (#3095)
ecb685a is described below
commit ecb685a6cf8aa463e9a3e1836900e56c52ed5597
Author: Siyao Meng <50...@users.noreply.github.com>
AuthorDate: Mon Mar 7 12:09:51 2022 -0800
HDDS-6275. [Multi-Tenant] Add feature documentation and CLI quick start guide (#3095)
---
.../docs/content/feature/Nonrolling-Upgrade.md | 1 +
.../feature/S3-Multi-Tenancy-Access-Control.md | 68 ++++
.../docs/content/feature/S3-Multi-Tenancy-Setup.md | 108 +++++++
.../docs/content/feature/S3-Multi-Tenancy.md | 75 +++++
.../docs/content/feature/S3-Tenant-Commands.md | 350 +++++++++++++++++++++
.../themes/ozonedoc/layouts/_default/section.html | 3 +-
.../ozonesecure/mockserverInitialization.json | 8 +
.../hadoop/ozone/shell/TestOzoneTenantShell.java | 2 +-
.../shell/tenant/TenantAssignAdminHandler.java | 2 +-
.../shell/tenant/TenantBucketLinkHandler.java | 5 +-
.../ozone/shell/tenant/TenantListUsersHandler.java | 3 +-
.../shell/tenant/TenantRevokeAdminHandler.java | 4 +-
12 files changed, 620 insertions(+), 9 deletions(-)
diff --git a/hadoop-hdds/docs/content/feature/Nonrolling-Upgrade.md b/hadoop-hdds/docs/content/feature/Nonrolling-Upgrade.md
index bd7eb80..0328e68 100644
--- a/hadoop-hdds/docs/content/feature/Nonrolling-Upgrade.md
+++ b/hadoop-hdds/docs/content/feature/Nonrolling-Upgrade.md
@@ -1,5 +1,6 @@
---
title: Non-Rolling Upgrades and Downgrades
+weight: 9
menu:
main:
parent: Features
diff --git a/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Access-Control.md b/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Access-Control.md
new file mode 100644
index 0000000..a31b895
--- /dev/null
+++ b/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Access-Control.md
@@ -0,0 +1,68 @@
+---
+title: "Access Control"
+weight: 13
+menu:
+ main:
+ parent: "S3 Multi-Tenancy"
+summary: Access Control with Ranger in Ozone Multi-Tenancy
+hideFromSectionPage: true
+---
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+### Ranger Policies
+
+When a tenant is created, Ozone will create a set of Ranger policies on the tenant's volume which allow the following:
+
+1. All users are able to create new buckets;
+2. Only the bucket owner (i.e. the user that creates the bucket) and tenant admins can access the bucket content.
+ - Note: For Ozone admins, typically there would be other Ranger policies that grants them full access to the cluster, if this is case they should be able to access the buckets as well. Though it is still possible to create new Ranger policies to explicitly deny them access to buckets.
+
+Ranger admin is responsible for manually adding new policies to grant or deny any other access patterns. For example:
+- Allow all users in a tenant read-only access to a bucket.
+ - Corresponding Ranger policy Allow Condition: `Roles = tenantName-UserRole, Permissions = READ,LIST`
+
+It is recommended to add new policies instead of editing the default tenant policies created by Ozone. **DO NOT** remove the **Policy Label** on those default tenant policies, or else the Ozone Manager might fail to sync with Ranger for those policies.
+
+### Ranger Roles
+
+These new Ranger policies would have the corresponding **Ranger roles** added in their **Allow Conditions**.
+
+Namely, `tenantName-UserRole` and `tenantName-AdminRole` Ranger roles are created when a tenant is created by an Ozone administrator under the CLI.
+
+`tenantName-UserRole` contains a list of all user names that are assigned to this tenant.
+
+`tenantName-AdminRole` contains a list of all tenant admins that are assigned to this tenant.
+
+We leverage Ranger roles mainly for the advantage of easier user management in a tenant:
+1. When new users are assigned to a tenant, Ozone Manager simply adds the new user to `tenantName-UserRole` Ranger role.
+2. When new tenant admins are assigned, Ozone Manager simply adds the user name to `tenantName-AdminRole` Ranger role. Delegated tenant admins will have the "Role Admin" checkbox checked, while non-delegated tenant admins won't.
+ - Role admins in a Ranger role has the permission to edit that Ranger role.
+3. And because `tenantName-AdminRole` is the "Role Admin" of `tenantName-UserRole`, whichever user in the `tenantName-AdminRole` automatically has the permission to add new users to the tenant, meaning all tenant admins (whether delegated or not) has the permission to assign and revoke users in this tenant.
+
+- **DO NOT** manually edit any Ranger roles created by Ozone. Any changes to them will be overwritten by the Ozone Manager's Ranger sync thread. Changes in tenant membership should be done using [Multi-Tenancy CLI commands]({{< ref "feature/S3-Tenant-Commands.md" >}}).
+
+
+### Ranger Sync
+
+A Ranger Sync thread has been implemented to keep the Ranger policy and role states in-sync with Ozone Manager database in case of Ozone Manager crashes during tenant administrative operations.
+
+The Ranger Sync thread does the following:
+1. Cleans up any default tenant policies if a tenant is already deleted.
+2. Checks if default tenant roles are out-of-sync (could be caused by OM crash during user assign/revoke operation). Overwrites them if this is the case.
+3. Performs all Ranger update (write) operations queued by Ozone tenant commands from the last sync, if any.
+ - This implies there will be a delay before Ranger policies and roles are updated for any tenant write operations (tenant create/delete, tenant user assign/revoke/assignadmin/revokeadmin, etc.).
diff --git a/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Setup.md b/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Setup.md
new file mode 100644
index 0000000..d6f77e7
--- /dev/null
+++ b/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy-Setup.md
@@ -0,0 +1,108 @@
+---
+title: "Setup"
+weight: 11
+menu:
+ main:
+ parent: "S3 Multi-Tenancy"
+summary: Preparing Ozone clusters to enable Multi-Tenancy feature
+hideFromSectionPage: true
+---
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+Steps to enable S3 Multi-Tenancy feature in Ozone clusters.
+
+
+## Setting up S3 Multi-Tenancy for production
+
+### Secure the cluster
+
+Follow [this guide]({{< ref "security/SecureOzone.md" >}}) to Kerberize (secure) the cluster if the cluster is not Kerberized yet.
+
+### Set up S3 Gateway
+
+Follow [this guide]({{< ref "interface/S3.md" >}}) the cluster to set up at least one S3 Gateway if the cluster doesn't have a S3 Gateway yet.
+
+### Set up Apache Ranger
+
+First make sure ACL is enabled, and `RangerOzoneAuthorizer` is the effective ACL authorizer implementation in-use for Ozone.
+If that is not the case, [follow this]({{< ref "security/SecurityWithRanger.md" >}}).
+
+Then simply add the following configs to `ozone-site.xml`:
+
+```xml
+<property>
+ <name>ozone.om.ranger.https-address</name>
+ <value>https://RANGER_HOSTNAME:6182</value>
+</property>
+<property>
+ <name>ozone.om.ranger.https.admin.api.user</name>
+ <value>RANGER_ADMIN_USERNAME</value>
+</property>
+<property>
+ <name>ozone.om.ranger.https.admin.api.passwd</name>
+ <value>RANGER_ADMIN_PASSWORD</value>
+</property>
+```
+
+Finally restart all OzoneManagers to apply the new configs.
+
+Now you can follow the [Multi-Tenancy CLI command]({{< ref "feature/S3-Tenant-Commands.md" >}}) guide to try the commands.
+
+
+## Try in a Docker Compose cluster (For developers)
+
+Developers are encouraged to try out the CLI commands inside the `./compose/ozonesecure/docker-compose.yaml` cluster environment that ships with Ozone.
+
+The Docker Compose cluster has Kerberos and security pre-configured.
+But note it differs from an actual production cluster that Ranger has been replaced with a mock server. And OzoneManager does **not** use Ranger for ACL.
+
+Because the mock server does not mock all Ranger endpoints, some operations that works for a real Ranger deployment will not work by default. e.g. assigning users to a tenant other than `tenantone`.
+But one can add new custom endpoints in `./compose/ozonesecure/mockserverInitialization.json` as needed.
+
+To launch the Docker Compose cluster locally, from Ozone distribution root:
+
+```shell
+cd compose/ozonesecure
+docker-compose up -d --scale datanode=3
+docker-compose exec scm bash
+```
+
+It might be necessary to run the following command first before testing the tenant commands in the `compose/ozonesecure` Docker environment
+in order to workaround a Docker-specific DNS issue when first contacting Ranger.
+
+```shell
+bash-4.2$ curl -k https://ranger:6182/
+{}
+```
+
+Then all subsequent requests to Ranger (mock server) should work as expected.
+
+Otherwise you might see such DNS error:
+
+```shell
+bash-4.2$ ozone tenant create tenantone
+2022-02-16 00:00:00,000 [main] INFO rpc.RpcClient: Creating Tenant: 'tenantone', with new volume: 'tenantone'
+INTERNAL_ERROR No subject alternative DNS name matching ranger found.
+```
+
+
+Operations requiring Ozone cluster administrator privilege are run as `om/om` user:
+
+```shell
+kinit -kt /etc/security/keytabs/om.keytab om/om@EXAMPLE.COM
+```
diff --git a/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy.md b/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy.md
new file mode 100644
index 0000000..9e90f0b
--- /dev/null
+++ b/hadoop-hdds/docs/content/feature/S3-Multi-Tenancy.md
@@ -0,0 +1,75 @@
+---
+title: "S3 Multi-Tenancy"
+weight: 10
+menu:
+ main:
+ parent: Features
+summary: Ozone Multi-Tenancy that allows multiple tenants to share the same Ozone cluster. Compatible with S3 API.
+---
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+Before Ozone multi-tenancy, all S3 access to Ozone (via [S3 Gateway]({{< ref "interface/S3.md" >}})) are
+confined to a **single** designated S3 volume (that is volume `s3v`, by default).
+
+Ozone multi-tenancy allows **multiple** S3-accessible volumes to be created.
+Each volume can be managed separately by their own tenant admins via CLI for user operations, and via Apache Ranger for access control.
+
+The concept **tenant** is introduced to Ozone by multi-tenancy.
+Each tenant has its own designated volume.
+Each user assigned to a tenant will be able to access the associated volume with an _Access ID & Secret Key_ pair
+generated when an Ozone cluster admin or tenant admin assigns the user to the tenant using CLI.
+
+> This multi-tenant feature allows Ozone resources to be compartmentalized in different isolation zones (tenants).
+
+> This multi-tenant support will also allow users to access Ozone volumes over AWS S3 APIs (without any modifications to the APIs).
+
+## Basics
+
+1. Initial tenant creation has to be done by an Ozone cluster admin under the CLI.
+2. The Ozone cluster admin will have to assign the first user of a tenant. Once assigned, an _Access ID & Secret Key pair_ (key pair) will be generated for that user for access via S3 Gateway.
+ - The key pair serves to authenticate the end user to the Ozone Manager (via client requests from S3 Gateway). Tenant volume is selected based on the Access ID.
+ - After successful authentication, Ozone Manager uses the underlying user name (not the Access ID) to identify the user. The user name is used to perform authorization checks in Apache Ranger.
+ - A key pair is tied to the user name it is assigned to. If a user is assigned key pairs in multiple tenants, all key pairs point to the same user name internally in Ozone Manager.
+ - A user can be only assigned one key pair in a same tenant. Ozone Manager rejects the tenant user assign request if a user is already assigned to the same tenant (i.e. when the user has already been assigned an Access ID in this tenant).
+ - One user can be assigned to multiple tenants. The user will have a different key pair to access each tenant. For instance, `testuser` could use `tenantone$testuser` to access `tenantone` buckets, and use `tenanttwo$testuser` to access `tenanttwo` buckets via S3 Gateway.
+ - A bucket link can be set up if cross-tenant (cross-volume) access is desired. See **Creating bucket links** section in [Tenant commands]({{< ref "feature/S3-Tenant-Commands.md" >}}).
+3. The Ozone cluster admin can then assign tenant admin roles to that user.
+4. Tenant admin are able to assign new users to the tenant.
+ - They can even assign new tenant admins in their tenant, if they are delegated tenant admins, which is the default. See **Assign a user as a tenant admin** section in [Tenant commands]({{< ref "feature/S3-Tenant-Commands.md" >}}).
+ - Note that tenant admins still need to use Ozone tenant CLI to assign new users to the tenant.
+ - Once tenant admins get the Kerberos TGT (via `kinit`), they can run `user assign` command to assign new users. Ozone Manager will recognize that they are the tenant admins and allow the user to do so in their tenants.
+5. After that, users can use any S3-compatible client (awscli, Python boto3 library, etc.) to access the buckets in the tenant volume via S3 Gateway using the generated key pairs.
+
+
+## Access Control
+
+Ozone multi-tenancy relies on [Apache Ranger]({{< ref "security/SecurityWithRanger.md" >}}) to enforce access control to resources.
+
+See [Access Control]({{< ref "feature/S3-Multi-Tenancy-Access-Control.md" >}}) for more information.
+
+## Setup
+
+See [Multi-Tenancy Setup]({{< ref "S3-Multi-Tenancy-Setup.md" >}}).
+
+## Usage
+
+See [Tenant Subcommands]({{< ref "feature/S3-Tenant-Commands.md" >}}).
+
+## References
+
+ * For developers: check out the upstream jira [HDDS-4944](https://issues.apache.org/jira/browse/HDDS-4944) and the attached design docs.
diff --git a/hadoop-hdds/docs/content/feature/S3-Tenant-Commands.md b/hadoop-hdds/docs/content/feature/S3-Tenant-Commands.md
new file mode 100644
index 0000000..247d488
--- /dev/null
+++ b/hadoop-hdds/docs/content/feature/S3-Tenant-Commands.md
@@ -0,0 +1,350 @@
+---
+title: "Tenant commands"
+weight: 12
+menu:
+ main:
+ parent: "S3 Multi-Tenancy"
+summary: Ozone subcommands for S3 tenant management
+hideFromSectionPage: true
+---
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+For a higher level understanding of multi-tenancy architecture, see [Multi-Tenancy feature]({{< ref "feature/S3-Multi-Tenancy.md" >}}).
+
+All Multi-Tenancy subcommands are located under CLI `ozone tenant`.
+
+The commands below assume a Kerberized Ozone cluster with Ranger install. Enabling HTTPS on S3 Gateway is optional but recommended.
+
+## Quick Start
+
+### Setup
+
+Follow the [Multi-Tenancy Setup]({{< ref "feature/S3-Multi-Tenancy-Setup.md" >}}) guide if you haven't done so.
+
+If the OzoneManagers are running in HA, append `--om-service-id=` accordingly to the commands.
+
+### Create a tenant
+
+Create a new tenant in the current Ozone cluster.
+This operation requires Ozone cluster administrator privilege.
+
+Apart from adding new OM DB entries, creating a tenant also does the following in the background:
+1. Creates a volume of the exact same name. Therefore, volume name restrictions apply to the tenant name as well. Specifying a custom volume name during tenant creation is not supported yet. Tenant volume cannot be changed once the tenant is created.
+2. Creates two new Ranger roles, `tenantName-UserRole` and `tenantName-AdminRole`.
+3. Creates new Ranger policies that allows all tenant users to list and create buckets by default under the tenant volume, but only bucket owners and tenant admins are allowed to access the bucket contents.
+
+```shell
+ozone tenant create <TENANT_NAME>
+```
+
+Example:
+
+```shell
+bash-4.2$ kinit -kt /etc/security/keytabs/om.keytab om/om@EXAMPLE.COM
+bash-4.2$ ozone tenant create tenantone
+2022-02-16 00:00:00,000 [main] INFO rpc.RpcClient: Creating Tenant: 'tenantone', with new volume: 'tenantone'
+Created tenant 'tenantone'.
+```
+
+
+### List tenants
+
+List all tenants in an Ozone cluster.
+
+```shell
+ozone tenant list
+```
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant list
+tenantone
+```
+
+
+### Assign a user to a tenant
+
+The first user in a tenant must be assigned by an Ozone cluster administrator.
+
+By default when user `testuser` is assigned to tenant `tenantone`, the generated Access ID for the user in this tenant is `tenantone$testuser`.
+
+- Be sure to enclose the Access ID in single quotes in Bash when using it so it doesn't get auto-translated into environment variables.
+
+It is possible to assign a user to multiple tenants.
+
+```shell
+ozone tenant user assign <USER_NAME> --tenant=<TENANT_NAME>
+```
+
+`<USER_NAME>` should be a short user name for a Kerberos principal, e.g. `testuser` when the Kerberos principal is `testuser/scm@EXAMPLE.COM`
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant user assign testuser --tenant=tenantone
+Assigned 'testuser' to 'tenantone' with accessId 'tenantone$testuser'.
+export AWS_ACCESS_KEY_ID='tenantone$testuser'
+export AWS_SECRET_ACCESS_KEY='<GENERATED_SECRET>'
+```
+
+
+### Assign a user as a tenant admin
+
+The first user in a tenant must be assigned by an Ozone cluster administrator.
+
+Both delegated and non-delegated tenant admin can assign and revoke **regular** tenant users.
+
+The only difference between delegated tenant admin and non-delegated tenant admin is that delegated tenant admin can assign and revoke tenant **admins** in the tenant,
+while non-delegated tenant admin can't.
+
+Unless `--delegated=false` is specified, `ozone tenant assignadmin` assigns **delegated** tenant admins by default.
+
+It is possible to assign a user to be tenant admins in multiple tenants.
+
+```shell
+ozone tenant user assignadmin <ACCESS_ID> --delegated=true --tenant=<TENANT_NAME>
+```
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant user assignadmin 'tenantone$testuser' --tenant=tenantone
+Assigned admin to 'tenantone$testuser' in tenant 'tenantone'
+```
+
+Once `testuser` becomes a tenant admin of `tenantone`, one can kinit as `testuser` and assign new users to the tenant,
+even new tenant admins (if delegated). Example commands for illustration:
+
+```shell
+kinit -kt /etc/security/keytabs/testuser.keytab testuser/scm@EXAMPLE.COM
+ozone tenant user assign testuser2 --tenant=tenantone
+ozone tenant user assignadmin 'tenantone$testuser2' --tenant=tenantone
+```
+
+
+### List users in a tenant
+
+```shell
+ozone tenant user list --tenant=<TENANT_NAME>
+```
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant user list --tenant=tenantone
+- User 'testuser' with accessId 'tenantone$testuser'
+```
+
+
+### Get tenant user info
+
+This command lists all tenants a user is assigned to.
+
+```shell
+ozone tenant user info <USER_NAME>
+```
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant user info testuser
+User 'testuser' is assigned to:
+- Tenant 'tenantone' delegated admin with accessId 'tenantone$testuser'
+```
+
+
+### Revoke a tenant admin
+
+```shell
+ozone tenant user revokeadmin <ACCESS_ID>
+```
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant user revokeadmin 'tenantone$testuser'
+Revoked admin role of 'tenantone$testuser'.
+```
+
+
+### Revoke user access from a tenant
+
+```shell
+ozone tenant user revoke <ACCESS_ID>
+```
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant user revoke 'tenantone$testuser'
+Revoked accessId 'tenantone$testuser'.
+```
+
+
+### Delete a tenant
+
+In order to be able to delete a tenant, the tenant has to be empty. i.e. All users need to be revoked before a tenant can be deleted.
+Otherwise OM will throw `TENANT_NOT_EMPTY` exception and refuse to delete the tenant.
+
+Note that it is intentional by design that the volume created and associated with the tenant during tenant creation is not removed.
+An admin has to remove the volume manually as prompt in the CLI, if deemed necessary.
+
+```shell
+ozone tenant delete <TENANT_NAME>
+```
+
+Example:
+
+```shell
+bash-4.2$ ozone tenant delete tenantone
+Deleted tenant 'tenantone'.
+But the associated volume 'tenantone' is not removed. To delete it, run
+ ozone sh volume delete tenantone
+```
+
+If an Ozone cluster admin (or whoever has the permission to delete the volume in Ranger) tries delete a volume before the tenant is deleted using the command above,
+the `ozone sh volume delete` command would fail because the volume reference count is not zero:
+
+```shell
+bash-4.2$ ozone sh volume delete tenantone
+VOLUME_IS_REFERENCED Volume reference count is not zero (1). Ozone features are enabled on this volume. Try `ozone tenant delete <tenantId>` first.
+```
+
+
+## Creating bucket links
+
+Bucket links can be used to allow access to buckets outside of the tenant volume.
+
+Bucket (sym)links are a special type of bucket that points to other buckets in the same Ozone cluster. It is similar to POSIX symbolic links.
+
+An example to create a bucket link:
+
+```shell
+$ ozone tenant linkbucket /vol1/bucket1 /tenantone/linked-bucket1
+```
+
+The command above creates a bucket symlink `linked-bucket1` in volume `tenantone`, which points to `bucket1` in `vol1`.
+
+As long as the user running this command has the permission to create a bucket in the target volume `tenantone`, the command will succeed.
+
+- The link bucket command itself does not check for permission to access the source volume and bucket.
+- The link bucket command will not even check if the source volume and bucket exists.
+- Permission check will be performed when the bucket symlink is actually accessed.
+ - In order to grant a user in tenant `tenantone` access the bucket, a new policy should be added by a Ranger admin that allow that user intended permissions (`READ, WRITE, LIST, CREATE, DELETE, ...`) to the source bucket `bucket1` in volume `vol1`.
+- At the moment, `ozone tenant linkbucket` command is equivalent to `ozone sh bucket link` command (see **Expose any volume** section in [S3 protocol]({{< ref "S3.md" >}})).
+
+
+## Example: Accessing a bucket in a tenant volume via S3 Gateway using S3 API
+
+Here is an example of accessing the bucket using AWS CLI in the Docker Compose cluster, with tenant `tenantone` created and `testuser` assigned to the tenant.
+
+### Configure AWS CLI
+
+```shell
+bash-4.2$ aws configure
+AWS Access Key ID [****************fslf]: tenantone$testuser
+AWS Secret Access Key [****************fslf]: <GENERATED_SECRET>
+Default region name [us-west-1]:
+Default output format [None]:
+```
+
+### List buckets, create a bucket
+
+```shell
+bash-4.2$ aws s3api --endpoint-url http://s3g:9878 list-buckets
+{
+ "Buckets": []
+}
+bash-4.2$ aws s3api --endpoint-url http://s3g:9878 create-bucket --bucket bucket-test1
+{
+ "Location": "http://s3g:9878/bucket-test1"
+}
+bash-4.2$ aws s3api --endpoint-url http://s3g:9878 list-buckets
+{
+ "Buckets": [
+ {
+ "Name": "bucket-test1",
+ "CreationDate": "2022-02-16T00:05:00.000Z"
+ }
+ ]
+}
+```
+
+In the Docker Compose cluster, the AWS CLI might report `AccessDenied` because it uses a mocked Ranger endpoint (which can't be used to perform authorization). A production Ranger setup uses [`RangerOzoneAuthorizer`](https://github.com/apache/ranger/blob/master/plugin-ozone/src/main/java/org/apache/ranger/authorization/ozone/authorizer/RangerOzoneAuthorizer.java) in OM for authorization while the `ozonesecure` Docker Compose cluster still uses `OzoneNativeAuthorizer`. So a workaround is [...]
+
+```shell
+ozone sh volume update tenantone --user=testuser
+```
+
+The bucket created with `aws s3api` is also visible under Ozone CLI:
+
+```shell
+bash-4.2$ ozone sh bucket list /tenantone
+[ {
+ "metadata" : { },
+ "volumeName" : "tenantone",
+ "name" : "bucket-test1",
+ "storageType" : "DISK",
+ "versioning" : false,
+ "usedBytes" : 0,
+ "usedNamespace" : 0,
+ "creationTime" : "2022-02-16T00:05:00.000Z",
+ "modificationTime" : "2022-02-16T00:05:00.000Z",
+ "quotaInBytes" : -1,
+ "quotaInNamespace" : -1,
+ "bucketLayout" : "OBJECT_STORE",
+ "owner" : "root",
+ "link" : false
+} ]
+```
+
+### Put object (key) to a bucket, list objects
+
+```shell
+bash-4.2$ aws s3api --endpoint-url http://s3g:9878 put-object --bucket bucket-test1 --key file1 --body README.md
+bash-4.2$ aws s3api --endpoint-url http://s3g:9878 list-objects --bucket bucket-test1
+{
+ "Contents": [
+ {
+ "Key": "file1",
+ "LastModified": "2022-02-16T00:10:00.000Z",
+ "ETag": "2022-02-16T00:10:00.000Z",
+ "Size": 3811,
+ "StorageClass": "STANDARD"
+ }
+ ]
+}
+```
+
+### Get object (key) from a bucket
+
+```shell
+bash-4.2$ aws s3api --endpoint-url http://s3g:9878 get-object --bucket bucket-test1 --key file1 file1-get.txt
+{
+ "AcceptRanges": "bytes",
+ "LastModified": "Wed, 16 Feb 2022 00:10:00 GMT",
+ "ContentLength": 3811,
+ "CacheControl": "no-cache",
+ "ContentType": "application/octet-stream",
+ "Expires": "Wed, 16 Feb 2022 00:15:00 GMT",
+ "Metadata": {}
+}
+bash-4.2$ diff file1-get.txt README.md
+```
+
diff --git a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html
index 2963c80..1f6dffa 100644
--- a/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html
+++ b/hadoop-hdds/docs/themes/ozonedoc/layouts/_default/section.html
@@ -36,7 +36,8 @@
{{ .Content }}
{{.Params.card}}
{{ if not (eq .Params.cards "false")}}
- {{ range $page_index, $page_val := .Pages }}
+ {{ $pagesToShow := where .Pages "Params.hideFromSectionPage" "!=" "true" }}
+ {{ range $page_index, $page_val := $pagesToShow }}
{{ $page_count := len .Pages }}
{{if (eq (mod $page_index 2) 0)}}
diff --git a/hadoop-ozone/dist/src/main/compose/ozonesecure/mockserverInitialization.json b/hadoop-ozone/dist/src/main/compose/ozonesecure/mockserverInitialization.json
index 4a54f4c..fbd7735 100644
--- a/hadoop-ozone/dist/src/main/compose/ozonesecure/mockserverInitialization.json
+++ b/hadoop-ozone/dist/src/main/compose/ozonesecure/mockserverInitialization.json
@@ -33,6 +33,14 @@
},
{
"httpRequest": {
+ "path": "/service/roles/roles/name/tenantone-AdminRole"
+ },
+ "httpResponse": {
+ "body": "{id: 222, users: []}"
+ }
+ },
+ {
+ "httpRequest": {
"path": "/service/roles/roles/222"
},
"httpResponse": {
diff --git a/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneTenantShell.java b/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneTenantShell.java
index cbbbc27..6aa868f 100644
--- a/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneTenantShell.java
+++ b/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneTenantShell.java
@@ -525,7 +525,7 @@ public class TestOzoneTenantShell {
"user", "revoke-admin", "dev$bob", "--tenant=dev"});
checkOutput(out, "", true);
checkOutput(err, "Revoked admin role of 'dev$bob' "
- + "from tenant 'dev'\n", true);
+ + "from tenant 'dev'.\n", true);
executeHA(tenantShell, new String[] {
"user", "info", "bob"});
diff --git a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantAssignAdminHandler.java b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantAssignAdminHandler.java
index 59a2cfd..2d9ec62 100644
--- a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantAssignAdminHandler.java
+++ b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantAssignAdminHandler.java
@@ -47,7 +47,7 @@ public class TenantAssignAdminHandler extends TenantHandler {
private String tenantId;
@CommandLine.Option(names = {"-d", "--delegated"}, defaultValue = "true",
- description = "Make delegated admin")
+ description = "Set to true (default) to assign delegated admin")
private boolean delegated;
// TODO: HDDS-6340. Add an option to print JSON result
diff --git a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantBucketLinkHandler.java b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantBucketLinkHandler.java
index 11a7972..3a03600 100644
--- a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantBucketLinkHandler.java
+++ b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantBucketLinkHandler.java
@@ -30,14 +30,13 @@ import picocli.CommandLine.Parameters;
import java.io.IOException;
/**
- * ozone tenant bucket-link.
+ * ozone tenant linkbucket.
*
* Note: Currently this command is exactly the same as `ozone sh bucket link`.
* We might expand this to add more functionality in the future, and different
* ObjectStore API(s) would be used by then.
*/
-@CommandLine.Command(name = "bucket-link",
- aliases = {"link-bucket", "bucketlink", "linkbucket"},
+@CommandLine.Command(name = "linkbucket",
description = "Create a symlink to another bucket")
public class TenantBucketLinkHandler extends TenantHandler {
diff --git a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantListUsersHandler.java b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantListUsersHandler.java
index 8b86f0d..e51d187 100644
--- a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantListUsersHandler.java
+++ b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantListUsersHandler.java
@@ -34,7 +34,8 @@ import picocli.CommandLine;
* Command to list users in a tenant along with corresponding accessId.
*/
@CommandLine.Command(name = "list",
- description = "List Tenant Users")
+ aliases = {"ls"},
+ description = "List users in a tenant")
public class TenantListUsersHandler extends S3Handler {
@CommandLine.Spec
diff --git a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantRevokeAdminHandler.java b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantRevokeAdminHandler.java
index 40ce594..21d40f5 100644
--- a/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantRevokeAdminHandler.java
+++ b/hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/shell/tenant/TenantRevokeAdminHandler.java
@@ -55,10 +55,10 @@ public class TenantRevokeAdminHandler extends TenantHandler {
// TODO: Make tenantRevokeAdmin return accessId, tenantId, user later.
objStore.tenantRevokeAdmin(accessId, tenantId);
err().println("Revoked admin role of '" + accessId +
- (tenantId != null ? "' from tenant '" + tenantId + "'" : ""));
+ (tenantId != null ? "' from tenant '" + tenantId : "") + "'.");
} catch (IOException e) {
err().println("Failed to revoke admin role of '" + accessId +
- (tenantId != null ? "' from tenant '" + tenantId + "'" : "") +
+ (tenantId != null ? "' from tenant '" + tenantId : "") + "'" +
": " + e.getMessage());
if (e instanceof OMException) {
final OMException omEx = (OMException) e;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@ozone.apache.org
For additional commands, e-mail: commits-help@ozone.apache.org