You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/04/12 16:32:18 UTC

[GitHub] [ozone] smengcl opened a new pull request, #3298: HDDS-6574. Owner field of a bucket created via S3g should be set to the actual user rather than "s3g"

smengcl opened a new pull request, #3298:
URL: https://github.com/apache/ozone/pull/3298

   ## What changes were proposed in this pull request?
   
   Currently if a bucket is created using S3 API via S3 Gateway, the bucket owner is always set to "s3g" (whoever S3 Gateway authenticated itself as to the Ozone Manager).
   
   ```bash
   $ ozone sh bucket list /tenant1
   [ {
     "metadata" : { },
     "volumeName" : "s3v",
     "name" : "bucket1",
     "storageType" : "DISK",
     "versioning" : false,
     "usedBytes" : 0,
     "usedNamespace" : 0,
     "creationTime" : "2022-04-07T14:51:49.053Z",
     "modificationTime" : "2022-04-07T14:51:49.053Z",
     "quotaInBytes" : -1,
     "quotaInNamespace" : -1,
     "bucketLayout" : "OBJECT_STORE",
     "owner" : "s3g",  <---
     "link" : false
   } ]
   ```
   
   This bucket owner field should be set to the user behind the accessId (short name) as this should be the intended behavior.
   
   ```bash
   $ ozone sh bucket list /tenant1
   [ {
   ...
     "bucketLayout" : "OBJECT_STORE",
     "owner" : "hive",  <---
     "link" : false
   } ]
   ```
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-6574
   
   ## How was this patch tested?
   
   - [x] Add an acceptance test case
   - [ ] All existing tests shall pass


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] smengcl commented on a diff in pull request #3298: HDDS-6574. Owner field of a bucket created via S3g should be set to the actual user rather than "s3g"

Posted by GitBox <gi...@apache.org>.
smengcl commented on code in PR #3298:
URL: https://github.com/apache/ozone/pull/3298#discussion_r848659291


##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java:
##########
@@ -533,8 +533,18 @@ public void createBucket(
     verifyCountsQuota(bucketArgs.getQuotaInNamespace());
     verifySpaceQuota(bucketArgs.getQuotaInBytes());
 
-    String owner = bucketArgs.getOwner() == null ?
-            ugi.getShortUserName() : bucketArgs.getOwner();
+    String owner;
+    // If S3 auth exists, set owner name to the short user name derived from the
+    //  accessId. Similar to RpcClient#getDEK
+    if (getThreadLocalS3Auth() != null) {
+      UserGroupInformation s3gUGI = UserGroupInformation.createRemoteUser(
+          getThreadLocalS3Auth().getAccessID());

Review Comment:
   Yes. In multi-tenancy branch we will just need to replace `getThreadLocalS3Auth().getAccessID()` with `getThreadLocalS3Auth().getUserPrincipal()` instead. Similar to this other patch:
   
   https://github.com/apache/ozone/commit/2e6754f6cc0f077ee6738ca76820ac7c6db391ef#diff-4474d511c0022f836152b8523c020d9448508d8dc5ecce2e5fb4a43b3b192e74R1064-R1068



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] avijayanhwx commented on a diff in pull request #3298: HDDS-6574. Owner field of a bucket created via S3g should be set to the actual user rather than "s3g"

Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on code in PR #3298:
URL: https://github.com/apache/ozone/pull/3298#discussion_r848648224


##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java:
##########
@@ -533,8 +533,18 @@ public void createBucket(
     verifyCountsQuota(bucketArgs.getQuotaInNamespace());
     verifySpaceQuota(bucketArgs.getQuotaInBytes());
 
-    String owner = bucketArgs.getOwner() == null ?
-            ugi.getShortUserName() : bucketArgs.getOwner();
+    String owner;
+    // If S3 auth exists, set owner name to the short user name derived from the
+    //  accessId. Similar to RpcClient#getDEK
+    if (getThreadLocalS3Auth() != null) {
+      UserGroupInformation s3gUGI = UserGroupInformation.createRemoteUser(
+          getThreadLocalS3Auth().getAccessID());

Review Comment:
   Will there be a follow up to fix this in the s3 Multi Tenancy branch? (Given the accessId is NOT the actual user name)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai commented on pull request #3298: HDDS-6574. Set owner of buckets created via S3 Gateway to actual user rather than `s3g`; print LinkBucket owner field on the client

Posted by GitBox <gi...@apache.org>.
adoroszlai commented on PR #3298:
URL: https://github.com/apache/ozone/pull/3298#issuecomment-1102254607

   Thanks @smengcl for the patch, @avijayanhwx and @kerneltime for the review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] kerneltime commented on pull request #3298: HDDS-6574. Set owner of buckets created via S3 Gateway to actual user rather than `s3g`; print LinkBucket owner field on the client

Posted by GitBox <gi...@apache.org>.
kerneltime commented on PR #3298:
URL: https://github.com/apache/ozone/pull/3298#issuecomment-1101811317

   Thanks for catching this! LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] smengcl commented on a diff in pull request #3298: HDDS-6574. Owner field of a bucket created via S3g should be set to the actual user rather than "s3g"

Posted by GitBox <gi...@apache.org>.
smengcl commented on code in PR #3298:
URL: https://github.com/apache/ozone/pull/3298#discussion_r848659291


##########
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java:
##########
@@ -533,8 +533,18 @@ public void createBucket(
     verifyCountsQuota(bucketArgs.getQuotaInNamespace());
     verifySpaceQuota(bucketArgs.getQuotaInBytes());
 
-    String owner = bucketArgs.getOwner() == null ?
-            ugi.getShortUserName() : bucketArgs.getOwner();
+    String owner;
+    // If S3 auth exists, set owner name to the short user name derived from the
+    //  accessId. Similar to RpcClient#getDEK
+    if (getThreadLocalS3Auth() != null) {
+      UserGroupInformation s3gUGI = UserGroupInformation.createRemoteUser(
+          getThreadLocalS3Auth().getAccessID());

Review Comment:
   Yes. In multi-tenancy branch we will just need to replace `getThreadLocalS3Auth().getAccessID()` with `getThreadLocalS3Auth().getUserPrincipal()` instead.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] smengcl commented on pull request #3298: HDDS-6574. Set owner of buckets created via S3 Gateway to actual user rather than `s3g`; print LinkBucket owner field on the client

Posted by GitBox <gi...@apache.org>.
smengcl commented on PR #3298:
URL: https://github.com/apache/ozone/pull/3298#issuecomment-1104324662

   Thanks @avijayanhwx and @kerneltime for reviewing this. Thanks @adoroszlai for merging this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] smengcl commented on pull request #3298: HDDS-6574. Set owner of buckets created via S3 Gateway to actual user rather than `s3g`; print LinkBucket owner field on the client

Posted by GitBox <gi...@apache.org>.
smengcl commented on PR #3298:
URL: https://github.com/apache/ozone/pull/3298#issuecomment-1098645832

   Ah. Finally got a good run after a bunch of test tweaking. @avijayanhwx would you take a look again when you have time. Thx


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai merged pull request #3298: HDDS-6574. Set owner of buckets created via S3 Gateway to actual user rather than `s3g`; print LinkBucket owner field on the client

Posted by GitBox <gi...@apache.org>.
adoroszlai merged PR #3298:
URL: https://github.com/apache/ozone/pull/3298


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org