You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "xBis7 (via GitHub)" <gi...@apache.org> on 2023/12/18 16:58:22 UTC

[PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

xBis7 opened a new pull request, #5816:
URL: https://github.com/apache/ozone/pull/5816

   ## What changes were proposed in this pull request?
   
   When doing a `listStatus` on a volume, we were setting the volume owner as the resource owner. Therefore, `ozone fs -ls` was resulting in displaying the volume owner as the owner of every bucket. 
   
   Check the jira description for a test-case for manually reproducing it. 
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-9950
   
   ## How was this patch tested?
   
   A new test was added under `TestRootedOzoneFileSystem`. The patch was also tested manually in ozone docker dev env like so
   ```
   > pwd
   /projects/ozone/hadoop-ozone/dist/target/ozone-1.5.0-SNAPSHOT/compose/ozone
   
     // Start docker env
   > docker-compose up --scale datanode=5 -d
   
     // Create volume with the default user, hadoop
   > docker-compose exec om ozone sh volume create /vol1
   
     // Create bucket and set the owner as testuser
   > docker-compose exec om ozone sh bucket create /vol1/bucket1 --user testuser 
   
     // Create bucket and set the owner as testuser2
   > docker-compose exec om ozone sh bucket create /vol1/bucket2 --user testuser2 
   
   > docker-compose exec om ozone fs -ls /vol1                
   Found 2 items
   drwxrwxrwx   - testuser  testuser           0 2023-12-18 16:57 /vol1/bucket1
   drwxrwxrwx   - testuser2 testuser2          0 2023-12-18 16:57 /vol1/bucket2
   
   > docker-compose exec om ozone sh volume update --user testuser /vol1
   {
     "metadata" : { },
     "name" : "vol1",
     "admin" : "hadoop",
     "owner" : "testuser",
     "quotaInBytes" : -1,
     "quotaInNamespace" : -1,
     "usedNamespace" : 2,
     "creationTime" : "2023-12-18T16:57:07.500Z",
     "modificationTime" : "2023-12-18T16:57:40.318Z",
     "acls" : [ {
       "type" : "USER",
       "name" : "hadoop",
       "aclScope" : "ACCESS",
       "aclList" : [ "ALL" ]
     }, {
       "type" : "GROUP",
       "name" : "hadoop",
       "aclScope" : "ACCESS",
       "aclList" : [ "ALL" ]
     } ],
     "refCount" : 0
   }
   
   > docker-compose exec om ozone fs -ls /vol1
   Found 2 items
   drwxrwxrwx   - testuser  testuser           0 2023-12-18 16:57 /vol1/bucket1
   drwxrwxrwx   - testuser2 testuser2          0 2023-12-18 16:57 /vol1/bucket2
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1865152775

   Thanks for the fix @xBis7. Thanks for the review @SaketaChalamchala 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1865105198

   Thanks for woking on this fix @xBis7. LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1865156724

   Thanks @aswinshakil @SaketaChalamchala for the review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil merged PR #5816:
URL: https://github.com/apache/ozone/pull/5816


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil commented on code in PR #5816:
URL: https://github.com/apache/ozone/pull/5816#discussion_r1430738541


##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java:
##########
@@ -790,15 +790,14 @@ private List<FileStatusAdapter> listStatusVolume(String volumeStr,
     OFSPath ofsStartPath = new OFSPath(startPath, config);
     // list buckets in the volume
     OzoneVolume volume = objectStore.getVolume(volumeStr);
-    UserGroupInformation ugi =
-        UserGroupInformation.createRemoteUser(volume.getOwner());
-    String owner = ugi.getShortUserName();
-    String group = getGroupName(ugi);
     Iterator<? extends OzoneBucket> iter =
         volume.listBuckets(null, ofsStartPath.getBucketName());
     List<FileStatusAdapter> res = new ArrayList<>();
     while (iter.hasNext() && res.size() < numEntries) {
       OzoneBucket bucket = iter.next();
+      UserGroupInformation ugi = UserGroupInformation.createRemoteUser(bucket.getOwner());
+      String owner = ugi.getShortUserName();
+      String group = getGroupName(ugi);
       res.add(getFileStatusAdapterForBucket(bucket, uri, owner, group));

Review Comment:
   Can we just move it inside `getFileStatusAdapterForBucket` so that the function takes care of setting owner and group. Similar to `getFileStatusAdapterForVolume`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #5816:
URL: https://github.com/apache/ozone/pull/5816#discussion_r1431320566


##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java:
##########
@@ -790,15 +790,14 @@ private List<FileStatusAdapter> listStatusVolume(String volumeStr,
     OFSPath ofsStartPath = new OFSPath(startPath, config);
     // list buckets in the volume
     OzoneVolume volume = objectStore.getVolume(volumeStr);
-    UserGroupInformation ugi =
-        UserGroupInformation.createRemoteUser(volume.getOwner());
-    String owner = ugi.getShortUserName();
-    String group = getGroupName(ugi);
     Iterator<? extends OzoneBucket> iter =
         volume.listBuckets(null, ofsStartPath.getBucketName());
     List<FileStatusAdapter> res = new ArrayList<>();
     while (iter.hasNext() && res.size() < numEntries) {
       OzoneBucket bucket = iter.next();
+      UserGroupInformation ugi = UserGroupInformation.createRemoteUser(bucket.getOwner());
+      String owner = ugi.getShortUserName();
+      String group = getGroupName(ugi);
       res.add(getFileStatusAdapterForBucket(bucket, uri, owner, group));

Review Comment:
   @aswinshakil Nice suggestion. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #5816:
URL: https://github.com/apache/ozone/pull/5816#discussion_r1431320566


##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java:
##########
@@ -790,15 +790,14 @@ private List<FileStatusAdapter> listStatusVolume(String volumeStr,
     OFSPath ofsStartPath = new OFSPath(startPath, config);
     // list buckets in the volume
     OzoneVolume volume = objectStore.getVolume(volumeStr);
-    UserGroupInformation ugi =
-        UserGroupInformation.createRemoteUser(volume.getOwner());
-    String owner = ugi.getShortUserName();
-    String group = getGroupName(ugi);
     Iterator<? extends OzoneBucket> iter =
         volume.listBuckets(null, ofsStartPath.getBucketName());
     List<FileStatusAdapter> res = new ArrayList<>();
     while (iter.hasNext() && res.size() < numEntries) {
       OzoneBucket bucket = iter.next();
+      UserGroupInformation ugi = UserGroupInformation.createRemoteUser(bucket.getOwner());
+      String owner = ugi.getShortUserName();
+      String group = getGroupName(ugi);
       res.add(getFileStatusAdapterForBucket(bucket, uri, owner, group));

Review Comment:
   @aswinshakil Nice suggestion. I made the change. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]

Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1861078698

   cc @SaketaChalamchala 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org