You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "xBis7 (via GitHub)" <gi...@apache.org> on 2023/12/18 16:58:22 UTC
[PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
xBis7 opened a new pull request, #5816:
URL: https://github.com/apache/ozone/pull/5816
## What changes were proposed in this pull request?
When doing a `listStatus` on a volume, we were setting the volume owner as the resource owner. Therefore, `ozone fs -ls` was resulting in displaying the volume owner as the owner of every bucket.
Check the jira description for a test-case for manually reproducing it.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-9950
## How was this patch tested?
A new test was added under `TestRootedOzoneFileSystem`. The patch was also tested manually in ozone docker dev env like so
```
> pwd
/projects/ozone/hadoop-ozone/dist/target/ozone-1.5.0-SNAPSHOT/compose/ozone
// Start docker env
> docker-compose up --scale datanode=5 -d
// Create volume with the default user, hadoop
> docker-compose exec om ozone sh volume create /vol1
// Create bucket and set the owner as testuser
> docker-compose exec om ozone sh bucket create /vol1/bucket1 --user testuser
// Create bucket and set the owner as testuser2
> docker-compose exec om ozone sh bucket create /vol1/bucket2 --user testuser2
> docker-compose exec om ozone fs -ls /vol1
Found 2 items
drwxrwxrwx - testuser testuser 0 2023-12-18 16:57 /vol1/bucket1
drwxrwxrwx - testuser2 testuser2 0 2023-12-18 16:57 /vol1/bucket2
> docker-compose exec om ozone sh volume update --user testuser /vol1
{
"metadata" : { },
"name" : "vol1",
"admin" : "hadoop",
"owner" : "testuser",
"quotaInBytes" : -1,
"quotaInNamespace" : -1,
"usedNamespace" : 2,
"creationTime" : "2023-12-18T16:57:07.500Z",
"modificationTime" : "2023-12-18T16:57:40.318Z",
"acls" : [ {
"type" : "USER",
"name" : "hadoop",
"aclScope" : "ACCESS",
"aclList" : [ "ALL" ]
}, {
"type" : "GROUP",
"name" : "hadoop",
"aclScope" : "ACCESS",
"aclList" : [ "ALL" ]
} ],
"refCount" : 0
}
> docker-compose exec om ozone fs -ls /vol1
Found 2 items
drwxrwxrwx - testuser testuser 0 2023-12-18 16:57 /vol1/bucket1
drwxrwxrwx - testuser2 testuser2 0 2023-12-18 16:57 /vol1/bucket2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1865152775
Thanks for the fix @xBis7. Thanks for the review @SaketaChalamchala
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1865105198
Thanks for woking on this fix @xBis7. LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1865156724
Thanks @aswinshakil @SaketaChalamchala for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil merged PR #5816:
URL: https://github.com/apache/ozone/pull/5816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "aswinshakil (via GitHub)" <gi...@apache.org>.
aswinshakil commented on code in PR #5816:
URL: https://github.com/apache/ozone/pull/5816#discussion_r1430738541
##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java:
##########
@@ -790,15 +790,14 @@ private List<FileStatusAdapter> listStatusVolume(String volumeStr,
OFSPath ofsStartPath = new OFSPath(startPath, config);
// list buckets in the volume
OzoneVolume volume = objectStore.getVolume(volumeStr);
- UserGroupInformation ugi =
- UserGroupInformation.createRemoteUser(volume.getOwner());
- String owner = ugi.getShortUserName();
- String group = getGroupName(ugi);
Iterator<? extends OzoneBucket> iter =
volume.listBuckets(null, ofsStartPath.getBucketName());
List<FileStatusAdapter> res = new ArrayList<>();
while (iter.hasNext() && res.size() < numEntries) {
OzoneBucket bucket = iter.next();
+ UserGroupInformation ugi = UserGroupInformation.createRemoteUser(bucket.getOwner());
+ String owner = ugi.getShortUserName();
+ String group = getGroupName(ugi);
res.add(getFileStatusAdapterForBucket(bucket, uri, owner, group));
Review Comment:
Can we just move it inside `getFileStatusAdapterForBucket` so that the function takes care of setting owner and group. Similar to `getFileStatusAdapterForVolume`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #5816:
URL: https://github.com/apache/ozone/pull/5816#discussion_r1431320566
##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java:
##########
@@ -790,15 +790,14 @@ private List<FileStatusAdapter> listStatusVolume(String volumeStr,
OFSPath ofsStartPath = new OFSPath(startPath, config);
// list buckets in the volume
OzoneVolume volume = objectStore.getVolume(volumeStr);
- UserGroupInformation ugi =
- UserGroupInformation.createRemoteUser(volume.getOwner());
- String owner = ugi.getShortUserName();
- String group = getGroupName(ugi);
Iterator<? extends OzoneBucket> iter =
volume.listBuckets(null, ofsStartPath.getBucketName());
List<FileStatusAdapter> res = new ArrayList<>();
while (iter.hasNext() && res.size() < numEntries) {
OzoneBucket bucket = iter.next();
+ UserGroupInformation ugi = UserGroupInformation.createRemoteUser(bucket.getOwner());
+ String owner = ugi.getShortUserName();
+ String group = getGroupName(ugi);
res.add(getFileStatusAdapterForBucket(bucket, uri, owner, group));
Review Comment:
@aswinshakil Nice suggestion. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #5816:
URL: https://github.com/apache/ozone/pull/5816#discussion_r1431320566
##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java:
##########
@@ -790,15 +790,14 @@ private List<FileStatusAdapter> listStatusVolume(String volumeStr,
OFSPath ofsStartPath = new OFSPath(startPath, config);
// list buckets in the volume
OzoneVolume volume = objectStore.getVolume(volumeStr);
- UserGroupInformation ugi =
- UserGroupInformation.createRemoteUser(volume.getOwner());
- String owner = ugi.getShortUserName();
- String group = getGroupName(ugi);
Iterator<? extends OzoneBucket> iter =
volume.listBuckets(null, ofsStartPath.getBucketName());
List<FileStatusAdapter> res = new ArrayList<>();
while (iter.hasNext() && res.size() < numEntries) {
OzoneBucket bucket = iter.next();
+ UserGroupInformation ugi = UserGroupInformation.createRemoteUser(bucket.getOwner());
+ String owner = ugi.getShortUserName();
+ String group = getGroupName(ugi);
res.add(getFileStatusAdapterForBucket(bucket, uri, owner, group));
Review Comment:
@aswinshakil Nice suggestion. I made the change. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-9950. 'ozone fs -ls' on volume shows the volume owner as the bucket owner [ozone]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on PR #5816:
URL: https://github.com/apache/ozone/pull/5816#issuecomment-1861078698
cc @SaketaChalamchala
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org