You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/04/22 15:13:28 UTC
[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #857: HDDS-3385. Simplify S3 -> Ozone volume mapping
adoroszlai opened a new pull request #857:
URL: https://github.com/apache/hadoop-ozone/pull/857
## What changes were proposed in this pull request?
Replace user-specific volume mapping with fix `/s3v` volume for S3 buckets. Unfortunately `s3` is not a valid volume name, at least 3-chars length is currently required.
* Remove server-side bucket mapping
* Remove `ozone s3 path` command
* Verify volume name and bucket name separately, to be able to throw `OMException` with the correct result code instead of `IllegalArgumentException`
Originally I intended to handle on-demand `/s3v` volume creation in Ozone client, as it's used by both S3 Gateway and CSI service. But volumes can only be created by administrators, while buckets can be created by others, too. Previously this was circumvented at the Ozone Manager server side in the S3-specific bucket creation handler. So we can either ask Ozone users to create the volume if they want to use S3 Gateway, or we have to keep the S3-specific OM API. For this patch I chose the first approach.
Also:
* add smoke test for deleting buckets
* check location in bucket create response
* fix a few typos / leftover items in the doc
https://issues.apache.org/jira/browse/HDDS-3385
## How was this patch tested?
https://github.com/adoroszlai/hadoop-ozone/runs/608414085
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on pull request #857: HDDS-3385. Simplify S3 -> Ozone volume mapping
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on pull request #857:
URL: https://github.com/apache/hadoop-ozone/pull/857#issuecomment-621985964
Sorry for the late review. Since it is marked for draft until 2 days back, not looked into it.
1. Previously listBuckets owned by an authenticated user. Now we list all buckets in /s3v will be shown (Meaning buckets which are created by other user also will be shown now). This is a behavior change from previous release, is the semantics of this API have changed with this.
2. And also this is a breaking change from the previous release, where old buckets will not be read by new S3G. (Do we want to support older release 0.5 beta or break the compatibility)
cc @arp7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #857: HDDS-3385. Simplify S3 -> Ozone volume mapping
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #857:
URL: https://github.com/apache/hadoop-ozone/pull/857#issuecomment-630116041
> So, now is there any plan to support the buckets which are created using AWS Accesskey id for volumes, until the mount is supported, or is there any plan to complete the mount before the next release?
Yes, I think it can be completed before the next release.
> is there any ACL changes
after one volume one s3 bucket changed to one volume, multiple user's
bucket?
Only the mapping is changed, I assume all the existing bucket level permission is the same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #857: HDDS-3385. Simplify S3 -> Ozone volume mapping
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on pull request #857:
URL: https://github.com/apache/hadoop-ozone/pull/857#issuecomment-621985964
Sorry for the late review. Since it is marked for draft until 2 days back, not looked into it.
Few comments during initial review of the PR.
1. Previously listBuckets owned by an authenticated user. Now we list all buckets in /s3v will be shown (Meaning buckets which are created by other user also will be shown now). This is a behavior change from previous release, is the semantics of this API have changed with this.
2. And also this is a breaking change from the previous release, where old buckets will not be read by new S3G. (Do we want to support older release 0.5 beta or break the compatibility)
cc @arp7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on pull request #857: HDDS-3385. Simplify S3 -> Ozone volume mapping
Posted by GitBox <gi...@apache.org>.
elek commented on pull request #857:
URL: https://github.com/apache/hadoop-ozone/pull/857#issuecomment-624482891
> And also this is a breaking change from the previous release, where old buckets will not be read by new S3G. (Do we want to support older release 0.5 beta or break the compatibility)
There was no promise about backward compatibility. We try to do our best to make storage scheme compatible to support early adopters.
In this specific case, I agree, this is an incompatible change (especially until 'bind mount' of volumes are not implemented).
That's why a warning was sent to the mailing list:
https://lists.apache.org/thread.html/r647c7564e6f60e065d0f45cf4bc34cbb9a14a94bc4389daf9577d27a%40%3Cozone-dev.hadoop.apache.org%3E
> !!! INCOMPATIBLE CHANGE ALERT !!!!!
>
> When 1 will be implemented, but 2, not yet. For a limited time of period
> we will share buckets only from one volume as s3 buckets. This is
> different from the current implementation when you can use s3buckets
> from multiple volumes.
>
> If it's a blocker for you, please share your opinion and we can schedule
> the implementation according to the feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on pull request #857: HDDS-3385. Simplify S3 -> Ozone volume mapping
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on pull request #857:
URL: https://github.com/apache/hadoop-ozone/pull/857#issuecomment-621758109
Thanks @elek for reviewing and committing it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on pull request #857: HDDS-3385. Simplify S3 -> Ozone volume mapping
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on pull request #857:
URL: https://github.com/apache/hadoop-ozone/pull/857#issuecomment-621985964
Sorry for the late review. Since it is marked as draft until 2 days back, not looked into it.
Few comments during initial review of the PR.
1. Previously listBuckets owned by an authenticated user. Now we list all buckets in /s3v will be shown (Meaning buckets which are created by other user also will be shown now). This is a behavior change from previous release, is the semantics of this API have changed with this.
2. And also this is a breaking change from the previous release, where old buckets will not be read by new S3G. (Do we want to support older release 0.5 beta or break the compatibility)
cc @arp7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org