You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "djordje-mijatovic (via GitHub)" <gi...@apache.org> on 2023/02/20 10:38:43 UTC
[GitHub] [ozone] djordje-mijatovic opened a new pull request, #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
djordje-mijatovic opened a new pull request, #4291:
URL: https://github.com/apache/ozone/pull/4291
## What changes were proposed in this pull request?
We have passed snapshot name via RequestContext to Ozone Ranger Plugin.
## What is the link to the Apache JIRA
[HDDS 6986](https://issues.apache.org/jira/browse/HDDS-6986)
## How was this patch tested?
Unit Tests and Manual dev test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1440746227
> Thanks @smengcl for the comments and questions.
>
> > A new snapshotName field alone may or may not suffice. Currently we only support the snapshot scope of a single bucket, but we would at least expand the snapshot scope to a volume.
>
> Snapshot path + snapshot name or UUID of snapshot should be a unique identifier.
>
> As an aside, this patch is for the Ranger authorizer for snapshot buckets that have an explicit policy defined for it (policy set by the admin for the particular snapshotpath/snapshot). If no explicit policy exists, then the "Active" bucket policy (current policy) is used for authorization checks. A couple of points that may help.
>
> i.) The patch is for tweaking the Ranger to use an explicit policy defined for the snapshot if it exists. Originally, I believe, we were going to use ranger to check if such an explicit policy exists for the snapshot, however it can be checked but I think it can't be applied for the access check - and so we were looking to make some changes for it.
>
> > It should be easy to add support for Ozone native ACL in OzoneNativeAuthorizer as well.
>
> ii.) Ozone native authorizer is unaffected. In the snapshot design Ozone native authoriizer checks should be unaffected as the acls from the snapshot are used for access control for snapshots (captured in snapshot).
>
> > Does this change imply we would have a new 4th field (e.g. Snapshot) other than what we currently have (volume, bucket and key) in the UI?
>
> Should be for the explicit snapshot policy defined for the snapshotpath/snapshot.
Make sense. Thanks @neils-dev .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #4291: HDDS-6986. Update ozone ranger plugin to handle snapshots
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1507423858
Thanks @djordje-mijatovic . Closing this one for now as Ranger folks would need to improve the query performance on their side in this specific use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] djordje-mijatovic commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "djordje-mijatovic (via GitHub)" <gi...@apache.org>.
djordje-mijatovic commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1440122468
Here you can fund how I did setup Ozone Ranger locally and how I have performed end to end snapshot auth test.
[Snapshot Instructions.pdf](https://github.com/apache/ozone/files/10804743/Snapshot.Instructions.pdf)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] prashantpogde commented on pull request #4291: HDDS-6986. Update ozone ranger plugin to handle snapshots
Posted by "prashantpogde (via GitHub)" <gi...@apache.org>.
prashantpogde commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1442225768
> Here you can find how I did setup Ozone Ranger locally and how I have performed end to end snapshot auth test.
> [Snapshot Instructions.pdf](https://github.com/apache/ozone/files/10804743/Snapshot.Instructions.pdf)
Please also attach this to corresponding Jira ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] prashantpogde commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "prashantpogde (via GitHub)" <gi...@apache.org>.
prashantpogde commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1438954853
This requires changes in Ozone Ranger plugin as well. What's the plan to get that done and raise a PR for that. I believe majority of the work lies there in this approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] djordje-mijatovic commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "djordje-mijatovic (via GitHub)" <gi...@apache.org>.
djordje-mijatovic commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1440417281
Here is the PR handling the changes on the Plugin side: https://github.com/apache/ranger/pull/218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1438742336
Thanks @djordje-mijatovic for the patch. I have several questions and comments:
1. Since the change is mostly done in `RequestContext`, it should be easy to add support for Ozone native ACL in `OzoneNativeAuthorizer` as well.
2. As the PR itself does not update `OzoneNativeAuthorizer` (in Ozone code base) or `RangerOzoneAuthorizer` (in Ranger code base), and serves to lays the groundwork for those upcoming authorizer changes, IMO it would be more accurate for the JIRA/PR title to be: `[Snapshot] Add snapshot field in Ozone ACL RequestContext`.
3. A new `snapshotName` field alone may or may not suffice. Currently we only support the snapshot scope of a single bucket, but we would at least expand the snapshot scope to a volume. I'd like to know the intended usage of this new ACL resource field. Could you give an example or two?
4. Does this change imply we would have a new 4th field (e.g. `Snapshot`) other than what we currently have (volume, bucket and key) in the UI? Thus some UI changes could be required on the Ranger side as well. For reference, currently the Ranger Ozone policy editor Web UI looks like this:
<img width="1200" alt="ranger-ozone-policy-ui" src="https://user-images.githubusercontent.com/50227127/220395512-199ff5d8-ed7e-41dd-8aa4-19f49a43f279.png">
cc @GeorgeJahad @neils-dev
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] neils-dev commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "neils-dev (via GitHub)" <gi...@apache.org>.
neils-dev commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1439153815
Thanks @smengcl for the comments and questions.
> A new snapshotName field alone may or may not suffice. Currently we only support the snapshot scope of a single bucket, but we would at least expand the snapshot scope to a volume.
Snapshot path + snapshot name or UUID of snapshot should be a unique identifier.
As an aside, this patch is for the Ranger authorizer for snapshot buckets that have an explicit policy defined for it (policy set by the admin for the particular snapshotpath/snapshot). If no explicit policy exists, then the "Active" bucket policy (current policy) is used for authorization checks. A couple of points that may help.
i.) The patch is for tweaking the Ranger to use an explicit policy defined for the snapshot if it exists. Originally, I believe, we were going to use ranger to check if such an explicit policy exists for the snapshot, however it can be checked but I think it can't be applied for the access check - and so we were looking to make some changes for it.
> It should be easy to add support for Ozone native ACL in OzoneNativeAuthorizer as well.
ii.) Ozone native authorizer is unaffected. In the snapshot design Ozone native authoriizer checks should be unaffected as the acls from the snapshot are used for access control for snapshots (captured in snapshot).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1440751259
> Here you can find how I did setup Ozone Ranger locally and how I have performed end to end snapshot auth test. [Snapshot Instructions.pdf](https://github.com/apache/ozone/files/10804743/Snapshot.Instructions.pdf)
Thanks for the doc @djordje-mijatovic .
> volume = somevolume
> bucket = somebucket/.snapshot/somesnapshot
> key = *
It looks a bit odd to me that the snapshot path is stitched after the bucket name. Is there a better way to handle this?
And does the following work the same?
> volume = somevolume
> bucket = somebucket
> key = .snapshot/somesnapshot/*
Also would you fix the checkstyle warning? Thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] djordje-mijatovic commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "djordje-mijatovic (via GitHub)" <gi...@apache.org>.
djordje-mijatovic commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1441843824
> And does the following work the same?
> ```
> volume = somevolume
> bucket = somebucket
> key = .snapshot/somesnapshot/*
> ```
Yes, this is doable. In s implemented now but it had to be done on the Ranger side. Check this PR: https://github.com/apache/ranger/pull/218
> Also would you fix the checkstyle warning? Thx.
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GeorgeJahad commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "GeorgeJahad (via GitHub)" <gi...@apache.org>.
GeorgeJahad commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1438919466
> Since the change is mostly done in RequestContext, it should be easy to add support for Ozone native ACL in OzoneNativeAuthorizer as well.
FYI @smengcl I believe the OzoneNativeAuthorizer in the master branch already works with snapshots. When I created the OmMetadataReader, I tried to implement it in such a way that ACL's would be read from the snapshot assigned to the OmMetadataReader.
I never wrote the tests for it because @prashantpogde asked me to put that work on hold.
But those tests are now being written by my team mate: @mladjan-gadzic
They appear to be all working, and he will submit the PR in the next few days.
Bottom line, I don't believe there are any more changes needed for the native authorizor, just some tests which will be submitted shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl closed pull request #4291: HDDS-6986. Update ozone ranger plugin to handle snapshots
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl closed pull request #4291: HDDS-6986. Update ozone ranger plugin to handle snapshots
URL: https://github.com/apache/ozone/pull/4291
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] GeorgeJahad commented on pull request #4291: HDDS 6986. Update ozone ranger plugin to handle snapshots
Posted by "GeorgeJahad (via GitHub)" <gi...@apache.org>.
GeorgeJahad commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1438956656
The Ranger PR is almost done. I'm hoping it will be ready for review tomorrow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] prashantpogde commented on pull request #4291: HDDS-6986. Update ozone ranger plugin to handle snapshots
Posted by "prashantpogde (via GitHub)" <gi...@apache.org>.
prashantpogde commented on PR #4291:
URL: https://github.com/apache/ozone/pull/4291#issuecomment-1442235064
> 3. A new `snapshotName` field alone may or may not suffice. Currently we only support the snapshot scope of a single bucket, but we would at least expand the snapshot scope to a volume. I'd like to know the intended usage of this new ACL resource field. Could you give an example or two?
Good point. We do not intend to support nested snapshots in Ozone. Therefore snapshot scope can be either a bucket or a volume and just one snapshot field would suffice. However we need another parameter to indicate whether the ".snapshot" path prefix should be applied at volume-level or bucket-level. It would also be good to do it such that it leaves room for supporting snapshots for key-prefixes in future without having to make changes in Ranger.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org