You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by GitBox <gi...@apache.org> on 2020/03/09 08:16:38 UTC
[GitHub] [submarine] yaooqinn opened a new pull request #211: SUBMARINE-411.
Basic SQL Standard Authorization ACL Management for Spark
yaooqinn opened a new pull request #211: SUBMARINE-411. Basic SQL Standard Authorization ACL Management for Spark
URL: https://github.com/apache/submarine/pull/211
### What is this PR for?
This pr aims to provide submarine spark jobs with ACL Management abilities.
In this pr, we create
1) a plugin that can talk to Apache Ranger Admin directly for privilege policies
2) a Spark SQL optimizer rule that can be injected to spark jobs with the flag - spark.sql.extensions
3) the privilege check happens in the Spark SQL Optimizer meets a resource e.g. a database, table, etc. If the user has the privilege for the resource he or she accesses, we return the original spark plan and let it continue, otherwise, an exception will be raised
The previous work can be found, this pr bring the ACL only and will do others in follow-ups
- https://github.com/yaooqinn/spark-authorizer
- https://github.com/yaooqinn/spark-ranger
### What type of PR is it?
feature
### Todos
* [ ] - Show Databases/Tables with filtered objects
* [ ] - Row-level filtering
* [ ] - Datamasking filtering
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-411
### How should this be tested?
* First time? Setup Travis CI as described on https://submarine.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
Add Unit test and enable Travis jobs
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, will do after finish all todos
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org
[GitHub] [submarine] asfgit closed pull request #211: SUBMARINE-411. Basic
SQL Standard Authorization ACL Management for Spark
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #211: SUBMARINE-411. Basic SQL Standard Authorization ACL Management for Spark
URL: https://github.com/apache/submarine/pull/211
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org
[GitHub] [submarine] yaooqinn commented on issue #211: SUBMARINE-411. Basic
SQL Standard Authorization ACL Management for Spark
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #211: SUBMARINE-411. Basic SQL Standard Authorization ACL Management for Spark
URL: https://github.com/apache/submarine/pull/211#issuecomment-596389285
cc @liuxunorg @yuanzac, thanks for reviewing in advance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org
[GitHub] [submarine] yaooqinn commented on issue #211: SUBMARINE-411. Basic
SQL Standard Authorization ACL Management for Spark
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #211: SUBMARINE-411. Basic SQL Standard Authorization ACL Management for Spark
URL: https://github.com/apache/submarine/pull/211#issuecomment-596408599
notice that submarine supports Hadoop 3.1/3.2, which are not supported by spark 2.3/2.4. so I excluded them in Travis Hadoop 3.x related job in just the test phase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org
[GitHub] [submarine] yaooqinn commented on issue #211: SUBMARINE-411. Basic
SQL Standard Authorization ACL Management for Spark
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #211: SUBMARINE-411. Basic SQL Standard Authorization ACL Management for Spark
URL: https://github.com/apache/submarine/pull/211#issuecomment-596403824
thanks @liuxunorg
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org
[GitHub] [submarine] liuxunorg commented on issue #211: SUBMARINE-411. Basic
SQL Standard Authorization ACL Management for Spark
Posted by GitBox <gi...@apache.org>.
liuxunorg commented on issue #211: SUBMARINE-411. Basic SQL Standard Authorization ACL Management for Spark
URL: https://github.com/apache/submarine/pull/211#issuecomment-596415134
Because there are several PRs to submit after this feature, I will merge this PR as soon as possible.
Will merge if no more comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org