You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kyuubi.apache.org by GitBox <gi...@apache.org> on 2021/11/24 23:28:27 UTC
[GitHub] [incubator-kyuubi] anumee opened a new issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
anumee opened a new issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the feature
Spark security module was retired from [submarine pull- 796](https://github.com/apache/submarine/pull/796)
and it is still missing here. This is needed with Spark 3.2 support.
### Motivation
Spark 3.2 support
### Describe the solution
1. Migrate Submarine security module here - https://github.com/apache/submarine/pull/796 and,
2. Update Spark version to 3.2
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004590290
Ok @pan3793, I am not aware of kyuubi code base.. can you please guide me in porting submarine spark security module into kyuubi ? I tried searching for spark security files in the codebase..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] izhangzhihao commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
izhangzhihao commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004594212
You can find code [here](https://github.com/apache/submarine/tree/40904405f03f944d7a83da9d059ee1770e0085bf/submarine-security/spark-security)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004600714
cc @yaooqinn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] izhangzhihao commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
izhangzhihao commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004593801
Code gets dropped in commit https://github.com/apache/submarine/commit/50ae48339df4a11d115ac4b76b363da9796a3260
previews commit: https://github.com/apache/submarine/commit/40904405f03f944d7a83da9d059ee1770e0085bf
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-993322663
This task is of medium priority and we don't have ETA now, volunteers are welcome :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka edited a comment on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka edited a comment on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004660487
> You can find code here
Please note that, I have taken the code from submarine project.. But I couldn't find the code of spark-security inside Kyuubi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] minyk commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
minyk commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004595939
@RamakrishnaChilaka I've done almost same thing for our product. I don't have the right to open the code, but this is commit list I made:
* parseRawDataType() removed by [SPARK-33641]
* child renamed to plan by [SPARK-34701]
* AlterTableRecoverPartitionsCommand renamed to RepairTableCommand by [SPARK-34518]
* CacheTableCommand renamed to CacheTalbe by [SPARK-33654]
* Change RunnableCommand to LeafRunnableCommand by [SPARK-34989]
* implements a withNewChildInteral method created by [SPARK-34989]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004692723
The spark security module currently contains 4 main features, including authorization, data masking, row-level filtering, and DCL(incomplete).
It contains a lot of legacy code for very old spark releases including spark 2.0, so I suggest we add them step by step starting with authorization.
FYI, https://issues.apache.org/jira/browse/SUBMARINE-409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004779477
Sure @yaooqinn. We will get started on this, as i am new to kyuubi, kindly, please guide me.. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn closed issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
yaooqinn closed issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004660487
> here
Please note that, I have taken the code from submarine project.. But I couldn't find the code of spark-security inside Kyuubi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004570623
@pan3793 ,can you please check this PR ? https://github.com/apache/submarine/pull/861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004586716
Thanks @RamakrishnaChilaka for working on this feature, but ranger spark module is deprecated since https://github.com/apache/submarine/pull/796, the test infrastructure become not runable in apache/submarine, would you please port this module to kyuubi firstly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] lordk911 commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
lordk911 commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-993066358
Wonder to know when will this Spark security module to support Spark 3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004659866
thanks @minyk , your list was truly helpful, I didn't knew about `SPARK-34989`, I have incorporated it, one small change to your list is `UnCacheTableCommand` is renamed to `UnCacheTable`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] izhangzhihao removed a comment on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
izhangzhihao removed a comment on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004594212
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] lordk911 commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
lordk911 commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1073719510
@yaooqinn will [[WIP] [KYUUBI #1451] Introduce Kyuubi Spark Authz Module #2160](https://github.com/apache/incubator-kyuubi/pull/2160) support ranger1.2 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka edited a comment on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka edited a comment on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004570623
@pan3793 ,can you please check this PR ? https://github.com/apache/submarine/pull/861, This is a rough PR, After your review i am willing to change this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004588258
sure @pan3793 , Kindly, Please check whether the current code is valid for spark 3.2 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] pan3793 commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004589180
> Please check whether the current code is valid for Spark 3.2?
We need CI to verify that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] RamakrishnaChilaka removed a comment on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka removed a comment on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1004779477
Sure @yaooqinn. We will get started on this, as i am new to kyuubi, kindly, please guide me.. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #1451: [FEATURE] Migrate Spark security module from submarine to kyuubi
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #1451:
URL: https://github.com/apache/incubator-kyuubi/issues/1451#issuecomment-1073734786
> @yaooqinn will [[WIP] [KYUUBI #1451] Introduce Kyuubi Spark Authz Module #2160](https://github.com/apache/incubator-kyuubi/pull/2160) support ranger1.2 ?
I don't know
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org