You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by GitBox <gi...@apache.org> on 2022/04/28 08:38:39 UTC
[GitHub] [incubator-kyuubi] minyk opened a new issue, #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
minyk opened a new issue, #2499:
URL: https://github.com/apache/incubator-kyuubi/issues/2499
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
We recently integrated Apache Iceberg into our product with Apache Kyuubi. We started from Ranger ACL plugin of Apache Submarine, but it has only DSv1 ACLs. So We add some DSv2 ACL checks to the module for Iceberg tables.
Below is the full list of added commands:
Spark DSv2 DDLs
- AddColumns, AlterColumn, DropColumns, RenameColumn, RenameTable,
- SetTableProperties, UnsetTableProperties
- CreateV2Table, DropTable, ReplaceTableAsSelect
- TruncateTable
Spark DSv2 DMLs
- AppendData
Iceberg commands: DDLs
- AddPartitionField/ DropPartitionField
- SetIdentifierFields/ DropIdentifierFields
- SetWriteDistributionAndOrdering
Iceberg commands: DMLs
- MergeIntoIcebergTable
- UpdateIcebergTable
- DeleteFromIcebergTable
We uses Kyuubi already, and now Kyuubi has it's own Authz module. So we're willing to contribute back to Kyuubi.
### How should we improve?
`PrivilegesBuilder.buildCommand` has almost all DSv1 and some DSv2 commands. Create `PrivilegesBuilder.buildV2Command` and `PrivilegesBuilder.buildIcebergCommand` for each purpose. Then call these functions from `PrivilegesBuilder.build` like:
```scala
def build(plan: LogicalPlan): (Seq[PrivilegeObject], Seq[PrivilegeObject]) = {
val inputObjs = new ArrayBuffer[PrivilegeObject]
val outputObjs = new ArrayBuffer[PrivilegeObject]
plan match {
// RunnableCommand
case cmd: Command =>
buildCommand(cmd, inputObjs, outputObjs)
buildV2Command(cmd, inputObjs, outputObjs)
buildIcebergCommand(cmd, inputObjs, outputObjs)
// Queries
case _ => buildQuery(plan, inputObjs)
}
(inputObjs, outputObjs)
}
```
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] yaooqinn commented on issue #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #2499:
URL: https://github.com/apache/kyuubi/issues/2499#issuecomment-1376855962
thanks @bowenliang123
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] yaooqinn closed issue #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
Posted by GitBox <gi...@apache.org>.
yaooqinn closed issue #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
URL: https://github.com/apache/kyuubi/issues/2499
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #2499:
URL: https://github.com/apache/incubator-kyuubi/issues/2499#issuecomment-1111932091
thanks @minyk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] yaooqinn commented on issue #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on issue #2499:
URL: https://github.com/apache/incubator-kyuubi/issues/2499#issuecomment-1118103283
feel free to send a WIP PR @minyk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [incubator-kyuubi] minyk commented on issue #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
Posted by GitBox <gi...@apache.org>.
minyk commented on issue #2499:
URL: https://github.com/apache/incubator-kyuubi/issues/2499#issuecomment-1115718633
@yaooqinn this is getting bigger than I thought. Currently, initial translation to kyuubi was done and start to write unit test.
May I send Working-In-Progress PR for discussion? Any restriction for WIP PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
[GitHub] [kyuubi] bowenliang123 commented on issue #2499: [Improvement] Add Iceberg (and minimal Spark DSv2) commands to Authz module
Posted by GitBox <gi...@apache.org>.
bowenliang123 commented on issue #2499:
URL: https://github.com/apache/kyuubi/issues/2499#issuecomment-1376849782
Both V2 commands/plans and Iceberg commands are covered in the current Authz implementation. This issue should be good to close.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org