You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by "cdmikechen (Jira)" <ji...@apache.org> on 2023/07/02 09:30:00 UTC
[jira] [Updated] (SUBMARINE-834) Ranger support for Spark Datasources
[ https://issues.apache.org/jira/browse/SUBMARINE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
cdmikechen updated SUBMARINE-834:
---------------------------------
Fix Version/s: (was: 0.8.0)
Target Version: 0.6.1 (was: 0.8.0)
> Ranger support for Spark Datasources
> ------------------------------------
>
> Key: SUBMARINE-834
> URL: https://issues.apache.org/jira/browse/SUBMARINE-834
> Project: Apache Submarine
> Issue Type: New Feature
> Components: Security
> Reporter: Adesh Kumar Rao
> Priority: Major
>
> Currently, the spark-security module is tightly coupled with "HiveTableRelation" and "MetastoreRelation" for authorization of non-transactional hive tables.
> This does not work for Hive transactional (ACID enabled) table. Since Hive has changed the way data/metadata is stored for transactional tables when compared to non-transactional tables. Therefore, Spark can not read Hive transactional tables directly.
> So even if security module may enforce security on such tables, spark can't actually read anything.
>
> Reading hive's transactional tables in spark, needs [Spark-Acid|https://github.com/qubole/spark-acid], implemented as Spark Datasource. Since security module is tightly coupled with "HiveTableRelation" and "MetastoreRelation", it does not provides authorization support for any datasource.
>
> The idea is to support Spark datasource authorization (which has db/table/column/partitions analogous to hive).
>
> We can create generic interfaces for datasource, which each datasource can implement and then it can be authorized using the existing codebase.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org