[jira] [Commented] (SPARK-29078) Spark shell fails if read permission is not granted to hive warehouse directory

["Peter Toth (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-29078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943095#comment-16943095 ] 

Peter Toth commented on SPARK-29078:
------------------------------------

[~misutoth], if we look closer at the stacktrace ({{at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)}}) it shows that the AccessControlException issue is with the default database existence check (on master branch this corresponds to https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala#L139:

{noformat}
    // Create default database if it doesn't exist
    if (!externalCatalog.databaseExists(SessionCatalog.DEFAULT_DATABASE)) {
      // There may be another Spark application creating default database at the same time, here we
{noformat}

This exception is expected if the user doesn't have access to the directory of the default database. In that case the user can't use Spark SQL.

I would suggest closing this ticket.

> Spark shell fails if read permission is not granted to hive warehouse directory
> -------------------------------------------------------------------------------
>
>                 Key: SPARK-29078
>                 URL: https://issues.apache.org/jira/browse/SPARK-29078
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Mihaly Toth
>            Priority: Major
>
> Similarly to SPARK-20256, in {{SharedSessionState}} when {{GlobalTempViewManager}} is created, it is checked that there is no database exists that has the same name as of the global temp database (name is configurable with {{spark.sql.globalTempDatabase}}) , because that is a special database, which should not exist in the metastore. For this, a read permission is required on the warehouse directory at the moment, which on the other hand would allow listing all the databases of all users.
> When such a read access is not granted for security reasons, an access violation exception should be ignored upon such initial validation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org