You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2022/06/23 05:28:00 UTC
[jira] [Resolved] (SPARK-39547) V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata
[ https://issues.apache.org/jira/browse/SPARK-39547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan resolved SPARK-39547.
---------------------------------
Fix Version/s: 3.4.0
3.3.1
Resolution: Fixed
> V2SessionCatalog should not throw NoSuchDatabaseException in loadNamespaceMetadata
> ----------------------------------------------------------------------------------
>
> Key: SPARK-39547
> URL: https://issues.apache.org/jira/browse/SPARK-39547
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Prashant Singh
> Priority: Minor
> Fix For: 3.4.0, 3.3.1
>
>
> DROP NAMESPACE IF EXISTS
> {table}
>
> if a catalog doesn't overrides `namespaceExists` it by default uses `loadNamespaceMetadata` and in case a `db` not exists loadNamespaceMetadata throws a `NoSuchDatabaseException` which is not catched and we see failures even with `if exists` clause. One such use case we observed was in iceberg table a post test clean up was failing with `NoSuchDatabaseException` now.
>
> Found {color:#000000}V2SessionCatalog `{color}{color:#00627a}loadNamespaceMetadata{color}{color:#000000}` {color}was also throwing the same unlike `{color:#000000}JDBCTableCatalog`{color}
> ref a stack trace :
> {quote}Database 'db' not found
> org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'db' not found
> at org.apache.spark.sql.catalyst.catalog.SessionCatalog.requireDbExists(SessionCatalog.scala:219)
> at org.apache.spark.sql.catalyst.catalog.SessionCatalog.getDatabaseMetadata(SessionCatalog.scala:284)
> at org.apache.spark.sql.execution.datasources.v2.V2SessionCatalog.loadNamespaceMetadata(V2SessionCatalog.scala:247)
> at org.apache.iceberg.spark.SparkSessionCatalog.loadNamespaceMetadata(SparkSessionCatalog.java:97)
> at org.apache.spark.sql.connector.catalog.SupportsNamespaces.namespaceExists(SupportsNamespaces.java:98)
> at org.apache.spark.sql.execution.datasources.v2.DropNamespaceExec.run(DropNamespaceExec.scala:40)
> at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:43)
> at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:43)
> at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:49)
> at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:98)
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org