[jira] [Created] (SPARK-34616) SessionCatalog#requireDbExists is called too many times affects SQL performance

["Zhang Jianguo (Jira)" <ji...@apache.org>]

Zhang Jianguo created SPARK-34616:
-------------------------------------

             Summary: SessionCatalog#requireDbExists is called too many times affects SQL performance
                 Key: SPARK-34616
                 URL: https://issues.apache.org/jira/browse/SPARK-34616
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.0.0, 2.4.5
            Reporter: Zhang Jianguo
         Attachments: db_existance.PNG

Almost all SQL need to check if database exist before execution.

Every check needs to connect to Metastore and check if the DB exists in Databse. This behavior could be too expensive.

 

A proposal is to add a switch to control if it's necessary to check DB's existance.

 

[requireDbExists]([https://github.com/apache/spark/blob/53e4dba7c489ac5c0ad61f0121c4e247de5b485c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala#L197)]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org