[jira] [Created] (SPARK-34800) Use fine-grained lock in SessionCatalog.tableExists

["rongchuan.jin (Jira)" <ji...@apache.org>]

rongchuan.jin created SPARK-34800:
-------------------------------------

             Summary: Use fine-grained lock in SessionCatalog.tableExists
                 Key: SPARK-34800
                 URL: https://issues.apache.org/jira/browse/SPARK-34800
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.1.1
            Reporter: rongchuan.jin


We have modified the underlying hive meta store which a different hive > database is placed in its own shard for performance. However, we found that > the synchronized limits the concurrency, we would like to fix it.

Related jstack trace like following:
{code:java}
"http-nio-7070-exec-257" #19961734 daemon prio=5 os_prio=0 tid=0x00007f45f4ce1000 nid=0x1a85e6 waiting for monitor entry [0x00007f45949df000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.tableExists(SessionCatalog.scala:415)
    - waiting to lock <0x000000011d983d90> (a org.apache.spark.sql.catalyst.catalog.SessionCatalog)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.requireTableExists(SessionCatalog.scala:185)
    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.getTableMetadata(SessionCatalog.scala:430)
    at org.apache.spark.sql.DdlOperation$.getTableDesc(SourceFile:123)
    at org.apache.spark.sql.DdlOperation.getTableDesc(SourceFile)
...
{code}
we fixed as discussed in mail [http://mail-archives.apache.org/mod_mbox/spark-dev/202103.mbox/browser]

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org