[jira] [Created] (HIVE-28545) Remove global lock on HMSHandler.getMSForConf which is prone to deadlock

["Zhihua Deng (Jira)" <ji...@apache.org>]

Zhihua Deng created HIVE-28545:
----------------------------------

             Summary: Remove global lock on HMSHandler.getMSForConf which is prone to deadlock
                 Key: HIVE-28545
                 URL: https://issues.apache.org/jira/browse/HIVE-28545
             Project: Hive
          Issue Type: Bug
      Security Level: Public (Viewable by anyone)
          Components: Standalone Metastore
            Reporter: Zhihua Deng
            Assignee: Zhihua Deng


We add the global lock on HMSHandler.getMSForConf by HIVE-27117, which is aim to address the flaky test: TestCompactionMetrics.testInitiatorFailuresCountedCorrectly, however this global lock could contribute to another deadlock problem. Imaging this, if there is no idle connection in the connection pool, HMSHandler needs getMS to commit or. rollback the transaction, a new client chimes in and takes over this global lock to initialize the ObjectStore, the initialization will be pending as there is no available connection, which prevents others to get the global lock to free the occupied connection, I wrote a test to demonstrate this case.

This global lock could result to performance penalty which can block other clients in concurrent workloads. In worst cases, it can make other clients read timeout.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)