[jira] [Updated] (HIVE-28545) Remove global lock on HMSHandler.getMSForConf which is prone to deadlock

["Naveen Gangam (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-28545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Naveen Gangam updated HIVE-28545:
---------------------------------
    External issue URL: https://github.com/apache/hive/pull/5477

> Remove global lock on HMSHandler.getMSForConf which is prone to deadlock
> ------------------------------------------------------------------------
>
>                 Key: HIVE-28545
>                 URL: https://issues.apache.org/jira/browse/HIVE-28545
>             Project: Hive
>          Issue Type: Bug
>      Security Level: Public(Viewable by anyone) 
>          Components: Standalone Metastore
>            Reporter: Zhihua Deng
>            Assignee: Zhihua Deng
>            Priority: Major
>             Fix For: 4.1.0
>
>
> We add the global lock on HMSHandler.getMSForConf by HIVE-27117, which is aim to address the flaky test: TestCompactionMetrics.testInitiatorFailuresCountedCorrectly, however this global lock could contribute to another deadlock problem. Imaging this, if there is no idle connection in the connection pool, HMSHandler needs getMS to commit or. rollback the transaction, a new client chimes in and takes over this global lock to initialize the ObjectStore, the initialization will be pending as there is no available connection, which prevents others to get the global lock to free the occupied connection, I wrote a test to demonstrate this case.
> This global lock could result to performance penalty which can block other clients in concurrent workloads. In worst cases, it can make other clients read timeout.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)