You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Zhihua Deng (Jira)" <ji...@apache.org> on 2022/09/01 06:35:00 UTC

[jira] [Created] (HIVE-26509) Introduce dynamic leader election in HMS

Zhihua Deng created HIVE-26509:
----------------------------------

             Summary: Introduce dynamic leader election in HMS
                 Key: HIVE-26509
                 URL: https://issues.apache.org/jira/browse/HIVE-26509
             Project: Hive
          Issue Type: New Feature
          Components: Standalone Metastore
            Reporter: Zhihua Deng


From HIVE-21841 we have a leader HMS selected by configuring metastore.housekeeping.leader.hostname on startup. This approach saves us from running duplicated HMS's housekeeping tasks cluster-wide. 

In this jira, we introduce another dynamic leader election: adopt hive lock to implement the leader election. Once a HMS owns the lock, then it becomes the leader, carries out the housekeeping tasks, and sends heartbeats to renew the lock before timeout. If the leader fails to reclaim the lock, then stops the already started tasks if it has, the electing event is audited. We can achieve a more dynamic leader when the original goes down or in the public cloud without well configured property, and reduce the leader’s burdens by running these tasks among different leaders.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)