[jira] [Created] (IMPALA-9405) Improvements for Frontend#metaStoreClientPool_

["Sahil Takiar (Jira)" <ji...@apache.org>]

Sahil Takiar created IMPALA-9405:
------------------------------------

             Summary: Improvements for Frontend#metaStoreClientPool_
                 Key: IMPALA-9405
                 URL: https://issues.apache.org/jira/browse/IMPALA-9405
             Project: IMPALA
          Issue Type: Improvement
          Components: Frontend
            Reporter: Sahil Takiar


While trying to resurrect {{tests/experiments/test_catalog_hms_failures.py}} I noticed the test {{TestCatalogHMSFailures::test_start_catalog_before_hms}} has started to fail. The reason is that when this test was written, only the catalogd was connecting to HMS, but with catalog v2 and ACID integration this is no longer the case.

It looks like catalog v2 honors {{initial_hms_cnxn_timeout_s}}, (at least {{DirectMetaProvider}} honors the flag, and I *think* that is part of the metadata v2 code), but the {{Frontend}} Java class has a member variable {{metaStoreClientPool_}} that does not use the flag. It looks like that pool was added for ACID integration.

The flag {{initial_hms_cnxn_timeout_s}} was added in IMPALA-4278 to help with concurrent startup of Impala and HMS.

Somewhat related to this issue, is that there seems to be multiple places where Impala creates a {{MetaStoreClientPool}}, I think it would make more sense to just have one global pool that is used across the process. Doing so would improve connection re-use and possibly decrease the number of HMS connections. There is actually a TODO in {{DirectMetaProvider}} as well that says {{msClientPool_}} should be a process wide singleton.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)