You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2024/08/15 21:15:00 UTC
[jira] [Commented] (IMPALA-13293) LocalCatalog's waitForCatalogUpdate() sleeps too much
[ https://issues.apache.org/jira/browse/IMPALA-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874059#comment-17874059 ]
ASF subversion and git services commented on IMPALA-13293:
----------------------------------------------------------
Commit 6323455425f946eee5c9bd78a94b99bf422c51de in impala's branch refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=632345542 ]
IMPALA-13293: Fix too long wait for initial catalog update
Coordinators always waited 2s in Frontend.WaitForCatalog() for the
first catalog update to arrive, even if the update arrived quicker.
This was obvious in case of LocalCatalog, but ImpaladCatalog looked
as if it would wake up when the update arrives by notifying
catalogUpdateEventNotifier_. In reality the arrival of the catalog
update led FeCatalogManager to create a new ImpaladCatalog instance
while Frontend was still waiting for the old instance which never
got notified.
The change fixes the wait/notify mechanism in ImpaladCatalog and
implements a similar logic for LocalCatalog in CatalogdMetaProvider.
Makes starting impala cluster faster:
ImpaladCatalog: ~12s->~11s
LocalCatalog: ~11->~10s
Change-Id: I4da1d0c0d25f80b02c39be863431201d1b913c5d
Reviewed-on: http://gerrit.cloudera.org:8080/21655
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> LocalCatalog's waitForCatalogUpdate() sleeps too much
> -----------------------------------------------------
>
> Key: IMPALA-13293
> URL: https://issues.apache.org/jira/browse/IMPALA-13293
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog, Frontend
> Reporter: Csaba Ringhofer
> Priority: Major
>
> Unlike ImpaladCatalog's waitForCatalogUpdate() the LocalCatalog version doesn't use a conditional variable and simply waits for timeoutMs. The timeout comes from MAX_CATALOG_UPDATE_WAIT_TIME_MS, which is 2 seconds. This means the the function will wait 2 seconds even it the catalog update arrived in the meantime. This 2 seconds is often nearly completely added to the impala cluster startup time.
> The sleep was added in https://gerrit.cloudera.org/#/c/11472/3/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
> Update: realized that this also doesn't work well for ImpaladCatalog - the FeCatalogManager creates a new ImpaladCatalog when a full topic update arrives, so the Object that waitForCatalogUpdate() waits for is never notified. My impression is that was broken a long time ago, even before LocalCatalog was added.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org