You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2025/06/11 16:43:00 UTC
[jira] [Commented] (IMPALA-13850) Catalogd should not start metadata operation until initialization is done if HA is enabled
[ https://issues.apache.org/jira/browse/IMPALA-13850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17959790#comment-17959790 ]
ASF subversion and git services commented on IMPALA-13850:
----------------------------------------------------------
Commit 48c4d31344eeedfb988d7bc2a715f265a23fb0d9 in impala's branch refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=48c4d3134 ]
IMPALA-14130: Remove wait_num_tables arg in start-impala-cluster.py
IMPALA-13850 changed the behavior of bin/start-impala-cluster.py to wait
for the number of tables to be at least one. This is needed to detect
that the catalog has seen at least one update. There is special logic in
dataload to start Impala without tables in that circumstance.
This broke the perf-AB-test job, which starts Impala before loading
data. There are other times when we want to start Impala without tables,
and it is inconvenient to need to specify --wait_num_tables each time.
It is actually not necessary to wait for catalog metric of Coordinator
to reach certain value. Frontend (Coordinator) will not open its service
port until it heard the first catalog topic update form CatalogD.
IMPALA-13850 (part 2) also ensure that CatalogD with
--catalog_topic_mode=minimal will block serving Coordinator request
until it begin its first reset() operation. Therefore, waiting
Coordinator's catalog version is not needed anymore and
--wait_num_tables parameter can be removed.
This patch also slightly change the "progress log" of
start-impala-cluster.py to print the Coordinator's catalog version
instead of num DB and tables cached. The sleep interval time now include
time spent checking Coordinator's metric.
Testing:
- Pass dataload with updated script.
- Manually run start-impala-cluster.py in both legacy and local catalog
mode and confirm it works.
- Pass custom cluster test_concurrent_ddls.py and test_catalogd_ha.py
Change-Id: I4a3956417ec83de4fb3fc2ef1e72eb3641099f02
Reviewed-on: http://gerrit.cloudera.org:8080/22994
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Riza Suminto <ri...@cloudera.com>
> Catalogd should not start metadata operation until initialization is done if HA is enabled
> ------------------------------------------------------------------------------------------
>
> Key: IMPALA-13850
> URL: https://issues.apache.org/jira/browse/IMPALA-13850
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Wenzhe Zhou
> Assignee: Riza Suminto
> Priority: Critical
>
> In a case reported by user, the catalogd initialization failed to complete. Log messages showed that catalog HA was enabled. catalogd was blocked when trying to acquire "CatalogServer.catalog_lock_" when calling CatalogServer::UpdateActiveCatalogd() during statestore subscriber registration.
> Log message showed that there was IM command issued before catalogd tried to register to statestore.
> {code:java}
> I0310 12:21:34.093617 1 CatalogServiceCatalog.java:2188] Invalidated all metadata.
> I0310 12:21:34.094341 1 thrift-server.cc:419] ThriftServer 'StatestoreSubscriber' started on port: 23020
> I0310 12:21:34.094341 1816 TAcceptQueueServer.cpp:329] connection_setup_thread_pool_size is set to 2
> I0310 12:21:34.094586 1 thrift-util.cc:198] TSocket::open() error on socket (after THRIFT_POLL) <Host: localhost Port: 23020>: Connection refused
> I0310 12:21:34.094790 1 statestore-subscriber.cc:745] Starting statestore subscriber
> {code}
> We should not allow any metadata operation until initialization is done. When HA is enabled, catalog-server should not hold "CatalogServer.catalog_lock_" for long time before active catalogd is assigned.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org