[jira] [Created] (IMPALA-15121) Queries fail with "catalogd is in standby mode" during HA failover

["Yida Wu (Jira)" <ji...@apache.org>]

Yida Wu created IMPALA-15121:
--------------------------------

             Summary: Queries fail with "catalogd is in standby mode" during HA failover
                 Key: IMPALA-15121
                 URL: https://issues.apache.org/jira/browse/IMPALA-15121
             Project: IMPALA
          Issue Type: Improvement
          Components: Frontend
    Affects Versions: Impala 4.5.0
            Reporter: Yida Wu


When Catalogd HA is enabled, queries can fail during a catalogd failover with the following error:
{code:java}
java.lang.RuntimeException: org.apache.impala.catalog.local.LocalCatalogException: Unable to load database names
CAUSED BY: TException: Request for Catalog service is rejected since catalogd <host>:<port> is in standby mode
{code}
Reproduce steps:
1. Kill the active catalogd.
2. Run a query that needs to access the catalog:
{code:java}
refresh functional.alltypes;
{code}
3. Restart the killed catalogd.

The root cause is that when the catalogd is down, the frontend of impalad will keep trying to do RPCs with the current active catalogd, until it eventually comes up again. Because it comes back in standby status, the frontend will receive the standby rejection error introduced in IMPALA-12849, which ends up as a query failure. We should retry when receiving this error on the impalad side so that it can wait to receive the statestore update and talk to the correct active catalogd without a query failure.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org