[jira] [Created] (IMPALA-10663) Coordinator might observe stale metadata in local catalog mode

["Zoltán Borók-Nagy (Jira)" <ji...@apache.org>]

Zoltán Borók-Nagy created IMPALA-10663:
------------------------------------------

             Summary: Coordinator might observe stale metadata in local catalog mode
                 Key: IMPALA-10663
                 URL: https://issues.apache.org/jira/browse/IMPALA-10663
             Project: IMPALA
          Issue Type: Bug
          Components: Catalog, Frontend
            Reporter: Zoltán Borók-Nagy


Coordinator might observe stale metadata in local catalog mode, even if the DDL operation was executed on the same coordinator.

The issue is reproducible in the dev environment using local catalog mode:
{noformat}
bin/start-impala-cluster.py --impalad_args --use_local_catalog=true --catalogd_args --catalog_topic_mode=minimal --catalogd_args --hms_event_polling_interval_s=1
{noformat}
{noformat}
CREATE TABLE doub (d double);
{noformat}
Renaming the table back and forth and querying from it reproduces the issue:

bin/impala-shell.sh -q "alter table doub rename to doub_tmp; select * from doub_tmp; alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub;"
{noformat}
...
... then, after a while ...
...
Query: alter table doub rename to doub_tmp
+--------------------------+
| summary                  |
+--------------------------+
| Renaming was successful. |
+--------------------------+
Fetched 1 row(s) in 0.02s
Query: select * from doub_tmp
Query submitted at: 2021-04-20 14:49:52 (Coordinator: http://localhost:25000)
ERROR: AnalysisException: Could not resolve table reference: 'doub_tmp'

Could not execute command: select * from doub_tmp
{noformat}
Then, if I execute the same SELECT a bit later:
{noformat}
$ bin/impala-shell.sh -q "select * from doub_tmp"

Fetched 0 row(s) in 0.12s
{noformat}
I think if the DDL operation succeeds, then the following operations on the same coordinator should see its effect.

The current behavior makes tests flaky. The workaround is to set sync_ddl=true, but it makes these operations very slow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org