[jira] [Commented] (IMPALA-10663) Coordinator might observe stale metadata in local catalog mode

["Quanlong Huang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504772#comment-17504772 ] 

Quanlong Huang commented on IMPALA-10663:
-----------------------------------------

Hi [~boroknagyz] , is the issue resolved now?

> Coordinator might observe stale metadata in local catalog mode
> --------------------------------------------------------------
>
>                 Key: IMPALA-10663
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10663
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog, Frontend
>            Reporter: Zoltán Borók-Nagy
>            Priority: Major
>
> Coordinator might observe stale metadata in local catalog mode, even if the DDL operation was executed on the same coordinator.
> The issue is reproducible in the dev environment using local catalog mode:
> {noformat}
> bin/start-impala-cluster.py --impalad_args --use_local_catalog=true --catalogd_args --catalog_topic_mode=minimal --catalogd_args --hms_event_polling_interval_s=1
> {noformat}
> {noformat}
> CREATE TABLE doub (d double);
> {noformat}
> Renaming the table back and forth and querying from it reproduces the issue:
> bin/impala-shell.sh -q "alter table doub rename to doub_tmp; select * from doub_tmp; alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub;"
> {noformat}
> ...
> ... then, after a while ...
> ...
> Query: alter table doub rename to doub_tmp
> +--------------------------+
> | summary                  |
> +--------------------------+
> | Renaming was successful. |
> +--------------------------+
> Fetched 1 row(s) in 0.02s
> Query: select * from doub_tmp
> Query submitted at: 2021-04-20 14:49:52 (Coordinator: http://localhost:25000)
> ERROR: AnalysisException: Could not resolve table reference: 'doub_tmp'
> Could not execute command: select * from doub_tmp
> {noformat}
> Then, if I execute the same SELECT a bit later:
> {noformat}
> $ bin/impala-shell.sh -q "select * from doub_tmp"
> Fetched 0 row(s) in 0.12s
> {noformat}
> I think if the DDL operation succeeds, then the following operations on the same coordinator should see its effect.
> The current behavior makes tests flaky. The workaround is to set sync_ddl=true, but it makes these operations very slow.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org