[jira] [Commented] (IMPALA-12835) Transactional tables are unsynced when hms_event_incremental_refresh_transactional_table is disabled

["Csaba Ringhofer (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17819713#comment-17819713 ] 

Csaba Ringhofer commented on IMPALA-12835:
------------------------------------------

I think that what actually broke this is IMPALA-11534
Without hms_event_incremental_refresh_transactional_table the only event catalogd processes during an INSERT to an unpartitioned ACID table is the ALTER_TABLE event -  since IMPALA-11534 most ALTER_TABLE events do not lead to reloading file medatata, so while HMS metadata will be reloaded, the file listing won't be refreshed (even though the validWriteIdList is refreshed)

Note that this issue only occurs with unpartitioned tables, partitioned tables are refreshed correctly when processing the ALTER_PARTITION events.

> Transactional tables are unsynced when hms_event_incremental_refresh_transactional_table is disabled
> ----------------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-12835
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12835
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog
>            Reporter: Quanlong Huang
>            Priority: Critical
>
> There are some test failures when hms_event_incremental_refresh_transactional_table is disabled:
>  * tests/metadata/test_event_processing.py::TestEventProcessing::test_transactional_insert_events
>  * tests/metadata/test_event_processing.py::TestEventProcessing::test_event_based_replication
> I can reproduce the issue locally:
> {noformat}
> $ bin/start-impala-cluster.py --catalogd_args=--hms_event_incremental_refresh_transactional_table=false
> impala-shell> create table txn_tbl (id int, val int) stored as parquet tblproperties ('transactional'='true','transactional_properties'='insert_only');
> impala-shell> describe txn_tbl;  -- make the table loaded in Impala
> hive> insert into txn_tbl values(101, 200);
> impala-shell> select * from txn_tbl; {noformat}
> Impala shows no results until a REFRESH runs on this table.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org