You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2024/03/12 15:21:00 UTC
[jira] [Created] (IMPALA-12895) REFRESH doesn't detect changes in partition locations in ACID tables
Csaba Ringhofer created IMPALA-12895:
----------------------------------------
Summary: REFRESH doesn't detect changes in partition locations in ACID tables
Key: IMPALA-12895
URL: https://issues.apache.org/jira/browse/IMPALA-12895
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Csaba Ringhofer
This was discovered by running test metadata.test_event_processing.TestEventProcessing.test_transact_partition_location_change_from_hive when flag hms_event_incremental_refresh_transactional_table is set to false.
[https://github.com/apache/impala/blob/ab6c9467f6347671b971dbce4c640bea032b6ed9/tests/metadata/test_event_processing.py#L164]
When hms_event_incremental_refresh_transactional_table is true (default), the alter partition event is processed correctly and the location change is detected. But if it is false or event processing is turned off, the change is not detected and running REFRESH on the table also doesn't update the location.
The different handling based on the flag seems intentional:
https://github.com/apache/impala/blob/ab6c9467f6347671b971dbce4c640bea032b6ed9/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java#L2606
This seems to be an old issues while the test was added in a recent commit:
[https://github.com/apache/impala/commit/32b29ff36fb3e05fd620a6714de88805052d0117]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)