You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2024/03/01 16:15:00 UTC
[jira] [Comment Edited] (IMPALA-12812) Send reload event after ALTER TABLE RECOVER PARTITIONS
[ https://issues.apache.org/jira/browse/IMPALA-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822631#comment-17822631 ]
Csaba Ringhofer edited comment on IMPALA-12812 at 3/1/24 4:14 PM:
------------------------------------------------------------------
I was wrong about this one:
" It also reloads the table (similarly to other DDLs) and detects new files in existing partitions. "
At the moment no refresh is done on partitions that already exist in HMS.
A valid workaround is to call both REFRESH after ALTER TABLE RECOVER PARTITIONS - REFRESH will both detect new files and send the reload event.
Closing the issue as it wouldn't be that useful.
was (Author: csringhofer):
I was wrong about this one:
"An HMS event is created for the new partitions but there is no event that would indicate that there are new files in existing partitions. "
At the moment no refresh is done on partitions that already exist in HMS.
A valid workaround is to call both REFRESH after ALTER TABLE RECOVER PARTITIONS - REFRESH will both detect new files and send the reload event.
Closing the issue as it wouldn't be that useful.
> Send reload event after ALTER TABLE RECOVER PARTITIONS
> ------------------------------------------------------
>
> Key: IMPALA-12812
> URL: https://issues.apache.org/jira/browse/IMPALA-12812
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Csaba Ringhofer
> Priority: Major
>
> IMPALA-11808 added support for sending reload events after REFRESH to allow other Impala cluster connecting to the same HMS to also reload their tables. REFRESH is often used when in external tables the files are written directly to filesystem without notifying HMS, so Impala needs to update its cache and can't rely on HMS notifications.
> The same could be useful for ALTER TABLE RECOVER PARTITIONS. It detects partition directories that were only created in the FS but not in HMS and creates them in HMS too. It also reloads the table (similarly to other DDLs) and detects new files in existing partitions. An HMS event is created for the new partitions but there is no event that would indicate that there are new files in existing partitions. As ALTER TABLE RECOVER PARTITIONS is called when the user expects changes in the filesystem (similarly to REFRESH), it could be useful to send a reload event after it is finished.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org