You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2025/06/25 07:19:00 UTC
[jira] [Commented] (IMPALA-14082) Batch processing RELOAD events on the same table
[ https://issues.apache.org/jira/browse/IMPALA-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17986024#comment-17986024 ]
Quanlong Huang commented on IMPALA-14082:
-----------------------------------------
There are multiple partition level RELOAD events on the same table and there are multiple tables that have such events. The RELOAD events of different tables might interleave. We can use the existing batching logic for ALTER_PARTITION events that already deals with this issue (IMPALA-12463).
> Batch processing RELOAD events on the same table
> ------------------------------------------------
>
> Key: IMPALA-14082
> URL: https://issues.apache.org/jira/browse/IMPALA-14082
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Sai Hemanth Gantasala
> Priority: Major
>
> Currently, only ALTER_PARTITION and INSERT partition events are processed in batches. RELOAD events are triggered by REFRESH in other Impala clusters. There could be lots of partition level REFRESH on the same table. Processing them one by one acquires the table lock multiple times to load individual partitions in sequence. This also keeps the table version changing which impacts performance of coordinators in local-catalog mode - query planning needs retry to handle InconsistentMetadataFetchException due to table version changes.
> Batch processing RELOAD events on the same table can load partiitons in parallel and also reduce duplicate reloads.
> CC [~hemanth619], [~VenuReddy]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org