You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@impala.apache.org by "Vihang Karajgaonkar (Jira)" <ji...@apache.org> on 2021/12/02 16:55:00 UTC

[jira] [Resolved] (IMPALA-9857) Batch ALTER_PARTITION events

     [ https://issues.apache.org/jira/browse/IMPALA-9857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vihang Karajgaonkar resolved IMPALA-9857.
-----------------------------------------
    Fix Version/s: Impala 4.1.0
       Resolution: Fixed

> Batch ALTER_PARTITION events
> ----------------------------
>
>                 Key: IMPALA-9857
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9857
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Catalog
>            Reporter: Vihang Karajgaonkar
>            Assignee: Vihang Karajgaonkar
>            Priority: Major
>             Fix For: Impala 4.1.0
>
>
> When Hive inserts data into partitioned tables, it generates a lot of ALTER_PARTITION (and possibly INSERT_EVENT) in quick succession. Currently, such events are processed one by one by EventsProcessor which is can be slow and can cause EventsProcessor to lag behind. This JIRA proposes to use batching for such ALTER_PARTITION events such that all the successive ALTER_PARTITION events for the same table are batched together into one ALTER_PARTITIONS event and then are processed together to refresh all the partitions from the events. This can significantly speed up the event processing in such cases.   



--
This message was sent by Atlassian Jira
(v8.20.1#820001)