You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/07/08 16:24:00 UTC

[jira] [Updated] (HIVE-28341) Iceberg: Change Major QB Full Table Compaction to compact partition by partition

     [ https://issues.apache.org/jira/browse/HIVE-28341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HIVE-28341:
----------------------------------
    Labels: hive iceberg pull-request-available  (was: hive iceberg)

> Iceberg: Change Major QB Full Table Compaction to compact partition by partition
> --------------------------------------------------------------------------------
>
>                 Key: HIVE-28341
>                 URL: https://issues.apache.org/jira/browse/HIVE-28341
>             Project: Hive
>          Issue Type: Task
>          Components: Hive, Iceberg integration
>            Reporter: Dmitriy Fingerman
>            Assignee: Dmitriy Fingerman
>            Priority: Major
>              Labels: hive, iceberg, pull-request-available
>
> Currently, Major compaction compacts a whole table in one step. If a table is partition and has a lot of data this operation can take a lot of time and it risks getting write conflicts at the commit stage. This can be improved to work partition by partition. Also, for each partition it will create one snapshot instead of 2 snapshots (truncate+IOW) created now when compacting the whole table in one step.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)