You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Dmitriy Fingerman (Jira)" <ji...@apache.org> on 2024/08/06 13:49:00 UTC

[jira] [Resolved] (HIVE-28341) Iceberg: Change Major QB Full Table Compaction to compact partitions in parallel

     [ https://issues.apache.org/jira/browse/HIVE-28341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Fingerman resolved HIVE-28341.
--------------------------------------
    Fix Version/s: 4.0.1
       Resolution: Fixed

Thanks [~dkuzmenko]  for review!

> Iceberg: Change Major QB Full Table Compaction to compact partitions in parallel
> --------------------------------------------------------------------------------
>
>                 Key: HIVE-28341
>                 URL: https://issues.apache.org/jira/browse/HIVE-28341
>             Project: Hive
>          Issue Type: Task
>          Components: Hive, Iceberg integration
>            Reporter: Dmitriy Fingerman
>            Assignee: Dmitriy Fingerman
>            Priority: Major
>              Labels: hive, iceberg, pull-request-available
>             Fix For: 4.0.1
>
>
> Currently, Major compaction compacts a whole table in one step. If a table is partition and has a lot of data this operation can take a lot of time and it risks getting write conflicts at the commit stage. This can be improved to work partition by partition. Also, for each partition it will create one snapshot instead of 2 snapshots (truncate+IOW) created now when compacting the whole table in one step.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)