[jira] [Resolved] (HIVE-26183) Create delete writer for the UPDATE statemens

["Peter Vary (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-26183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Peter Vary resolved HIVE-26183.
-------------------------------
    Fix Version/s: 4.0.0
       Resolution: Fixed

Pushed to master.
Thanks for the review [~szita] and [~Marton Bod]!

> Create delete writer for the UPDATE statemens
> ---------------------------------------------
>
>                 Key: HIVE-26183
>                 URL: https://issues.apache.org/jira/browse/HIVE-26183
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Peter Vary
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>          Time Spent: 6h
>  Remaining Estimate: 0h
>
> During the investigation of the updates of partitioned table we had the following issue:
> - Iceberg inserts are needed to be sorted by the new partition keys
> - Iceberg deletes are needed to be sorted by the old partition keys and filenames
> This could contradict each other. OTOH Hive updates create a single query and writes out the insert/delete record for ever row. This would mean plenty of open writers.
> We might want to create something like a https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/io/SortedPosDeleteWriter.java, but we do not want to keep the whole rows in memory.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)