[jira] [Work started] (HIVE-25975) Optimize ClusteredWriter for bucketed Iceberg tables

["Ádám Szita (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on HIVE-25975 started by Ádám Szita.
-----------------------------------------
> Optimize ClusteredWriter for bucketed Iceberg tables
> ----------------------------------------------------
>
>                 Key: HIVE-25975
>                 URL: https://issues.apache.org/jira/browse/HIVE-25975
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Ádám Szita
>            Assignee: Ádám Szita
>            Priority: Major
>
> The first version of the ClusteredWriter in Hive-Iceberg will be lenient for bucketed tables: i.e. the records do not need to be ordered by the bucket values, the writer will just close its current file and open a new one for out-of-order records. 
> This is suboptimal for the long-term due to creating many small files. Spark uses a UDF to compute the bucket value for each record and therefore it is able to order the records by bucket values, achieving optimal clustering.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)