[jira] [Assigned] (HUDI-2535) Late arriving records and global index with partition path update set to true

["sivabalan narayanan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan reassigned HUDI-2535:
-----------------------------------------

    Assignee: sivabalan narayanan

> Late arriving records and global index with partition path update set to true
> -----------------------------------------------------------------------------
>
>                 Key: HUDI-2535
>                 URL: https://issues.apache.org/jira/browse/HUDI-2535
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: Writer Core
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Major
>              Labels: sev:high, user-support-issues
>
> incase of a global index, we have a config to update partition path. if this is set to true, if there is an incoming record to a newer partition compared to whats in storage, older record will be deleted and new incoming record will be routed to new partition. 
> But it could run into issues if new incoming is a late arriving record. Expected behavior is, old record is retained and new one is discarded it it has lower preCombine value. But in this case, we may not honor that. 
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)