You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "voon (Jira)" <ji...@apache.org> on 2025/05/02 02:51:02 UTC

[jira] [Updated] (HUDI-6460) Fix Hbase Index for deletes

     [ https://issues.apache.org/jira/browse/HUDI-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

voon updated HUDI-6460:
-----------------------
    Fix Version/s: 1.1.0
                       (was: 1.0.2)

> Fix Hbase Index for deletes
> ---------------------------
>
>                 Key: HUDI-6460
>                 URL: https://issues.apache.org/jira/browse/HUDI-6460
>             Project: Apache Hudi
>          Issue Type: Sub-task
>          Components: index, tests-ci
>            Reporter: sivabalan narayanan
>            Assignee: Prashant Wason
>            Priority: Major
>             Fix For: 1.1.0
>
>
> With  adding delete support for RLI, [https://github.com/apache/hudi/pull/9058/files] 
> Hbase index needs some fixes. 
> Test that is failing is:
> TestSparkHoodieHBaseIndex.
> testTagLocationAndPartitionPathUpdateWithExplicitRollback
>  
> Root cause:
> when update partition path is set to true, within same batch we have a deleted record and a new insert record. So, to hbase we are sending both the records and for some inserts take precedence, while for others deletes take precedence. 
>  
> we need to fix SparkHoodieHbaseIndex.
> updateLocation
> to do one pass overWriteStatus and ensure we de-dup if we have two records where one of them is deleted and another is inserted. 
> but there are chances that only deletes are present, so in such cases, we need to ensure deletes are routed to hbase. 
>  
>  
>  
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)