[jira] [Assigned] (HUDI-2443) KVComparator in HFile for metadata table is tied to HBase version and shading

["Rajesh Mahindra (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajesh Mahindra reassigned HUDI-2443:
-------------------------------------

    Assignee: Sagar Sumit  (was: Manoj Govindassamy)

> KVComparator in HFile for metadata table is tied to HBase version and shading
> -----------------------------------------------------------------------------
>
>                 Key: HUDI-2443
>                 URL: https://issues.apache.org/jira/browse/HUDI-2443
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Ryan Pifer
>            Assignee: Sagar Sumit
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.10.0
>
>
> There is no guarantee of compatibility between HFile reading/writing from different versions of ** HBase. For instance, HBase version 2.2.6 modified the comparator class stored in HFile metadata to {{org.apache.hadoop.hbase.KeyValue.KVComparator}} and {{org.apache.hadoop.hbase.KeyValue$KeyComparator}} no longer exists which may create issues when reading HFiles created by earlier versions of Hudi. And this comparator class is already marked as deprecated for {{CellComparatorImpl.}}
> {{This also creates issues when shading HBase dependencies under Hudi namespace}}
> {{Can we refactor HoodieKVComparator which is used in bootstrap index and have it applied for all use of HFile in Hudi. }}
> {{[https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/bootstrap/index/HFileBootstrapIndex.java#L580-L586]}}
>  
> There will need to be some logic as well for backward compatibility when reading metadata which was written from earlier version of Hudi as well. We can maybe enforce that user must perform an upgrade step beforehand which re-writes metadata base file
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)