[jira] [Updated] (HUDI-3605) hbase dependency need relocation or use other version for hive3

["Raymond Xu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raymond Xu updated HUDI-3605:
-----------------------------
    Issue Type: Bug  (was: Epic)

> hbase dependency need relocation or use other version for hive3
> ---------------------------------------------------------------
>
>                 Key: HUDI-3605
>                 URL: https://issues.apache.org/jira/browse/HUDI-3605
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: rex xiong
>            Priority: Major
>             Fix For: 0.11.0
>
>
> Hive 3 now uses hbase version 2.0.0-alpha4，which has conflict with the hbase version of hudi. See the problem as below.
> {code:java}
> hive> select * from hudi_mor_part_table_spark3_0110_0309_ro;
> OK
> Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hbase.io.hfile.HFile
> .createReader(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hado
> op/fs/Path;Lorg/apache/hadoop/hbase/io/FSDataInputStreamWrapper;JLorg/apache/hadoop/hbase/io/hfile/CacheConfig;Lorg/apache/hadoop/conf/Configuration;)Lorg/ap
> ache/hadoop/hbase/io/hfile/HFile$Reader;
>         at org.apache.hudi.io.storage.HoodieHFileReader.<init>(HoodieHFileReader.java:98)
>         at org.apache.hudi.common.table.log.block.HoodieHFileDataBlock.deserializeRecords(HoodieHFileDataBlock.java:158)
>         at org.apache.hudi.common.table.log.block.HoodieDataBlock.readRecordsFromBlockPayload(HoodieDataBlock.java:167)
>         at org.apache.hudi.common.table.log.block.HoodieDataBlock.getRecordItr(HoodieDataBlock.java:125)
>         at org.apache.hudi.common.table.log.block.HoodieDataBlock.getRecordItr(HoodieDataBlock.java:151)
>         at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processDataBlock(AbstractHoodieLogRecordReader.java:363)
>         at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.processQueuedBlocksForInstant(AbstractHoodieLogRecordReader.java:427)
>         at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scan(AbstractHoodieLogRecordReader.java:242)
>         at org.apache.hudi.common.table.log.AbstractHoodieLogRecordReader.scan(AbstractHoodieLogRecordReader.java:181)
>         at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.performScan(HoodieMergedLogRecordScanner.java:101)
>         at org.apache.hudi.metadata.HoodieMetadataMergedLogRecordReader.<init>(HoodieMetadataMergedLogRecordReader.java:71)
>         at org.apache.hudi.metadata.HoodieMetadataMergedLogRecordReader.<init>(HoodieMetadataMergedLogRecordReader.java:51)
>         at org.apache.hudi.metadata.HoodieMetadataMergedLogRecordReader$Builder.build(HoodieMetadataMergedLogRecordReader.java:246)
>         at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:379)
>         at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$openReadersIfNeeded$4(HoodieBackedTableMetadata.java:295)
>         at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
>         at org.apache.hudi.metadata.HoodieBackedTableMetadata.openReadersIfNeeded(H
> oodieBackedTableMetadata.java:283)
>         at org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getRecordsByKeys$0(HoodieBackedTableMetadata.java:139)
>         at java.util.HashMap.forEach(HashMap.java:1289)
>         at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:138)
>         at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:128)
>         at org.apache.hudi.metadata.BaseTableMetadata.fetchAllPartitionPaths(BaseTableMetadata.java:274)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)