[jira] [Created] (HUDI-3823) Enabling MT by default on Read path makes HiveSync fail

["Alexey Kudinkin (Jira)" <ji...@apache.org>]

Alexey Kudinkin created HUDI-3823:
-------------------------------------

             Summary: Enabling MT by default on Read path makes HiveSync fail
                 Key: HUDI-3823
                 URL: https://issues.apache.org/jira/browse/HUDI-3823
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Alexey Kudinkin
            Assignee: Ethan Guo
             Fix For: 0.11.0


{code:java}
#################################################################################################
49273 [main] INFO  org.apache.hudi.integ.ITTestBase  - Container : /adhoc-1, Running command :spark-submit --class org.apache.hudi.hive.HiveSyncTool /var/hoodie/ws/docker/hoodie/hadoop/hive_base/target/hoodie-hive-sync-bundle.jar --database default --table stock_ticks_cow --base-path /user/hive/warehouse/stock_ticks_cow --base-file-format PARQUET --user hive --pass hive --jdbc-url jdbc:hive2://hiveserver:10000 --partitioned-by dt
49273 [main] INFO  org.apache.hudi.integ.ITTestBase  - 
#################################################################################################
59728 [dockerjava-jaxrs-async-8] INFO  org.apache.hudi.integ.ITTestBase  - onComplete called
59732 [main] INFO  org.apache.hudi.integ.ITTestBase  - Exit code for command : 1
59732 [main] ERROR org.apache.hudi.integ.ITTestBase  -  ###### Stdout #######59733 [main] ERROR org.apache.hudi.integ.ITTestBase  -  ###### Stderr #######
22/04/07 21:32:24 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/util/Bytes
    at org.apache.hudi.common.bootstrap.index.HFileBootstrapIndex.<clinit>(HFileBootstrapIndex.java:94)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.hudi.common.util.ReflectionUtils.getClass(ReflectionUtils.java:54)
    at org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:89)
    at org.apache.hudi.common.bootstrap.index.BootstrapIndex.getBootstrapIndex(BootstrapIndex.java:163)
    at org.apache.hudi.common.table.view.AbstractTableFileSystemView.init(AbstractTableFileSystemView.java:108)
    at org.apache.hudi.common.table.view.HoodieTableFileSystemView.init(HoodieTableFileSystemView.java:108)
    at org.apache.hudi.common.table.view.HoodieTableFileSystemView.<init>(HoodieTableFileSystemView.java:102)
    at org.apache.hudi.common.table.view.HoodieTableFileSystemView.<init>(HoodieTableFileSystemView.java:93)
    at org.apache.hudi.metadata.HoodieTableMetadataUtil.getFileSystemView(HoodieTableMetadataUtil.java:1014)
    at org.apache.hudi.metadata.HoodieTableMetadataUtil.getPartitionFileSlices(HoodieTableMetadataUtil.java:1032)
    at org.apache.hudi.metadata.HoodieTableMetadataUtil.getPartitionLatestMergedFileSlices(HoodieTableMetadataUtil.java:980)
    at org.apache.hudi.metadata.HoodieBackedTableMetadata.getPartitionFileSliceToKeysMapping(HoodieBackedTableMetadata.java:370)
    at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:197)
    at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:140)
    at org.apache.hudi.metadata.BaseTableMetadata.fetchAllPartitionPaths(BaseTableMetadata.java:281)
    at org.apache.hudi.metadata.BaseTableMetadata.getAllPartitionPaths(BaseTableMetadata.java:111)
    at org.apache.hudi.common.fs.FSUtils.getAllPartitionPaths(FSUtils.java:299)
    at org.apache.hudi.sync.common.AbstractSyncHoodieClient.getPartitionsWrittenToSince(AbstractSyncHoodieClient.java:189)
    at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:226)
    at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:150)
    at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:138)
    at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:433)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.util.Bytes
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 36 more59742 [main] INFO  org.apache.hudi.integ.ITTestBase  -  {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)