[jira] (HIVE-24706) Spark SQL access hive on HBase table access exception

["Mehul Thakkar (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-24706 ]


    Mehul Thakkar deleted comment on HIVE-24706:
    --------------------------------------

was (Author: JIRAUSER297345):
I encountered the same issue with Spark 3.x and Scala 2.12.x. It works fine with Spark 2.4.x and Scala 2.11.x

 

*Exception in thread "main" java.io.IOException: Cannot create a record reader because of a previous error. Please look at the previous logs lines from the task's full log for more details. at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:253) at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:131) at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:300) I am calling spark-submit as follows.*

 

The status of this bug is still open. I am wondering when it will be fixed in the official release.{*}{*}

> Spark SQL access hive on HBase table access exception
> -----------------------------------------------------
>
>                 Key: HIVE-24706
>                 URL: https://issues.apache.org/jira/browse/HIVE-24706
>             Project: Hive
>          Issue Type: Bug
>          Components: HBase Handler
>            Reporter: zhangzhanchang
>            Priority: Major
>         Attachments: image-2021-01-30-15-51-58-665.png
>
>
> Hivehbasetableinputformat relies on two versions of inputformat，one is org.apache.hadoop.mapred.InputFormat, the other is org.apache.hadoop.mapreduce.InputFormat,Causes
> spark 3.0(https://github.com/apache/spark/pull/31302) both conditions to be true:
>  # classOf[oldInputClass[_, _]].isAssignableFrom(inputFormatClazz) is true
>  # classOf[newInputClass[_, _]].isAssignableFrom(inputFormatClazz) is true
> !image-2021-01-30-15-51-58-665.png|width=430,height=137!
> Hivehbasetableinputformat relies on inputformat to be changed to org.apache.hadoop.mapreduce or org.apache.hadoop.mapred?
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)