You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "wineternity (Jira)" <ji...@apache.org> on 2021/09/27 03:55:00 UTC
[jira] [Created] (SPARK-36860) Create the external hive table for
HBase failed
wineternity created SPARK-36860:
-----------------------------------
Summary: Create the external hive table for HBase failed
Key: SPARK-36860
URL: https://issues.apache.org/jira/browse/SPARK-36860
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.2
Reporter: wineternity
We use follow sql to create hive external table , which read from hbase
{code:java}
CREATE EXTERNAL TABLE if not exists dev.sanyu_spotlight_headline_material(
rowkey string COMMENT 'HBase主键',
content string COMMENT '图文正文')
USING HIVE
ROW FORMAT SERDE
'org.apache.hadoop.hive.hbase.HBaseSerDe'
STORED BY
'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
'hbase.columns.mapping'=':key, cf1:content'
)
TBLPROPERTIES (
'hbase.table.name'='spotlight_headline_material'
);
{code}
But the sql failed in Spark 3.1.2, which throw this exception
{code:java}
21/09/27 11:44:24 INFO scheduler.DAGScheduler: Asked to cancel job group 26d7459f-7b58-4c18-9939-5f2737525ff2
21/09/27 11:44:24 ERROR thriftserver.SparkExecuteStatementOperation: Error executing query with 26d7459f-7b58-4c18-9939-5f2737525ff2, currentState RUNNING,
org.apache.spark.sql.catalyst.parser.ParseException:
Operation not allowed: Unexpected combination of ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' and STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'WITHSERDEPROPERTIES('hbase.columns.mapping'=':key, cf1:content')(line 5, pos 0)
{code}
this check was introduced from this change: [https://github.com/apache/spark/pull/28026]
Could anyone gave the introduction how to create the external table for hbase in spark3 now ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org