You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Sourav Mazumder <so...@gmail.com> on 2015/08/28 22:26:20 UTC

Support for Hive Storage Handle in Spark SQL and core Spark

Hi,

I have data written in HDFS using a custom storage handler of Hive. Can I
access that data in Spark using Spark SQL .

For example can I write a Spark SQL to access the data from a hive table in
HDFS which was created as -

CREATE TABLE custom_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.CustomStorageHandler'
WITH SERDEPROPERTIES (
"custom.columns.mapping" = "cf:string",
"custom.table.name" = "table_0"
);

Regards,
Sourav