Spark read hive content

["Ji, Hao Wei Jeffery" <Je...@dnvgl.com>]

Hi,

I've installed Zeppelin with HDP2.3.  I've a question regarding the spark. As I tried in command line, spark sql  can query table created in hive. So I'm wondering in Zeppline, if I input %sql select * from <hive-table>, it shall work. But it returns an error says cannot find table. But if I change %sql to %hive, it works well.

Anyone know how to setup it?

OS: Centos 6.7
Tools installed with Ambari
Hadoop: 2.7.1
Hive: 1.2.1
HBase: 1.1.1
Spark: 1.4.1
Zeppelin 3.9

Spark Interpreter:
name

value

args

master

local[*]

spark.app.name

Zeppelin

spark.cores.max

spark.executor.memory

512m

spark.home

/usr/hdp/current/spark-client

spark.yarn.jar

/usr/share/zeppelin/interpreter/spark/zeppelin-spark-0.6.0-incubating-SNAPSHOT.jar

zeppelin.dep.additionalRemoteRepository

spark-packages,http://dl.bintray.com/spark-packages/maven,false;

zeppelin.dep.localrepo

local-repo

zeppelin.pyspark.python

python

zeppelin.spark.concurrentSQL

true

zeppelin.spark.maxResult

1000

zeppelin.spark.useHiveContext

true



**************************************************************************************
This e-mail and any attachments thereto may contain confidential information and/or information protected by intellectual property rights for the exclusive attention of the intended addressees named above. If you have received this transmission in error, please immediately notify the sender by return e-mail and delete this message and its attachments. Unauthorized use, copying or further full or partial distribution of this e-mail or its contents is prohibited.
**************************************************************************************

Re: Spark read hive content

[moon soo Lee <mo...@apache.org>]

Hi,

I think you'll need to manage get Zeppelin's Spark interpreter read
hive-site.xml from it's classpath. Simplest way is placing hive-site.xml
under conf/ directory of Zeppelin.

Hope this helps.

Best,
moon

On Sun, Nov 1, 2015 at 1:09 AM Ji, Hao Wei Jeffery <Je...@dnvgl.com>
wrote:

> Hi,
>
>
>
> I’ve installed Zeppelin with HDP2.3.  I’ve a question regarding the spark.
> As I tried in command line, spark sql  can query table created in hive. So
> I’m wondering in Zeppline, if I input %sql select * from <hive-table>, it
> shall work. But it returns an error says cannot find table. But if I change
> %sql to %hive, it works well.
>
>
>
> Anyone know how to setup it?
>
>
>
> OS: Centos 6.7
>
> Tools installed with Ambari
>
> Hadoop: 2.7.1
>
> Hive: 1.2.1
>
> HBase: 1.1.1
>
> Spark: 1.4.1
>
> Zeppelin 3.9
>
>
>
> Spark Interpreter:
>
> *name*
>
> *value*
>
> args
>
> master
>
> local[*]
>
> spark.app.name
>
> Zeppelin
>
> spark.cores.max
>
> spark.executor.memory
>
> 512m
>
> spark.home
>
> /usr/hdp/current/spark-client
>
> spark.yarn.jar
>
>
> /usr/share/zeppelin/interpreter/spark/zeppelin-spark-0.6.0-incubating-SNAPSHOT.jar
>
> zeppelin.dep.additionalRemoteRepository
>
> spark-packages,http://dl.bintray.com/spark-packages/maven,false;
>
> zeppelin.dep.localrepo
>
> local-repo
>
> zeppelin.pyspark.python
>
> python
>
> zeppelin.spark.concurrentSQL
>
> true
>
> zeppelin.spark.maxResult
>
> 1000
>
> zeppelin.spark.useHiveContext
>
> true
>
>
>
>
> **************************************************************************************
> This e-mail and any attachments thereto may contain confidential
> information and/or information protected by intellectual property rights
> for the exclusive attention of the intended addressees named above. If you
> have received this transmission in error, please immediately notify the
> sender by return e-mail and delete this message and its attachments.
> Unauthorized use, copying or further full or partial distribution of this
> e-mail or its contents is prohibited.
>
> **************************************************************************************
>