You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by "Vadla, Karthik" <ka...@intel.com> on 2015/06/28 07:20:38 UTC
Not able to see hive metastore in Zeppelin
Hi All,
Recently I have imported new databases into my hive meta store on cluster.
When I use %sql interpreter with below commands. I'm not able to see newly imported databases.
Command used:
%sql
show databases
It shows only default databases not the newly imported databases. But when I use hive command from command line
$hive -e "show databases"
It shows all the newly imported databases too.
Do anyone have clue on this, why I'm not able to see with zeppelin
Appreciate your help.
Thanks
Karthik Vadla
Re: Not able to see hive metastore in Zeppelin
Posted by moon soo Lee <mo...@apache.org>.
Hi,
Try either
a) place your hive-site.xml in ZEPPELIN_HOME/conf dir,
b) export HADOOP_CONF_DIR in ZEPPELIN_HOME/conf/zeppelin-env.sh to the
directory that contains your hive-site.xml
Thanks,
moon
On Sun, Jun 28, 2015 at 6:12 AM Brian McDevitt <br...@nerdery.com>
wrote:
> If you don't have the hive-site.xml in your SPARK_HOME/conf directory,
> spark will create a new Hive metastore in the current directory. See
> https://spark.apache.org/docs/1.3.1/sql-programming-guide.html#hive-tables
>
> You can test this by running the spark-shell and trying to use
> sqlContext.sql("show databases"). When you quit the spark-shell, if
> there's a metastore directory created, then you don't have spark configured
> correctly for Hive.
>
> HTH
>
>
> Thanks,
> Brian McDevitt
> Software Engineer
> The Nerdery
>
> On Sun, Jun 28, 2015 at 1:22 AM, IT CTO <go...@gmail.com> wrote:
>
>> Can you access these databases? e.g. use %hive to list the DB and then
>> use %sql to select from them?
>> I have the same behavior
>> Eran
>>
>> On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <ka...@intel.com>
>> wrote:
>>
>>> Hi All,
>>>
>>>
>>>
>>> Recently I have imported new databases into my hive meta store on
>>> cluster.
>>>
>>> When I use %sql interpreter with below commands. I’m not able to see
>>> newly imported databases.
>>>
>>>
>>>
>>> Command used:
>>>
>>>
>>>
>>> *%sql*
>>>
>>>
>>>
>>> *show databases*
>>>
>>>
>>>
>>> It shows only default databases not the newly imported databases. But
>>> when I use hive command from command line
>>>
>>>
>>>
>>> *$hive –e “show databases”*
>>>
>>>
>>>
>>> It shows all the newly imported databases too.
>>>
>>>
>>>
>>>
>>>
>>> Do anyone have clue on this, why I’m not able to see with zeppelin
>>>
>>> Appreciate your help.
>>>
>>>
>>>
>>> Thanks
>>>
>>> Karthik Vadla
>>>
>>
>
Re: Not able to see hive metastore in Zeppelin
Posted by Brian McDevitt <br...@nerdery.com>.
If you don't have the hive-site.xml in your SPARK_HOME/conf directory,
spark will create a new Hive metastore in the current directory. See
https://spark.apache.org/docs/1.3.1/sql-programming-guide.html#hive-tables
You can test this by running the spark-shell and trying to use
sqlContext.sql("show databases"). When you quit the spark-shell, if
there's a metastore directory created, then you don't have spark configured
correctly for Hive.
HTH
Thanks,
Brian McDevitt
Software Engineer
The Nerdery
On Sun, Jun 28, 2015 at 1:22 AM, IT CTO <go...@gmail.com> wrote:
> Can you access these databases? e.g. use %hive to list the DB and then use
> %sql to select from them?
> I have the same behavior
> Eran
>
> On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <ka...@intel.com>
> wrote:
>
>> Hi All,
>>
>>
>>
>> Recently I have imported new databases into my hive meta store on
>> cluster.
>>
>> When I use %sql interpreter with below commands. I’m not able to see
>> newly imported databases.
>>
>>
>>
>> Command used:
>>
>>
>>
>> *%sql*
>>
>>
>>
>> *show databases*
>>
>>
>>
>> It shows only default databases not the newly imported databases. But
>> when I use hive command from command line
>>
>>
>>
>> *$hive –e “show databases”*
>>
>>
>>
>> It shows all the newly imported databases too.
>>
>>
>>
>>
>>
>> Do anyone have clue on this, why I’m not able to see with zeppelin
>>
>> Appreciate your help.
>>
>>
>>
>> Thanks
>>
>> Karthik Vadla
>>
>
RE: Not able to see hive metastore in Zeppelin
Posted by "Vadla, Karthik" <ka...@intel.com>.
Hi Eran,
Even I’m looking for a clarify in this regard.
1. In my one-node CDH 5.4 cluster, when I configured zeppelin. I’m able to access hive metastore with all hives commands using %sql (ex: show databases, show tables..etc)
2. In my multi-node CDH 5.4 cluster, when I configured zeppelin. I’m able to access hive metastore only with %hive interpreter. Not able to access any hive tables using %sql interpreter. I really don’t have clue what is happening at backend.
3. Yes, it uses spark but in local mode. I did not configure SPARK_HOME in zeppelin but yet it uses local spark.
I hope this helps. Please correct me if go wrong anywhere.
Thanks
Karthik
From: IT CTO [mailto:goi.cto@gmail.com]
Sent: Sunday, June 28, 2015 11:51 PM
To: users@zeppelin.incubator.apache.org
Subject: Re: Not able to see hive metastore in Zeppelin
I just want to understand what is the expected behavior?
1) show tables works only using hive interpreter
2) %sql allow access directly to hive tables
3) %sql does not show tables when running "show tables"
Is that correct? this is what is happening on my configuration.
BTW, When using %SQL to access a hive table is that runs spark behind the scene? should I expect different performance when using spark code directly on hdfs files ves spark SQL through hive tables?
Eran
On Mon, Jun 29, 2015 at 12:36 AM Vadla, Karthik <ka...@intel.com>> wrote:
Thank you Eran, I did help me.
Now I’m able to see all databases.
Regards
Karthik
From: IT CTO [mailto:goi.cto@gmail.com<ma...@gmail.com>]
Sent: Saturday, June 27, 2015 11:22 PM
To: users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>
Subject: Re: Not able to see hive metastore in Zeppelin
Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?
I have the same behavior
Eran
On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <ka...@intel.com>> wrote:
Hi All,
Recently I have imported new databases into my hive meta store on cluster.
When I use %sql interpreter with below commands. I’m not able to see newly imported databases.
Command used:
%sql
show databases
It shows only default databases not the newly imported databases. But when I use hive command from command line
$hive –e “show databases”
It shows all the newly imported databases too.
Do anyone have clue on this, why I’m not able to see with zeppelin
Appreciate your help.
Thanks
Karthik Vadla
Re: Not able to see hive metastore in Zeppelin
Posted by IT CTO <go...@gmail.com>.
I just want to understand what is the expected behavior?
1) show tables works only using hive interpreter
2) %sql allow access directly to hive tables
3) %sql does not show tables when running "show tables"
Is that correct? this is what is happening on my configuration.
BTW, When using %SQL to access a hive table is that runs spark behind the
scene? should I expect different performance when using spark code directly
on hdfs files ves spark SQL through hive tables?
Eran
On Mon, Jun 29, 2015 at 12:36 AM Vadla, Karthik <ka...@intel.com>
wrote:
> Thank you Eran, I did help me.
>
> Now I’m able to see all databases.
>
>
>
> Regards
>
> Karthik
>
>
>
> *From:* IT CTO [mailto:goi.cto@gmail.com]
> *Sent:* Saturday, June 27, 2015 11:22 PM
> *To:* users@zeppelin.incubator.apache.org
> *Subject:* Re: Not able to see hive metastore in Zeppelin
>
>
>
> Can you access these databases? e.g. use %hive to list the DB and then use
> %sql to select from them?
>
> I have the same behavior
>
> Eran
>
>
>
> On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <ka...@intel.com>
> wrote:
>
> Hi All,
>
>
>
> Recently I have imported new databases into my hive meta store on cluster.
>
> When I use %sql interpreter with below commands. I’m not able to see newly
> imported databases.
>
>
>
> Command used:
>
>
>
> *%sql*
>
>
>
> *show databases*
>
>
>
> It shows only default databases not the newly imported databases. But when
> I use hive command from command line
>
>
>
> *$hive –e “show databases”*
>
>
>
> It shows all the newly imported databases too.
>
>
>
>
>
> Do anyone have clue on this, why I’m not able to see with zeppelin
>
> Appreciate your help.
>
>
>
> Thanks
>
> Karthik Vadla
>
>
RE: Not able to see hive metastore in Zeppelin
Posted by "Vadla, Karthik" <ka...@intel.com>.
Thank you Eran, I did help me.
Now I’m able to see all databases.
Regards
Karthik
From: IT CTO [mailto:goi.cto@gmail.com]
Sent: Saturday, June 27, 2015 11:22 PM
To: users@zeppelin.incubator.apache.org
Subject: Re: Not able to see hive metastore in Zeppelin
Can you access these databases? e.g. use %hive to list the DB and then use %sql to select from them?
I have the same behavior
Eran
On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <ka...@intel.com>> wrote:
Hi All,
Recently I have imported new databases into my hive meta store on cluster.
When I use %sql interpreter with below commands. I’m not able to see newly imported databases.
Command used:
%sql
show databases
It shows only default databases not the newly imported databases. But when I use hive command from command line
$hive –e “show databases”
It shows all the newly imported databases too.
Do anyone have clue on this, why I’m not able to see with zeppelin
Appreciate your help.
Thanks
Karthik Vadla
Re: Not able to see hive metastore in Zeppelin
Posted by IT CTO <go...@gmail.com>.
Can you access these databases? e.g. use %hive to list the DB and then use
%sql to select from them?
I have the same behavior
Eran
On Sun, Jun 28, 2015 at 8:21 AM Vadla, Karthik <ka...@intel.com>
wrote:
> Hi All,
>
>
>
> Recently I have imported new databases into my hive meta store on cluster.
>
> When I use %sql interpreter with below commands. I’m not able to see newly
> imported databases.
>
>
>
> Command used:
>
>
>
> *%sql*
>
>
>
> *show databases*
>
>
>
> It shows only default databases not the newly imported databases. But when
> I use hive command from command line
>
>
>
> *$hive –e “show databases”*
>
>
>
> It shows all the newly imported databases too.
>
>
>
>
>
> Do anyone have clue on this, why I’m not able to see with zeppelin
>
> Appreciate your help.
>
>
>
> Thanks
>
> Karthik Vadla
>