You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Yang <te...@gmail.com> on 2013/12/30 20:48:55 UTC
HiveMetaStoreClient only sees one of my DBs ?
if I log into my hive shell, do "show databases;" , I see many DBs:
Logging initialized using configuration in
file:/etc/hive/conf/hive-log4j.properties
hive> show databases;
OK
conf
confnew
default
money
testdb
Time taken: 1.57 seconds, Fetched: 6 row(s)
but somehow if I run the following java code using the client API, it only
shows "default"
public static void main(String args[]) throws MetaException {
HiveMetaStoreClient cli = new HiveMetaStoreClient(new HiveConf());
for(String db: cli.getAllDatabases())
System.out.println(db);
}
stence)
[2013-12-30 11:44:02,900] INFO Validating 2 unique key(s) for table DBS
(DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,913] INFO Validating 0 foreign key(s) for table DBS
(DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,919] INFO Validating 2 index(es) for table DBS
(DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,926] INFO Validating 1 unique key(s) for table
DATABASE_PARAMS (DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,937] INFO Validating 1 foreign key(s) for table
DATABASE_PARAMS (DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,942] INFO Validating 2 index(es) for table
DATABASE_PARAMS (DataNucleus.Datastore.Schema)
[2013-12-30 11:44:03,087] INFO Listener found initialisation for
persistable class org.apache.hadoop.hive.metastore.model.MDatabase
(DataNucleus.MetaData)
[2013-12-30 11:44:03,211] INFO 0: get_all_databases
(org.apache.hadoop.hive.metastore.HiveMetaStore)
[2013-12-30 11:44:03,261] INFO ugi=myuser_name ip=unknown-ip-addr
cmd=get_all_databases (org.apache.hadoop.hive.metastore.HiveMetaStore.audit)
default
why is it showing only 1 db? what setttings of default are different from
the others to enable it to be shown? also I wonder how is that HiveConf()
initialized ? how does it even know the hive port and config settings ? is
it hardcoded to /etc/hive/conf/hive-site.xml ?
thanks
Yang
Re: HiveMetaStoreClient only sees one of my DBs ?
Posted by Yang <te...@gmail.com>.
thanks, I fixed it. it turns out that I need to put my hive-site.xml into
classpath, without this, it still mysteriously works and somehow gave me a
"default" db. (I wish it had given a more explicit error )
On Mon, Dec 30, 2013 at 12:42 PM, java8964 <ja...@hotmail.com> wrote:
> Best mailing list for this question is hive, but I will try to give my
> guess here anyway.
>
> If you only see 'default' database, most likely you are using hive
> 'LocalMetaStore'. For helping yourself to find out the problem, try to find
> out following information:
>
> 1) What kind of Hive metastore you are using? Did you configure any thing?
> Most widely used are Derby or Mysql.
> 2) If you are using a remote Hive metastore, did the hive-conf folder
> passed to the class path of your Java program? From the following log, it
> looks like wrong hive conf files are passed in:
>
> [2013-12-30 11:44:03,261] INFO ugi=myuser_name ip=unknown-ip-addr
> cmd=get_all_databases
>
> Yong
>
> ------------------------------
> From: teddyyyy123@gmail.com
> Date: Mon, 30 Dec 2013 11:48:55 -0800
> Subject: HiveMetaStoreClient only sees one of my DBs ?
> To: hive-user@hadoop.apache.org
>
>
> if I log into my hive shell, do "show databases;" , I see many DBs:
>
> Logging initialized using configuration in
> file:/etc/hive/conf/hive-log4j.properties
> hive> show databases;
> OK
> conf
> confnew
> default
> money
> testdb
> Time taken: 1.57 seconds, Fetched: 6 row(s)
>
>
> but somehow if I run the following java code using the client API, it only
> shows "default"
> public static void main(String args[]) throws MetaException {
> HiveMetaStoreClient cli = new HiveMetaStoreClient(new HiveConf());
> for(String db: cli.getAllDatabases())
> System.out.println(db);
> }
>
>
> stence)
> [2013-12-30 11:44:02,900] INFO Validating 2 unique key(s) for table DBS
> (DataNucleus.Datastore.Schema)
> [2013-12-30 11:44:02,913] INFO Validating 0 foreign key(s) for table DBS
> (DataNucleus.Datastore.Schema)
> [2013-12-30 11:44:02,919] INFO Validating 2 index(es) for table DBS
> (DataNucleus.Datastore.Schema)
> [2013-12-30 11:44:02,926] INFO Validating 1 unique key(s) for table
> DATABASE_PARAMS (DataNucleus.Datastore.Schema)
> [2013-12-30 11:44:02,937] INFO Validating 1 foreign key(s) for table
> DATABASE_PARAMS (DataNucleus.Datastore.Schema)
> [2013-12-30 11:44:02,942] INFO Validating 2 index(es) for table
> DATABASE_PARAMS (DataNucleus.Datastore.Schema)
> [2013-12-30 11:44:03,087] INFO Listener found initialisation for
> persistable class org.apache.hadoop.hive.metastore.model.MDatabase
> (DataNucleus.MetaData)
> [2013-12-30 11:44:03,211] INFO 0: get_all_databases
> (org.apache.hadoop.hive.metastore.HiveMetaStore)
> [2013-12-30 11:44:03,261] INFO ugi=myuser_name ip=unknown-ip-addr
> cmd=get_all_databases
> (org.apache.hadoop.hive.metastore.HiveMetaStore.audit)
> default
>
>
>
>
>
>
> why is it showing only 1 db? what setttings of default are different from
> the others to enable it to be shown? also I wonder how is that HiveConf()
> initialized ? how does it even know the hive port and config settings ? is
> it hardcoded to /etc/hive/conf/hive-site.xml ?
>
>
> thanks
> Yang
>
>
>
>
>
>
RE: HiveMetaStoreClient only sees one of my DBs ?
Posted by java8964 <ja...@hotmail.com>.
Best mailing list for this question is hive, but I will try to give my guess here anyway.
If you only see 'default' database, most likely you are using hive 'LocalMetaStore'. For helping yourself to find out the problem, try to find out following information:
1) What kind of Hive metastore you are using? Did you configure any thing? Most widely used are Derby or Mysql. 2) If you are using a remote Hive metastore, did the hive-conf folder passed to the class path of your Java program? From the following log, it looks like wrong hive conf files are passed in:
[2013-12-30 11:44:03,261] INFO ugi=myuser_name ip=unknown-ip-addr cmd=get_all_databases
Yong
From: teddyyyy123@gmail.com
Date: Mon, 30 Dec 2013 11:48:55 -0800
Subject: HiveMetaStoreClient only sees one of my DBs ?
To: hive-user@hadoop.apache.org
if I log into my hive shell, do "show databases;" , I see many DBs:
Logging initialized using configuration in file:/etc/hive/conf/hive-log4j.propertieshive> show databases;
OKconfconfnewdefault
moneytestdbTime taken: 1.57 seconds, Fetched: 6 row(s)
but somehow if I run the following java code using the client API, it only shows "default"
public static void main(String args[]) throws MetaException { HiveMetaStoreClient cli = new HiveMetaStoreClient(new HiveConf());
for(String db: cli.getAllDatabases()) System.out.println(db);
}
stence)[2013-12-30 11:44:02,900] INFO Validating 2 unique key(s) for table DBS (DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,913] INFO Validating 0 foreign key(s) for table DBS (DataNucleus.Datastore.Schema)[2013-12-30 11:44:02,919] INFO Validating 2 index(es) for table DBS (DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,926] INFO Validating 1 unique key(s) for table DATABASE_PARAMS (DataNucleus.Datastore.Schema)[2013-12-30 11:44:02,937] INFO Validating 1 foreign key(s) for table DATABASE_PARAMS (DataNucleus.Datastore.Schema)
[2013-12-30 11:44:02,942] INFO Validating 2 index(es) for table DATABASE_PARAMS (DataNucleus.Datastore.Schema)[2013-12-30 11:44:03,087] INFO Listener found initialisation for persistable class org.apache.hadoop.hive.metastore.model.MDatabase (DataNucleus.MetaData)
[2013-12-30 11:44:03,211] INFO 0: get_all_databases (org.apache.hadoop.hive.metastore.HiveMetaStore)[2013-12-30 11:44:03,261] INFO ugi=myuser_name ip=unknown-ip-addr cmd=get_all_databases (org.apache.hadoop.hive.metastore.HiveMetaStore.audit)
default
why is it showing only 1 db? what setttings of default are different from the others to enable it to be shown? also I wonder how is that HiveConf() initialized ? how does it even know the hive port and config settings ? is it hardcoded to /etc/hive/conf/hive-site.xml ?
thanksYang