You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sumit Khanna <su...@askme.in> on 2016/08/08 11:34:51 UTC
hive session drops tables when restarted
Hello,
I am facing a weird issue with hive. Initially we had hive configured
against the default derby, and hardly select * / raw columns were working
for my parquet tables.
However, having configured it against mysql metastore db now, the all my
queries involving min / max / order by / where etc which involve a reducer
stage ( parallel/distributed calls to hive db/hdfs) are working too.
However, the problem that still persists here is that when I quit the hive
session, and then restart it, all my tables are gone, as in show tables
yields nothing.
I thence have to manually fire create table and alter table add partition
queries and run the same query thenafter to get the desired results.
Can anyone help ? the problem faced being that hive drops / loses all the
tables when the session is restarted.
Thanks,
Sumit
Re: hive session drops tables when restarted
Posted by Sumit Khanna <su...@askme.in>.
Hello Mich,
Well there was a little problem, as of now it asks for adding mysql
connector to classpath. I simply added the mysql connector jars to
/opt/hive/lib/
I believe that should have sufficed isnt it?
ie.
mysql-metadata-storage-0.9.1.1.tar.gz
mysql-connector-java-5.1.39.tar.gz
it still gives me the same error :
Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an
error : The specified datastore driver ("com.mysql.jdbc.Driver") was not
found in the CLASSPATH.
. need I edit anything in the hive-site.xml file
Thanks,
On Tue, Aug 9, 2016 at 8:33 AM, Sumit Khanna <su...@askme.in> wrote:
>
> have you configured your hive.site.xml to refer to the new metadata on
> MySQL ?
>
> <property>
>
> <name>javax.jdo.option.ConnectionURL</name>
>
> <value>jdbc:mysql://localhost/metastore?
> createDatabaseIfNotExist=true</value>
>
> <description>metadata is stored in a MySQL server</description>
>
> </property>
>
> <property>
>
> <name>javax.jdo.option.ConnectionDriverName</name>
>
> <value>com.mysql.jdbc.Driver</value>
>
> <description>MySQL JDBC driver class</description>
>
> </property>
>
> <property>
>
> <name>javax.jdo.option.ConnectionUserName</name>
>
> <value>hiveuser</value>
>
> <description>user name for connecting to mysql server</description>
>
> </property>
>
> <property>
>
> <name>javax.jdo.option.ConnectionPassword</name>
>
> <value>diracdelta</value>
>
> <description>password for connecting to mysql server</description>
>
> </property>
>
> can you log in to MySQL database and look at table TBLS see if there are
> any tables
>
> yes but there arent any tables.
>
>
> 2) Have you started the metastore connection
>
> And also Hiver thrift server
>
> Can you see them
>
>
> Yes, they are all up. but now, even the hive session isnt starting up.
>
>
> Thanks,
>
> On Mon, Aug 8, 2016 at 7:48 PM, Mich Talebzadeh <mich.talebzadeh@gmail.com
> > wrote:
>
>> have you configured your hive.site.xml to refer to the new metadata on
>> MySQL ?
>>
>> Mine is on oracle
>>
>> <property>
>> <name>javax.jdo.option.ConnectionURL</name>
>> <value>jdbc:oracle:thin:@rhes564:1521:mydb12</value>
>> <description>JDBC connect string for a JDBC metastore</description>
>> </property>
>>
>> and username and password
>>
>> can you log in to MySQL database and look at table TBLS see if there are
>> any tables
>>
>> Also look to see the connections there
>>
>> 2) Have you started the metastore connection
>>
>> $HIVE_HOME/bin/hive --service metastore &
>>
>>
>> And also Hiver thrift server
>>
>>
>> $HIVE_HOME/bin/hiveserver2 &
>>
>> be default they run on port 9083 and 10000
>>
>> Can you see them
>>
>> netstat -pltenp 'Local|1000|9083'
>>
>>
>> HTH
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>> On 8 August 2016 at 12:34, Sumit Khanna <su...@askme.in> wrote:
>>
>>> Hello,
>>>
>>> I am facing a weird issue with hive. Initially we had hive configured
>>> against the default derby, and hardly select * / raw columns were working
>>> for my parquet tables.
>>>
>>> However, having configured it against mysql metastore db now, the all my
>>> queries involving min / max / order by / where etc which involve a reducer
>>> stage ( parallel/distributed calls to hive db/hdfs) are working too.
>>>
>>> However, the problem that still persists here is that when I quit the
>>> hive session, and then restart it, all my tables are gone, as in show
>>> tables yields nothing.
>>>
>>> I thence have to manually fire create table and alter table add
>>> partition queries and run the same query thenafter to get the desired
>>> results.
>>>
>>> Can anyone help ? the problem faced being that hive drops / loses all
>>> the tables when the session is restarted.
>>>
>>> Thanks,
>>> Sumit
>>>
>>>
>>
>
Re: hive session drops tables when restarted
Posted by Sumit Khanna <su...@askme.in>.
have you configured your hive.site.xml to refer to the new metadata on
MySQL ?
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost/metastore?createDatabaseIfNotExist=true</value>
<description>metadata is stored in a MySQL server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>MySQL JDBC driver class</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hiveuser</value>
<description>user name for connecting to mysql server</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>diracdelta</value>
<description>password for connecting to mysql server</description>
</property>
can you log in to MySQL database and look at table TBLS see if there are
any tables
yes but there arent any tables.
2) Have you started the metastore connection
And also Hiver thrift server
Can you see them
Yes, they are all up. but now, even the hive session isnt starting up.
Thanks,
On Mon, Aug 8, 2016 at 7:48 PM, Mich Talebzadeh <mi...@gmail.com>
wrote:
> have you configured your hive.site.xml to refer to the new metadata on
> MySQL ?
>
> Mine is on oracle
>
> <property>
> <name>javax.jdo.option.ConnectionURL</name>
> <value>jdbc:oracle:thin:@rhes564:1521:mydb12</value>
> <description>JDBC connect string for a JDBC metastore</description>
> </property>
>
> and username and password
>
> can you log in to MySQL database and look at table TBLS see if there are
> any tables
>
> Also look to see the connections there
>
> 2) Have you started the metastore connection
>
> $HIVE_HOME/bin/hive --service metastore &
>
>
> And also Hiver thrift server
>
>
> $HIVE_HOME/bin/hiveserver2 &
>
> be default they run on port 9083 and 10000
>
> Can you see them
>
> netstat -pltenp 'Local|1000|9083'
>
>
> HTH
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 8 August 2016 at 12:34, Sumit Khanna <su...@askme.in> wrote:
>
>> Hello,
>>
>> I am facing a weird issue with hive. Initially we had hive configured
>> against the default derby, and hardly select * / raw columns were working
>> for my parquet tables.
>>
>> However, having configured it against mysql metastore db now, the all my
>> queries involving min / max / order by / where etc which involve a reducer
>> stage ( parallel/distributed calls to hive db/hdfs) are working too.
>>
>> However, the problem that still persists here is that when I quit the
>> hive session, and then restart it, all my tables are gone, as in show
>> tables yields nothing.
>>
>> I thence have to manually fire create table and alter table add partition
>> queries and run the same query thenafter to get the desired results.
>>
>> Can anyone help ? the problem faced being that hive drops / loses all
>> the tables when the session is restarted.
>>
>> Thanks,
>> Sumit
>>
>>
>
Re: hive session drops tables when restarted
Posted by Mich Talebzadeh <mi...@gmail.com>.
have you configured your hive.site.xml to refer to the new metadata on
MySQL ?
Mine is on oracle
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:oracle:thin:@rhes564:1521:mydb12</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
and username and password
can you log in to MySQL database and look at table TBLS see if there are
any tables
Also look to see the connections there
2) Have you started the metastore connection
$HIVE_HOME/bin/hive --service metastore &
And also Hiver thrift server
$HIVE_HOME/bin/hiveserver2 &
be default they run on port 9083 and 10000
Can you see them
netstat -pltenp 'Local|1000|9083'
HTH
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On 8 August 2016 at 12:34, Sumit Khanna <su...@askme.in> wrote:
> Hello,
>
> I am facing a weird issue with hive. Initially we had hive configured
> against the default derby, and hardly select * / raw columns were working
> for my parquet tables.
>
> However, having configured it against mysql metastore db now, the all my
> queries involving min / max / order by / where etc which involve a reducer
> stage ( parallel/distributed calls to hive db/hdfs) are working too.
>
> However, the problem that still persists here is that when I quit the hive
> session, and then restart it, all my tables are gone, as in show tables
> yields nothing.
>
> I thence have to manually fire create table and alter table add partition
> queries and run the same query thenafter to get the desired results.
>
> Can anyone help ? the problem faced being that hive drops / loses all the
> tables when the session is restarted.
>
> Thanks,
> Sumit
>
>