You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by yogendra reddy <yo...@gmail.com> on 2014/12/01 06:19:13 UTC
hive transactions
Hi All,
I'm caught up with the error pasted below after trying to use hive
transaction feature.I have followed this wiki to set required
configuration.
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
set hive.support.concurrency=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.compactor.initiator.on=true;
set hive.compactor.worker.threads=1;
After setting the above I just ran this command "describe formatted
acidtest;"
any pointers as to why this occurs?
FAILED: Error in acquiring locks: Error communicating with the metastore
14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks: Error
communicating with the metastore
org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
the metastore
at
org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
at
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
at
org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.thrift.TApplicationException: Internal error
processing lock
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy9.lock(Unknown Source)
at
org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
... 17 more
p.s : Before setting the transaction specific properties I was able to run
hive queries successfully
Thanks,
Yogendra
Re: hive transactions
Posted by Alan Gates <ga...@hortonworks.com>.
Can you post the log from your metastore at the same time as this log
from your client? That should tell what problem the metastore is seeing
to cause the lock request to fail. Note that you can't attach your log
to mail sent to the list. So you can cut and paste any relevant errors
or post part somewhere public and send the link.
Alan.
> Yogendra reddy <ma...@gmail.com>
> December 3, 2014 at 18:28
> Thanks Alan, I'm able to see TXNS table and yes I have upgraded from
> 0.13 to 0.14.
>
> any idea as to wht else might cause this issue?
>
> Thanks,
> Yogendra
> ------------------------------------------------------------------------
> From: Alan Gates <ma...@hortonworks.com>
> Sent: 12/3/2014 9:51 AM
> To: user@hive.apache.org <ma...@hive.apache.org>
> Subject: Re: hive transactions
>
> Usually this means the metastore database was not properly upgraded.
> You can check this by logging into your RDBMS and listing all the
> tables. If you see a table named TXNS then the proper tables exist.
>
> Did you install Hive 0.14 fresh or upgrade from a previous version?
> If you upgraded did you run the upgrade scripts? In the Hive 0.12 to
> Hive 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql,
> which will create these tables. If you're upgrading from 13 to 14
> there shouldn't be a need to create the tables.
>
> Alan.
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
> entity to which it is addressed and may contain information that is
> confidential, privileged and exempt from disclosure under applicable
> law. If the reader of this message is not the intended recipient, you
> are hereby notified that any printing, copying, dissemination,
> distribution, disclosure or forwarding of this communication is
> strictly prohibited. If you have received this communication in error,
> please contact the sender immediately and delete it from your system.
> Thank You.
> Alan Gates <ma...@hortonworks.com>
> December 2, 2014 at 18:07
> Usually this means the metastore database was not properly upgraded.
> You can check this by logging into your RDBMS and listing all the
> tables. If you see a table named TXNS then the proper tables exist.
>
> Did you install Hive 0.14 fresh or upgrade from a previous version?
> If you upgraded did you run the upgrade scripts? In the Hive 0.12 to
> Hive 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql,
> which will create these tables. If you're upgrading from 13 to 14
> there shouldn't be a need to create the tables.
>
> Alan.
>
>
> yogendra reddy <ma...@gmail.com>
> November 30, 2014 at 21:19
> Hi All,
>
> I'm caught up with the error pasted below after trying to use hive
> transaction feature.I have followed this wiki to set required
> configuration.
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>
>
> set hive.support.concurrency=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> set hive.compactor.initiator.on=true;
> set hive.compactor.worker.threads=1;
>
> After setting the above I just ran this command "describe formatted
> acidtest;"
> any pointers as to why this occurs?
>
> FAILED: Error in acquiring locks: Error communicating with the metastore
> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks:
> Error communicating with the metastore
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating
> with the metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
> at
> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.thrift.TApplicationException: Internal error
> processing lock
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
> at com.sun.proxy.$Proxy9.lock(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
> ... 17 more
>
> p.s : Before setting the transaction specific properties I was able to
> run hive queries successfully
>
> Thanks,
> Yogendra
--
Sent with Postbox <http://www.getpostbox.com>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hive transactions
Posted by hadoop hive <ha...@gmail.com>.
Can you just bounce one mysql db
On Dec 4, 2014 9:26 AM, "yogendra reddy" <yo...@gmail.com> wrote:
> Thanks Vikas Srivastava, I do have lot of space available in my metastore
> db
>
>
> Thanks,
> Yogendra
>
> On Thu, Dec 4, 2014 at 9:12 AM, hadoop hive <ha...@gmail.com> wrote:
>
>> Hi yogendra,
>>
>> You need to check if space is available on you backend metadata db server
>> like mysql or oracle server.
>>
>> It might be possible the your queries are getting stuck becoz there is no
>> enough space for mysql logging.
>>
>> Thanks
>> Vikas Srivastava
>> On Dec 4, 2014 7:59 AM, "Yogendra reddy" <yo...@gmail.com> wrote:
>>
>>> Thanks Alan, I'm able to see TXNS table and yes I have upgraded from
>>> 0.13 to 0.14.
>>>
>>> any idea as to wht else might cause this issue?
>>>
>>> Thanks,
>>> Yogendra
>>> ------------------------------
>>> From: Alan Gates <ga...@hortonworks.com>
>>> Sent: 12/3/2014 9:51 AM
>>> To: user@hive.apache.org
>>> Subject: Re: hive transactions
>>>
>>> Usually this means the metastore database was not properly upgraded.
>>> You can check this by logging into your RDBMS and listing all the tables.
>>> If you see a table named TXNS then the proper tables exist.
>>>
>>> Did you install Hive 0.14 fresh or upgrade from a previous version? If
>>> you upgraded did you run the upgrade scripts? In the Hive 0.12 to Hive
>>> 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which will
>>> create these tables. If you're upgrading from 13 to 14 there shouldn't be
>>> a need to create the tables.
>>>
>>> Alan.
>>>
>>> yogendra reddy <yo...@gmail.com>
>>> November 30, 2014 at 21:19
>>> Hi All,
>>>
>>> I'm caught up with the error pasted below after trying to use hive
>>> transaction feature.I have followed this wiki to set required
>>> configuration.
>>> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>>>
>>>
>>> set hive.support.concurrency=true;
>>> set hive.exec.dynamic.partition.mode=nonstrict;
>>> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
>>> set hive.compactor.initiator.on=true;
>>> set hive.compactor.worker.threads=1;
>>>
>>> After setting the above I just ran this command "describe formatted
>>> acidtest;"
>>> any pointers as to why this occurs?
>>>
>>> FAILED: Error in acquiring locks: Error communicating with the metastore
>>> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks:
>>> Error communicating with the metastore
>>> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating
>>> with the metastore
>>> at
>>> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
>>> at
>>> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
>>> at
>>> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
>>> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
>>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
>>> at
>>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
>>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
>>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>>> Caused by: org.apache.thrift.TApplicationException: Internal error
>>> processing lock
>>> at
>>> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
>>> at
>>> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
>>> at
>>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
>>> at
>>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
>>> at
>>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
>>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at
>>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
>>> at com.sun.proxy.$Proxy9.lock(Unknown Source)
>>> at
>>> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
>>> ... 17 more
>>>
>>> p.s : Before setting the transaction specific properties I was able to
>>> run hive queries successfully
>>>
>>> Thanks,
>>> Yogendra
>>>
>>>
>>> --
>>> Sent with Postbox <http://www.getpostbox.com>
>>>
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>>> to which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>>
>>
>
Re: hive transactions
Posted by yogendra reddy <yo...@gmail.com>.
Thanks Vikas Srivastava, I do have lot of space available in my metastore db
Thanks,
Yogendra
On Thu, Dec 4, 2014 at 9:12 AM, hadoop hive <ha...@gmail.com> wrote:
> Hi yogendra,
>
> You need to check if space is available on you backend metadata db server
> like mysql or oracle server.
>
> It might be possible the your queries are getting stuck becoz there is no
> enough space for mysql logging.
>
> Thanks
> Vikas Srivastava
> On Dec 4, 2014 7:59 AM, "Yogendra reddy" <yo...@gmail.com> wrote:
>
>> Thanks Alan, I'm able to see TXNS table and yes I have upgraded from 0.13
>> to 0.14.
>>
>> any idea as to wht else might cause this issue?
>>
>> Thanks,
>> Yogendra
>> ------------------------------
>> From: Alan Gates <ga...@hortonworks.com>
>> Sent: 12/3/2014 9:51 AM
>> To: user@hive.apache.org
>> Subject: Re: hive transactions
>>
>> Usually this means the metastore database was not properly upgraded. You
>> can check this by logging into your RDBMS and listing all the tables. If
>> you see a table named TXNS then the proper tables exist.
>>
>> Did you install Hive 0.14 fresh or upgrade from a previous version? If
>> you upgraded did you run the upgrade scripts? In the Hive 0.12 to Hive
>> 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which will
>> create these tables. If you're upgrading from 13 to 14 there shouldn't be
>> a need to create the tables.
>>
>> Alan.
>>
>> yogendra reddy <yo...@gmail.com>
>> November 30, 2014 at 21:19
>> Hi All,
>>
>> I'm caught up with the error pasted below after trying to use hive
>> transaction feature.I have followed this wiki to set required
>> configuration.
>> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>>
>>
>> set hive.support.concurrency=true;
>> set hive.exec.dynamic.partition.mode=nonstrict;
>> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
>> set hive.compactor.initiator.on=true;
>> set hive.compactor.worker.threads=1;
>>
>> After setting the above I just ran this command "describe formatted
>> acidtest;"
>> any pointers as to why this occurs?
>>
>> FAILED: Error in acquiring locks: Error communicating with the metastore
>> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks:
>> Error communicating with the metastore
>> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
>> the metastore
>> at
>> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
>> at
>> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
>> at
>> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
>> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>> Caused by: org.apache.thrift.TApplicationException: Internal error
>> processing lock
>> at
>> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
>> at
>> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
>> at
>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
>> at
>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
>> at
>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
>> at com.sun.proxy.$Proxy9.lock(Unknown Source)
>> at
>> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
>> ... 17 more
>>
>> p.s : Before setting the transaction specific properties I was able to
>> run hive queries successfully
>>
>> Thanks,
>> Yogendra
>>
>>
>> --
>> Sent with Postbox <http://www.getpostbox.com>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
RE: hive transactions
Posted by hadoop hive <ha...@gmail.com>.
Hi yogendra,
You need to check if space is available on you backend metadata db server
like mysql or oracle server.
It might be possible the your queries are getting stuck becoz there is no
enough space for mysql logging.
Thanks
Vikas Srivastava
On Dec 4, 2014 7:59 AM, "Yogendra reddy" <yo...@gmail.com> wrote:
> Thanks Alan, I'm able to see TXNS table and yes I have upgraded from 0.13
> to 0.14.
>
> any idea as to wht else might cause this issue?
>
> Thanks,
> Yogendra
> ------------------------------
> From: Alan Gates <ga...@hortonworks.com>
> Sent: 12/3/2014 9:51 AM
> To: user@hive.apache.org
> Subject: Re: hive transactions
>
> Usually this means the metastore database was not properly upgraded. You
> can check this by logging into your RDBMS and listing all the tables. If
> you see a table named TXNS then the proper tables exist.
>
> Did you install Hive 0.14 fresh or upgrade from a previous version? If
> you upgraded did you run the upgrade scripts? In the Hive 0.12 to Hive
> 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which will
> create these tables. If you're upgrading from 13 to 14 there shouldn't be
> a need to create the tables.
>
> Alan.
>
> yogendra reddy <yo...@gmail.com>
> November 30, 2014 at 21:19
> Hi All,
>
> I'm caught up with the error pasted below after trying to use hive
> transaction feature.I have followed this wiki to set required
> configuration.
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>
>
> set hive.support.concurrency=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> set hive.compactor.initiator.on=true;
> set hive.compactor.worker.threads=1;
>
> After setting the above I just ran this command "describe formatted
> acidtest;"
> any pointers as to why this occurs?
>
> FAILED: Error in acquiring locks: Error communicating with the metastore
> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks: Error
> communicating with the metastore
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
> the metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
> at
> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.thrift.TApplicationException: Internal error
> processing lock
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
> at com.sun.proxy.$Proxy9.lock(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
> ... 17 more
>
> p.s : Before setting the transaction specific properties I was able to run
> hive queries successfully
>
> Thanks,
> Yogendra
>
>
> --
> Sent with Postbox <http://www.getpostbox.com>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
RE: hive transactions
Posted by Yogendra reddy <yo...@gmail.com>.
Thanks Alan, I'm able to see TXNS table and yes I have upgraded from 0.13 to 0.14.
any idea as to wht else might cause this issue?
Thanks,
Yogendra
-----Original Message-----
From: "Alan Gates" <ga...@hortonworks.com>
Sent: 12/3/2014 9:51 AM
To: "user@hive.apache.org" <us...@hive.apache.org>
Subject: Re: hive transactions
Usually this means the metastore database was not properly upgraded. You can check this by logging into your RDBMS and listing all the tables. If you see a table named TXNS then the proper tables exist.
Did you install Hive 0.14 fresh or upgrade from a previous version? If you upgraded did you run the upgrade scripts? In the Hive 0.12 to Hive 0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which will create these tables. If you're upgrading from 13 to 14 there shouldn't be a need to create the tables.
Alan.
yogendra reddy
November 30, 2014 at 21:19
Hi All,
I'm caught up with the error pasted below after trying to use hive transaction feature.I have followed this wiki to set required configuration. https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
set hive.support.concurrency=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.compactor.initiator.on=true;
set hive.compactor.worker.threads=1;
After setting the above I just ran this command "describe formatted acidtest;"
any pointers as to why this occurs?
FAILED: Error in acquiring locks: Error communicating with the metastore
14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks: Error communicating with the metastore
org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore
at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.thrift.TApplicationException: Internal error processing lock
at org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
at com.sun.proxy.$Proxy9.lock(Unknown Source)
at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
... 17 more
p.s : Before setting the transaction specific properties I was able to run hive queries successfully
Thanks,
Yogendra
--
Sent with Postbox
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: hive transactions
Posted by Alan Gates <ga...@hortonworks.com>.
Usually this means the metastore database was not properly upgraded.
You can check this by logging into your RDBMS and listing all the
tables. If you see a table named TXNS then the proper tables exist.
Did you install Hive 0.14 fresh or upgrade from a previous version? If
you upgraded did you run the upgrade scripts? In the Hive 0.12 to Hive
0.13 upgrade script it calls hive-txn-schema-0.13.0.mysql.sql, which
will create these tables. If you're upgrading from 13 to 14 there
shouldn't be a need to create the tables.
Alan.
> yogendra reddy <ma...@gmail.com>
> November 30, 2014 at 21:19
> Hi All,
>
> I'm caught up with the error pasted below after trying to use hive
> transaction feature.I have followed this wiki to set required
> configuration.
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>
>
> set hive.support.concurrency=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> set hive.compactor.initiator.on=true;
> set hive.compactor.worker.threads=1;
>
> After setting the above I just ran this command "describe formatted
> acidtest;"
> any pointers as to why this occurs?
>
> FAILED: Error in acquiring locks: Error communicating with the metastore
> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks:
> Error communicating with the metastore
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating
> with the metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
> at
> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.thrift.TApplicationException: Internal error
> processing lock
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
> at com.sun.proxy.$Proxy9.lock(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
> ... 17 more
>
> p.s : Before setting the transaction specific properties I was able to
> run hive queries successfully
>
> Thanks,
> Yogendra
--
Sent with Postbox <http://www.getpostbox.com>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: hive transactions
Posted by yogendra reddy <yo...@gmail.com>.
It is running. As I mentioned earlier only after setting transaction
properties I'm getting the error, was able to run normal hive queries if I
disable transaction related properties!
On Mon, Dec 1, 2014 at 1:52 PM, unmesha sreeveni <un...@gmail.com>
wrote:
>
> On Mon, Dec 1, 2014 at 12:31 PM, yogendra reddy <yo...@gmail.com>
> wrote:
>
>> WARN metastore.RetryingMetaStoreClient: MetaStoreClient lost connection.
>>
>
> It looks like hive-metastore service is not running.
> Can you please check the same?
>
>
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: hive transactions
Posted by unmesha sreeveni <un...@gmail.com>.
On Mon, Dec 1, 2014 at 12:31 PM, yogendra reddy <yo...@gmail.com>
wrote:
> WARN metastore.RetryingMetaStoreClient: MetaStoreClient lost connection.
>
It looks like hive-metastore service is not running.
Can you please check the same?
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: hive transactions
Posted by yogendra reddy <yo...@gmail.com>.
Thanks unmesha for the detailed article,
I have set the required configurations correctly but still the error
remains!.. I also see this warning in the trace [ WARN
metastore.RetryingMetaStoreClient: MetaStoreClient lost connection.
Attempting to reconnect.] don't know why this only occurs when I set the
transaction configurations!
Thanks,
Yogendra
On Mon, Dec 1, 2014 at 11:49 AM, unmesha sreeveni <un...@gmail.com>
wrote:
> Hi yogendra,
>
> I have tried the same thing and I wrote a doc on ACID properties.
>
>
> http://unmeshasreeveni.blogspot.in/2014/11/updatedeleteinsert-in-hive-0140.html
>
> Please follow that steps.
>
> let me know if that stucks you in any point
>
> On Mon, Dec 1, 2014 at 10:49 AM, yogendra reddy <yo...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> I'm caught up with the error pasted below after trying to use hive
>> transaction feature.I have followed this wiki to set required
>> configuration.
>> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>>
>>
>> set hive.support.concurrency=true;
>> set hive.exec.dynamic.partition.mode=nonstrict;
>> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
>> set hive.compactor.initiator.on=true;
>> set hive.compactor.worker.threads=1;
>>
>> After setting the above I just ran this command "describe formatted
>> acidtest;"
>> any pointers as to why this occurs?
>>
>> FAILED: Error in acquiring locks: Error communicating with the metastore
>> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks:
>> Error communicating with the metastore
>> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
>> the metastore
>> at
>> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
>> at
>> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
>> at
>> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
>> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
>> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
>> at
>> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
>> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
>> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>> Caused by: org.apache.thrift.TApplicationException: Internal error
>> processing lock
>> at
>> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
>> at
>> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
>> at
>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
>> at
>> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
>> at
>> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
>> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at
>> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
>> at com.sun.proxy.$Proxy9.lock(Unknown Source)
>> at
>> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
>> ... 17 more
>>
>> p.s : Before setting the transaction specific properties I was able to
>> run hive queries successfully
>>
>> Thanks,
>> Yogendra
>>
>
>
>
> --
> *Thanks & Regards *
>
>
> *Unmesha Sreeveni U.B*
> *Hadoop, Bigdata Developer*
> *Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
> http://www.unmeshasreeveni.blogspot.in/
>
>
>
Re: hive transactions
Posted by unmesha sreeveni <un...@gmail.com>.
Hi yogendra,
I have tried the same thing and I wrote a doc on ACID properties.
http://unmeshasreeveni.blogspot.in/2014/11/updatedeleteinsert-in-hive-0140.html
Please follow that steps.
let me know if that stucks you in any point
On Mon, Dec 1, 2014 at 10:49 AM, yogendra reddy <yo...@gmail.com>
wrote:
> Hi All,
>
> I'm caught up with the error pasted below after trying to use hive
> transaction feature.I have followed this wiki to set required
> configuration.
> https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
>
>
> set hive.support.concurrency=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> set hive.compactor.initiator.on=true;
> set hive.compactor.worker.threads=1;
>
> After setting the above I just ran this command "describe formatted
> acidtest;"
> any pointers as to why this occurs?
>
> FAILED: Error in acquiring locks: Error communicating with the metastore
> 14/12/01 05:02:05 ERROR ql.Driver: FAILED: Error in acquiring locks: Error
> communicating with the metastore
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
> the metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:101)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:202)
> at
> org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:939)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1166)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:247)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.thrift.TApplicationException: Internal error
> processing lock
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
> at com.sun.proxy.$Proxy9.lock(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:80)
> ... 17 more
>
> p.s : Before setting the transaction specific properties I was able to run
> hive queries successfully
>
> Thanks,
> Yogendra
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Centre for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/