You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sarath Chandra <sa...@algofusiontech.com> on 2015/08/03 15:29:38 UTC
Error communicating with metastore
Hi All,
Earlier I was using hive 0.13.0 and now trying to migrate to latest version
to utilize the transaction support introduced from hive 0.14.0.
I downloaded hive 1.2.1, created a metastore in oracle database and
provided all the required configuration parameters in conf/hive-site.xml to
enable transactions. For the parameter "hive.txn.manager" given the value
"org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>From the hive prompt when I fire the command "show tables;" I'm getting the
below exception -
*FAILED: Error in determining valid transactions: Error communicating with
the metastore*
But if disable the "hive.txn.manager" parameter in hive-site.xml then the
command works fine.
Is there anything else to be configured which I'm missing?
Thanks & Regards,
Sarath.
Re: Error communicating with metastore
Posted by Sarath Chandra <sa...@algofusiontech.com>.
Thanks for all who replied.
After several retries in vain, I felt the issue might be because of version
mismatch. So I took Hadoop 2.7.1, Spark 1.3.1 and Hive 1.2.1. Tried with
these combination and I could successfully test the hive transactions
without any issue.
Regards,
Sarath.
On Fri, Aug 7, 2015 at 10:32 PM, Alan Gates <al...@gmail.com> wrote:
> What version of Hive did you say you were using? In 0.14 we switched to
> make sure all jdbc connections are either serializable or read_committed,
> yet the error message below seems to indicate you're seeing a connection
> request that doesn't match this.
>
> The only other thing I know to try i switching the JDBC pool provider. By
> default Hive uses bonecp (which is what you're using according to the
> logs). I have seen issues with Oracle 12 and bonecp that went away when
> you switch to using dbcp. You can try this by setting
> datanucleus.connectionPoolingType to dbcp on your thrift metastore and
> then restarting the thrift metastore.
>
> Alan.
>
> Sarath Chandra <sa...@algofusiontech.com>
> August 7, 2015 at 6:13
> Thanks Eugene, Alan.
>
> @Alan,
> As suggested checked the logs, here is what I found -
>
> - On starting metastore server, I'm seeing following messages in the
> log file -
>
> *2015-08-07 18:32:56,678 ERROR [Thread-7]: compactor.Initiator
> (Initiator.java:run(134)) - Caught an exception in the main loop of
> compactor initiator, exiting MetaException(message:Unable to get jdbc
> connection from pool, READ_COMMITTED and SERIALIZABLE are the only valid
> transaction levels)*
> * at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.getDbConn(TxnHandler.java:811)*
> * at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.revokeFromLocalWorkers(CompactionTxnHandler.java:443)*
> * at
> org.apache.hadoop.hive.ql.txn.compactor.Initiator.recoverFailedCompactions(Initiator.java:147)*
> * at
> org.apache.hadoop.hive.ql.txn.compactor.Initiator.run(Initiator.java:64)*
>
> - On bringing up the hive shell, I get the following messages -
>
> tion - enable connectionWatch for additional debugging assistance or set
> disableConnectionTracking to true to disable this feature entirely.
> 2015-08-07 18:38:51,614 WARN
> [org.spark-project.guava.common.base.internal.Finalizer]:
> bonecp.ConnectionPartition (ConnectionPartition.java:finalizeReferent(162))
> - BoneCP detected an unclosed connection and will now attempt to close it
> for you. You should be closing this connection in your application - enable
> connectionWatch for additional debugging assistance or set
> disableConnectionTracking to true to disable this feature entirely.
> 2015-08-07 18:38:51,768 DEBUG [pool-3-thread-1]: metastore.ObjectStore
> (ObjectStore.java:debugLog(6435)) - Commit transaction: count = 0, isactive
> true at:
>
> org.apache.hadoop.hive.metastore.ObjectStore.getFunctions(ObjectStore.java:6657)
>
> - On firing "show tables" command, I get the following messages in the
> log file -
>
> 2015-08-07 18:41:02,511 INFO [main]: hive.metastore
> (HiveMetaStoreClient.java:open(297)) - Trying to connect to metastore with
> URI thrift://sarath:9083
> 2015-08-07 18:41:02,511 INFO [main]: hive.metastore
> (HiveMetaStoreClient.java:open(385)) - Connected to metastore.
> 2015-08-07 18:41:22,549 ERROR [main]: ql.Driver
> (SessionState.java:printError(545)) - FAILED: Error in determing valid
> transactions: Error communicating with the metastore
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
> the metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:281)
> at
> org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:842)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1036)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
> at
> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_open_txns(ThriftHiveMetastore.java:3367)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_open_txns(ThriftHiveMetastore.java:3355)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1545)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:279)
> ... 15 more
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> ... 24 more
>
> Let me know if there is anything to be taken care in the configuration or
> setup.
>
>
> Alan Gates <al...@gmail.com>
> August 4, 2015 at 16:10
> Ok, the next step is to look at the logs from your Hive metastore server
> and see exactly what's happening. The error you're seeing is from the
> client. On your metastore server there should be logs with the same
> timestamp giving details on why the transaction operation failed.
>
> Alan.
>
> Sarath Chandra <sa...@algofusiontech.com>
> August 3, 2015 at 20:02
> Thanks Alan.
>
> Yes I've run metastore scripts for oracle instance. Infact I've removed my
> previous metastore and created a fresh one by running the schema creation
> script for 1.2.1. I've looked into the new schema and able to see the table
> TXNS. I've also removed the hdfs location "/user/hive/warehouse" and
> created a fresh one.
>
> But still I'm facing this issue.
>
>
>
> Alan Gates <al...@gmail.com>
> August 3, 2015 at 8:29
> Did you run the hive metastore upgrade scripts for your oracle instance?
> This error message usually means the transaction related tables have not
> been created in your database. Somewhere in your distribution there should
> be a set of upgrade scripts. Look for scripts of the form:
>
> scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
>
> You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14,
> 0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added the
> transaction tables as part of upgrading to Hive 0.13. If you did not you
> will need to first run hive-txn-schema-0.13.0.oracle.sql which will create
> the initial transaction tables. You can determine whether this was done by
> looking for a table named TXNS in the hive schema on your Oracle db.
>
> Alan.
>
> Sarath Chandra <sa...@algofusiontech.com>
> August 3, 2015 at 6:29
> Hi All,
>
> Earlier I was using hive 0.13.0 and now trying to migrate to latest
> version to utilize the transaction support introduced from hive 0.14.0.
>
> I downloaded hive 1.2.1, created a metastore in oracle database and
> provided all the required configuration parameters in conf/hive-site.xml to
> enable transactions. For the parameter "hive.txn.manager" given the value
> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>
> From the hive prompt when I fire the command "show tables;" I'm getting
> the below exception -
> *FAILED: Error in determining valid transactions: Error communicating with
> the metastore*
>
> But if disable the "hive.txn.manager" parameter in hive-site.xml then the
> command works fine.
>
> Is there anything else to be configured which I'm missing?
>
> Thanks & Regards,
> Sarath.
>
>
Re: Error communicating with metastore
Posted by Alan Gates <al...@gmail.com>.
What version of Hive did you say you were using? In 0.14 we switched to
make sure all jdbc connections are either serializable or
read_committed, yet the error message below seems to indicate you're
seeing a connection request that doesn't match this.
The only other thing I know to try i switching the JDBC pool provider.
By default Hive uses bonecp (which is what you're using according to the
logs). I have seen issues with Oracle 12 and bonecp that went away when
you switch to using dbcp. You can try this by setting
datanucleus.connectionPoolingTypetodbcp on your thrift metastore and
then restarting the thrift metastore.
Alan.
> Sarath Chandra <ma...@algofusiontech.com>
> August 7, 2015 at 6:13
> Thanks Eugene, Alan.
>
> @Alan,
> As suggested checked the logs, here is what I found -
>
> * On starting metastore server, I'm seeing following messages in the
> log file -
>
> /2015-08-07 18:32:56,678 ERROR [Thread-7]: compactor.Initiator
> (Initiator.java:run(134)) - Caught an exception in the main loop of
> compactor initiator, exiting MetaException(message:Unable to get jdbc
> connection from pool, READ_COMMITTED and SERIALIZABLE are the only
> valid transaction levels)/
> / at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.getDbConn(TxnHandler.java:811)/
> / at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.revokeFromLocalWorkers(CompactionTxnHandler.java:443)/
> / at
> org.apache.hadoop.hive.ql.txn.compactor.Initiator.recoverFailedCompactions(Initiator.java:147)/
> / at
> org.apache.hadoop.hive.ql.txn.compactor.Initiator.run(Initiator.java:64)/
>
> * On bringing up the hive shell, I get the following messages -
>
> tion - enable connectionWatch for additional debugging assistance or
> set disableConnectionTracking to true to disable this feature entirely.
> 2015-08-07 18:38:51,614 WARN
> [org.spark-project.guava.common.base.internal.Finalizer]:
> bonecp.ConnectionPartition
> (ConnectionPartition.java:finalizeReferent(162)) - BoneCP detected an
> unclosed connection and will now attempt to close it for you. You
> should be closing this connection in your application - enable
> connectionWatch for additional debugging assistance or set
> disableConnectionTracking to true to disable this feature entirely.
> 2015-08-07 18:38:51,768 DEBUG [pool-3-thread-1]: metastore.ObjectStore
> (ObjectStore.java:debugLog(6435)) - Commit transaction: count = 0,
> isactive true at:
>
> org.apache.hadoop.hive.metastore.ObjectStore.getFunctions(ObjectStore.java:6657)
>
> * On firing "show tables" command, I get the following messages in
> the log file -
>
> 2015-08-07 18:41:02,511 INFO [main]: hive.metastore
> (HiveMetaStoreClient.java:open(297)) - Trying to connect to metastore
> with URI thrift://sarath:9083
> 2015-08-07 18:41:02,511 INFO [main]: hive.metastore
> (HiveMetaStoreClient.java:open(385)) - Connected to metastore.
> 2015-08-07 18:41:22,549 ERROR [main]: ql.Driver
> (SessionState.java:printError(545)) - FAILED: Error in determing valid
> transactions: Error communicating with the metastore
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating
> with the metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:281)
> at
> org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:842)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1036)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
> at
> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_open_txns(ThriftHiveMetastore.java:3367)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_open_txns(ThriftHiveMetastore.java:3355)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1545)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:279)
> ... 15 more
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> ... 24 more
>
> Let me know if there is anything to be taken care in the configuration
> or setup.
>
>
> Alan Gates <ma...@gmail.com>
> August 4, 2015 at 16:10
> Ok, the next step is to look at the logs from your Hive metastore
> server and see exactly what's happening. The error you're seeing is
> from the client. On your metastore server there should be logs with
> the same timestamp giving details on why the transaction operation failed.
>
> Alan.
>
> Sarath Chandra <ma...@algofusiontech.com>
> August 3, 2015 at 20:02
> Thanks Alan.
>
> Yes I've run metastore scripts for oracle instance. Infact I've
> removed my previous metastore and created a fresh one by running the
> schema creation script for 1.2.1. I've looked into the new schema and
> able to see the table TXNS. I've also removed the hdfs location
> "/user/hive/warehouse" and created a fresh one.
>
> But still I'm facing this issue.
>
>
>
> Alan Gates <ma...@gmail.com>
> August 3, 2015 at 8:29
> Did you run the hive metastore upgrade scripts for your oracle
> instance? This error message usually means the transaction related
> tables have not been created in your database. Somewhere in your
> distribution there should be a set of upgrade scripts. Look for
> scripts of the form:
>
> scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
>
> You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14,
> 0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added
> the transaction tables as part of upgrading to Hive 0.13. If you did
> not you will need to first run hive-txn-schema-0.13.0.oracle.sql which
> will create the initial transaction tables. You can determine whether
> this was done by looking for a table named TXNS in the hive schema on
> your Oracle db.
>
> Alan.
>
> Sarath Chandra <ma...@algofusiontech.com>
> August 3, 2015 at 6:29
> Hi All,
>
> Earlier I was using hive 0.13.0 and now trying to migrate to latest
> version to utilize the transaction support introduced from hive 0.14.0.
>
> I downloaded hive 1.2.1, created a metastore in oracle database and
> provided all the required configuration parameters in
> conf/hive-site.xml to enable transactions. For the parameter
> "hive.txn.manager" given the value
> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>
> From the hive prompt when I fire the command "show tables;" I'm
> getting the below exception -
> /FAILED: Error in determining valid transactions: Error communicating
> with the metastore/
>
> But if disable the "hive.txn.manager" parameter in hive-site.xml then
> the command works fine.
>
> Is there anything else to be configured which I'm missing?
>
> Thanks & Regards,
> Sarath.
Re: Error communicating with metastore
Posted by "kulkarni.swarnim@gmail.com" <ku...@gmail.com>.
Sarath,
I assume that the failure you are seeing doesn't happen immediately? The
current timeout on the client is set to 5 minutes. A socket timeout usually
means that the client timed out before it can even get a response from the
server. So the server could either be very busy doing something if you are
pulling in tons of data and/or you might also be running into this bug[1]
where the HMS connections are leaked.
To start with, can you try bumping up the timeout to like 20 minutes and
see if your queries succeed? This can be done directly from cli via
"set hive.metastore.client.socket.timeout=1200".
[1] https://issues.apache.org/jira/browse/HIVE-10956
On Fri, Aug 7, 2015 at 8:13 AM, Sarath Chandra <
sarathchandra.josyam@algofusiontech.com> wrote:
> Thanks Eugene, Alan.
>
> @Alan,
> As suggested checked the logs, here is what I found -
>
> - On starting metastore server, I'm seeing following messages in the
> log file -
>
> *2015-08-07 18:32:56,678 ERROR [Thread-7]: compactor.Initiator
> (Initiator.java:run(134)) - Caught an exception in the main loop of
> compactor initiator, exiting MetaException(message:Unable to get jdbc
> connection from pool, READ_COMMITTED and SERIALIZABLE are the only valid
> transaction levels)*
> * at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.getDbConn(TxnHandler.java:811)*
> * at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.revokeFromLocalWorkers(CompactionTxnHandler.java:443)*
> * at
> org.apache.hadoop.hive.ql.txn.compactor.Initiator.recoverFailedCompactions(Initiator.java:147)*
> * at
> org.apache.hadoop.hive.ql.txn.compactor.Initiator.run(Initiator.java:64)*
>
> - On bringing up the hive shell, I get the following messages -
>
> tion - enable connectionWatch for additional debugging assistance or set
> disableConnectionTracking to true to disable this feature entirely.
> 2015-08-07 18:38:51,614 WARN
> [org.spark-project.guava.common.base.internal.Finalizer]:
> bonecp.ConnectionPartition (ConnectionPartition.java:finalizeReferent(162))
> - BoneCP detected an unclosed connection and will now attempt to close it
> for you. You should be closing this connection in your application - enable
> connectionWatch for additional debugging assistance or set
> disableConnectionTracking to true to disable this feature entirely.
> 2015-08-07 18:38:51,768 DEBUG [pool-3-thread-1]: metastore.ObjectStore
> (ObjectStore.java:debugLog(6435)) - Commit transaction: count = 0, isactive
> true at:
>
> org.apache.hadoop.hive.metastore.ObjectStore.getFunctions(ObjectStore.java:6657)
>
> - On firing "show tables" command, I get the following messages in the
> log file -
>
> 2015-08-07 18:41:02,511 INFO [main]: hive.metastore
> (HiveMetaStoreClient.java:open(297)) - Trying to connect to metastore with
> URI thrift://sarath:9083
> 2015-08-07 18:41:02,511 INFO [main]: hive.metastore
> (HiveMetaStoreClient.java:open(385)) - Connected to metastore.
> 2015-08-07 18:41:22,549 ERROR [main]: ql.Driver
> (SessionState.java:printError(545)) - FAILED: Error in determing valid
> transactions: Error communicating with the metastore
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
> the metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:281)
> at
> org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:842)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1036)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
> at
> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_open_txns(ThriftHiveMetastore.java:3367)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_open_txns(ThriftHiveMetastore.java:3355)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1545)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:279)
> ... 15 more
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:129)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> ... 24 more
>
> Let me know if there is anything to be taken care in the configuration or
> setup.
>
> On Wed, Aug 5, 2015 at 4:40 AM, Alan Gates <al...@gmail.com> wrote:
>
>> Ok, the next step is to look at the logs from your Hive metastore server
>> and see exactly what's happening. The error you're seeing is from the
>> client. On your metastore server there should be logs with the same
>> timestamp giving details on why the transaction operation failed.
>>
>> Alan.
>>
>> Sarath Chandra <sa...@algofusiontech.com>
>> August 3, 2015 at 20:02
>> Thanks Alan.
>>
>> Yes I've run metastore scripts for oracle instance. Infact I've removed
>> my previous metastore and created a fresh one by running the schema
>> creation script for 1.2.1. I've looked into the new schema and able to see
>> the table TXNS. I've also removed the hdfs location "/user/hive/warehouse"
>> and created a fresh one.
>>
>> But still I'm facing this issue.
>>
>>
>>
>> Alan Gates <al...@gmail.com>
>> August 3, 2015 at 8:29
>> Did you run the hive metastore upgrade scripts for your oracle instance?
>> This error message usually means the transaction related tables have not
>> been created in your database. Somewhere in your distribution there should
>> be a set of upgrade scripts. Look for scripts of the form:
>>
>> scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
>>
>> You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14,
>> 0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added the
>> transaction tables as part of upgrading to Hive 0.13. If you did not you
>> will need to first run hive-txn-schema-0.13.0.oracle.sql which will create
>> the initial transaction tables. You can determine whether this was done by
>> looking for a table named TXNS in the hive schema on your Oracle db.
>>
>> Alan.
>>
>> Sarath Chandra <sa...@algofusiontech.com>
>> August 3, 2015 at 6:29
>> Hi All,
>>
>> Earlier I was using hive 0.13.0 and now trying to migrate to latest
>> version to utilize the transaction support introduced from hive 0.14.0.
>>
>> I downloaded hive 1.2.1, created a metastore in oracle database and
>> provided all the required configuration parameters in conf/hive-site.xml to
>> enable transactions. For the parameter "hive.txn.manager" given the value
>> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>>
>> From the hive prompt when I fire the command "show tables;" I'm getting
>> the below exception -
>> *FAILED: Error in determining valid transactions: Error communicating
>> with the metastore*
>>
>> But if disable the "hive.txn.manager" parameter in hive-site.xml then the
>> command works fine.
>>
>> Is there anything else to be configured which I'm missing?
>>
>> Thanks & Regards,
>> Sarath.
>>
>>
>
--
Swarnim
Re: Error communicating with metastore
Posted by Sarath Chandra <sa...@algofusiontech.com>.
Thanks Eugene, Alan.
@Alan,
As suggested checked the logs, here is what I found -
- On starting metastore server, I'm seeing following messages in the log
file -
*2015-08-07 18:32:56,678 ERROR [Thread-7]: compactor.Initiator
(Initiator.java:run(134)) - Caught an exception in the main loop of
compactor initiator, exiting MetaException(message:Unable to get jdbc
connection from pool, READ_COMMITTED and SERIALIZABLE are the only valid
transaction levels)*
* at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getDbConn(TxnHandler.java:811)*
* at
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.revokeFromLocalWorkers(CompactionTxnHandler.java:443)*
* at
org.apache.hadoop.hive.ql.txn.compactor.Initiator.recoverFailedCompactions(Initiator.java:147)*
* at
org.apache.hadoop.hive.ql.txn.compactor.Initiator.run(Initiator.java:64)*
- On bringing up the hive shell, I get the following messages -
tion - enable connectionWatch for additional debugging assistance or set
disableConnectionTracking to true to disable this feature entirely.
2015-08-07 18:38:51,614 WARN
[org.spark-project.guava.common.base.internal.Finalizer]:
bonecp.ConnectionPartition (ConnectionPartition.java:finalizeReferent(162))
- BoneCP detected an unclosed connection and will now attempt to close it
for you. You should be closing this connection in your application - enable
connectionWatch for additional debugging assistance or set
disableConnectionTracking to true to disable this feature entirely.
2015-08-07 18:38:51,768 DEBUG [pool-3-thread-1]: metastore.ObjectStore
(ObjectStore.java:debugLog(6435)) - Commit transaction: count = 0, isactive
true at:
org.apache.hadoop.hive.metastore.ObjectStore.getFunctions(ObjectStore.java:6657)
- On firing "show tables" command, I get the following messages in the
log file -
2015-08-07 18:41:02,511 INFO [main]: hive.metastore
(HiveMetaStoreClient.java:open(297)) - Trying to connect to metastore with
URI thrift://sarath:9083
2015-08-07 18:41:02,511 INFO [main]: hive.metastore
(HiveMetaStoreClient.java:open(385)) - Connected to metastore.
2015-08-07 18:41:22,549 ERROR [main]: ql.Driver
(SessionState.java:printError(545)) - FAILED: Error in determing valid
transactions: Error communicating with the metastore
org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with
the metastore
at
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:281)
at org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:842)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1036)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:911)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:901)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
at
org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_open_txns(ThriftHiveMetastore.java:3367)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_open_txns(ThriftHiveMetastore.java:3355)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidTxns(HiveMetaStoreClient.java:1545)
at
org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:279)
... 15 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
... 24 more
Let me know if there is anything to be taken care in the configuration or
setup.
On Wed, Aug 5, 2015 at 4:40 AM, Alan Gates <al...@gmail.com> wrote:
> Ok, the next step is to look at the logs from your Hive metastore server
> and see exactly what's happening. The error you're seeing is from the
> client. On your metastore server there should be logs with the same
> timestamp giving details on why the transaction operation failed.
>
> Alan.
>
> Sarath Chandra <sa...@algofusiontech.com>
> August 3, 2015 at 20:02
> Thanks Alan.
>
> Yes I've run metastore scripts for oracle instance. Infact I've removed my
> previous metastore and created a fresh one by running the schema creation
> script for 1.2.1. I've looked into the new schema and able to see the table
> TXNS. I've also removed the hdfs location "/user/hive/warehouse" and
> created a fresh one.
>
> But still I'm facing this issue.
>
>
>
> Alan Gates <al...@gmail.com>
> August 3, 2015 at 8:29
> Did you run the hive metastore upgrade scripts for your oracle instance?
> This error message usually means the transaction related tables have not
> been created in your database. Somewhere in your distribution there should
> be a set of upgrade scripts. Look for scripts of the form:
>
> scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
>
> You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14,
> 0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added the
> transaction tables as part of upgrading to Hive 0.13. If you did not you
> will need to first run hive-txn-schema-0.13.0.oracle.sql which will create
> the initial transaction tables. You can determine whether this was done by
> looking for a table named TXNS in the hive schema on your Oracle db.
>
> Alan.
>
> Sarath Chandra <sa...@algofusiontech.com>
> August 3, 2015 at 6:29
> Hi All,
>
> Earlier I was using hive 0.13.0 and now trying to migrate to latest
> version to utilize the transaction support introduced from hive 0.14.0.
>
> I downloaded hive 1.2.1, created a metastore in oracle database and
> provided all the required configuration parameters in conf/hive-site.xml to
> enable transactions. For the parameter "hive.txn.manager" given the value
> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>
> From the hive prompt when I fire the command "show tables;" I'm getting
> the below exception -
> *FAILED: Error in determining valid transactions: Error communicating with
> the metastore*
>
> But if disable the "hive.txn.manager" parameter in hive-site.xml then the
> command works fine.
>
> Is there anything else to be configured which I'm missing?
>
> Thanks & Regards,
> Sarath.
>
>
Re: Error communicating with metastore
Posted by Alan Gates <al...@gmail.com>.
Ok, the next step is to look at the logs from your Hive metastore server
and see exactly what's happening. The error you're seeing is from the
client. On your metastore server there should be logs with the same
timestamp giving details on why the transaction operation failed.
Alan.
> Sarath Chandra <ma...@algofusiontech.com>
> August 3, 2015 at 20:02
> Thanks Alan.
>
> Yes I've run metastore scripts for oracle instance. Infact I've
> removed my previous metastore and created a fresh one by running the
> schema creation script for 1.2.1. I've looked into the new schema and
> able to see the table TXNS. I've also removed the hdfs location
> "/user/hive/warehouse" and created a fresh one.
>
> But still I'm facing this issue.
>
>
>
> Alan Gates <ma...@gmail.com>
> August 3, 2015 at 8:29
> Did you run the hive metastore upgrade scripts for your oracle
> instance? This error message usually means the transaction related
> tables have not been created in your database. Somewhere in your
> distribution there should be a set of upgrade scripts. Look for
> scripts of the form:
>
> scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
>
> You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14,
> 0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added
> the transaction tables as part of upgrading to Hive 0.13. If you did
> not you will need to first run hive-txn-schema-0.13.0.oracle.sql which
> will create the initial transaction tables. You can determine whether
> this was done by looking for a table named TXNS in the hive schema on
> your Oracle db.
>
> Alan.
>
> Sarath Chandra <ma...@algofusiontech.com>
> August 3, 2015 at 6:29
> Hi All,
>
> Earlier I was using hive 0.13.0 and now trying to migrate to latest
> version to utilize the transaction support introduced from hive 0.14.0.
>
> I downloaded hive 1.2.1, created a metastore in oracle database and
> provided all the required configuration parameters in
> conf/hive-site.xml to enable transactions. For the parameter
> "hive.txn.manager" given the value
> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>
> From the hive prompt when I fire the command "show tables;" I'm
> getting the below exception -
> /FAILED: Error in determining valid transactions: Error communicating
> with the metastore/
>
> But if disable the "hive.txn.manager" parameter in hive-site.xml then
> the command works fine.
>
> Is there anything else to be configured which I'm missing?
>
> Thanks & Regards,
> Sarath.
Re: Error communicating with metastore
Posted by Eugene Koifman <ek...@hortonworks.com>.
could you execute "set hive.txn.manager" and "set hive.metastore.uris" in Hive CLI to confirm that your client is configured correctly?
From: Sarath Chandra <sa...@algofusiontech.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Monday, August 3, 2015 at 8:02 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Re: Error communicating with metastore
Thanks Alan.
Yes I've run metastore scripts for oracle instance. Infact I've removed my previous metastore and created a fresh one by running the schema creation script for 1.2.1. I've looked into the new schema and able to see the table TXNS. I've also removed the hdfs location "/user/hive/warehouse" and created a fresh one.
But still I'm facing this issue.
On Mon, Aug 3, 2015 at 8:59 PM, Alan Gates <al...@gmail.com>> wrote:
Did you run the hive metastore upgrade scripts for your oracle instance? This error message usually means the transaction related tables have not been created in your database. Somewhere in your distribution there should be a set of upgrade scripts. Look for scripts of the form:
scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14, 0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added the transaction tables as part of upgrading to Hive 0.13. If you did not you will need to first run hive-txn-schema-0.13.0.oracle.sql which will create the initial transaction tables. You can determine whether this was done by looking for a table named TXNS in the hive schema on your Oracle db.
Alan.
[cid:part1.03030509.04050004@gmail.com]
Sarath Chandra<ma...@algofusiontech.com>
August 3, 2015 at 6:29
Hi All,
Earlier I was using hive 0.13.0 and now trying to migrate to latest version to utilize the transaction support introduced from hive 0.14.0.
I downloaded hive 1.2.1, created a metastore in oracle database and provided all the required configuration parameters in conf/hive-site.xml to enable transactions. For the parameter "hive.txn.manager" given the value "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>From the hive prompt when I fire the command "show tables;" I'm getting the below exception -
FAILED: Error in determining valid transactions: Error communicating with the metastore
But if disable the "hive.txn.manager" parameter in hive-site.xml then the command works fine.
Is there anything else to be configured which I'm missing?
Thanks & Regards,
Sarath.
Re: Error communicating with metastore
Posted by Sarath Chandra <sa...@algofusiontech.com>.
Thanks Alan.
Yes I've run metastore scripts for oracle instance. Infact I've removed my
previous metastore and created a fresh one by running the schema creation
script for 1.2.1. I've looked into the new schema and able to see the table
TXNS. I've also removed the hdfs location "/user/hive/warehouse" and
created a fresh one.
But still I'm facing this issue.
On Mon, Aug 3, 2015 at 8:59 PM, Alan Gates <al...@gmail.com> wrote:
> Did you run the hive metastore upgrade scripts for your oracle instance?
> This error message usually means the transaction related tables have not
> been created in your database. Somewhere in your distribution there should
> be a set of upgrade scripts. Look for scripts of the form:
>
> scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
>
> You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14,
> 0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added the
> transaction tables as part of upgrading to Hive 0.13. If you did not you
> will need to first run hive-txn-schema-0.13.0.oracle.sql which will create
> the initial transaction tables. You can determine whether this was done by
> looking for a table named TXNS in the hive schema on your Oracle db.
>
> Alan.
>
> Sarath Chandra <sa...@algofusiontech.com>
> August 3, 2015 at 6:29
> Hi All,
>
> Earlier I was using hive 0.13.0 and now trying to migrate to latest
> version to utilize the transaction support introduced from hive 0.14.0.
>
> I downloaded hive 1.2.1, created a metastore in oracle database and
> provided all the required configuration parameters in conf/hive-site.xml to
> enable transactions. For the parameter "hive.txn.manager" given the value
> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>
> From the hive prompt when I fire the command "show tables;" I'm getting
> the below exception -
> *FAILED: Error in determining valid transactions: Error communicating with
> the metastore*
>
> But if disable the "hive.txn.manager" parameter in hive-site.xml then the
> command works fine.
>
> Is there anything else to be configured which I'm missing?
>
> Thanks & Regards,
> Sarath.
>
>
Re: Error communicating with metastore
Posted by Alan Gates <al...@gmail.com>.
Did you run the hive metastore upgrade scripts for your oracle
instance? This error message usually means the transaction related
tables have not been created in your database. Somewhere in your
distribution there should be a set of upgrade scripts. Look for scripts
of the form:
scripts/metastore/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
You'll want to run all of the ones from 0.13 to 1.2 (0.13->0.14,
0.14->1.1, 1.1->1.2). The 0.13->0.14 scripts assume that you added the
transaction tables as part of upgrading to Hive 0.13. If you did not
you will need to first run hive-txn-schema-0.13.0.oracle.sql which will
create the initial transaction tables. You can determine whether this
was done by looking for a table named TXNS in the hive schema on your
Oracle db.
Alan.
> Sarath Chandra <ma...@algofusiontech.com>
> August 3, 2015 at 6:29
> Hi All,
>
> Earlier I was using hive 0.13.0 and now trying to migrate to latest
> version to utilize the transaction support introduced from hive 0.14.0.
>
> I downloaded hive 1.2.1, created a metastore in oracle database and
> provided all the required configuration parameters in
> conf/hive-site.xml to enable transactions. For the parameter
> "hive.txn.manager" given the value
> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>
> From the hive prompt when I fire the command "show tables;" I'm
> getting the below exception -
> /FAILED: Error in determining valid transactions: Error communicating
> with the metastore/
>
> But if disable the "hive.txn.manager" parameter in hive-site.xml then
> the command works fine.
>
> Is there anything else to be configured which I'm missing?
>
> Thanks & Regards,
> Sarath.
Re: Error communicating with metastore
Posted by Sarath Chandra <sa...@algofusiontech.com>.
Thanks Sharath for your reply. I'm specifying these parameters already.
I've followed the documentation at
https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions
But it still throws me this error
On Mon, Aug 3, 2015 at 8:45 PM, Sharath Punreddy <sr...@gmail.com>
wrote:
> Here is another one:
>
> http://stackoverflow.com/questions/28867368/hive-transactions-are-crashing
>
> On Mon, Aug 3, 2015 at 10:13 AM, Sharath Punreddy <sr...@gmail.com>
> wrote:
>
>> Check out:
>> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_dataintegration/content/ch_using-hive-transactions.html
>>
>> You need to specify the following properties as well:
>>
>> -
>>
>> hive.txn.driver
>> -
>>
>> hive.compactor.initiator.on
>> -
>>
>> hive.compactor.worker.threads
>>
>>
>> On Mon, Aug 3, 2015 at 8:29 AM, Sarath Chandra <
>> sarathchandra.josyam@algofusiontech.com> wrote:
>>
>>> Hi All,
>>>
>>> Earlier I was using hive 0.13.0 and now trying to migrate to latest
>>> version to utilize the transaction support introduced from hive 0.14.0.
>>>
>>> I downloaded hive 1.2.1, created a metastore in oracle database and
>>> provided all the required configuration parameters in conf/hive-site.xml to
>>> enable transactions. For the parameter "hive.txn.manager" given the value
>>> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>>>
>>> From the hive prompt when I fire the command "show tables;" I'm getting
>>> the below exception -
>>> *FAILED: Error in determining valid transactions: Error communicating
>>> with the metastore*
>>>
>>> But if disable the "hive.txn.manager" parameter in hive-site.xml then
>>> the command works fine.
>>>
>>> Is there anything else to be configured which I'm missing?
>>>
>>> Thanks & Regards,
>>> Sarath.
>>>
>>
>>
>>
>> --
>>
>>
>>
>> *Sharath Punreddy*
>> *Phone:626-470-7867 <626-470-7867>*
>>
>
>
>
> --
>
>
>
> *Sharath Punreddy*
> *Phone:626-470-7867*
>
Re: Error communicating with metastore
Posted by Sharath Punreddy <sr...@gmail.com>.
Check out:
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_dataintegration/content/ch_using-hive-transactions.html
You need to specify the following properties as well:
-
hive.txn.driver
-
hive.compactor.initiator.on
-
hive.compactor.worker.threads
On Mon, Aug 3, 2015 at 8:29 AM, Sarath Chandra <
sarathchandra.josyam@algofusiontech.com> wrote:
> Hi All,
>
> Earlier I was using hive 0.13.0 and now trying to migrate to latest
> version to utilize the transaction support introduced from hive 0.14.0.
>
> I downloaded hive 1.2.1, created a metastore in oracle database and
> provided all the required configuration parameters in conf/hive-site.xml to
> enable transactions. For the parameter "hive.txn.manager" given the value
> "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager".
>
> From the hive prompt when I fire the command "show tables;" I'm getting
> the below exception -
> *FAILED: Error in determining valid transactions: Error communicating with
> the metastore*
>
> But if disable the "hive.txn.manager" parameter in hive-site.xml then the
> command works fine.
>
> Is there anything else to be configured which I'm missing?
>
> Thanks & Regards,
> Sarath.
>
--
*Sharath Punreddy*
*Phone:626-470-7867*