Hive 9 error in HiveServer and DataNucleus

["Tuong Tr." <ir...@yahoo.com>]

Hi,

We ran into this problem with Hive 0.9.0 trying to get tables info from HiveMetastore via HiveServer/Thrift.  The problem seems to be fixed in Hive 11+, but our effort to locate the Jira has not been successfully.  I appreciate if someone familiar with this problem can point out the Jira/Patch to us.

We consistently hit this error when the number of calls to the HiveMetastore hits the hive.metastore.server.min.threads setting in the server's hive-site.xml . 

Error :

Hive Client side error:Exception in thread "main" MetaException(message:Got exception: org.apache.thrift.transport.TTransportException null)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:785)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getAllDatabases(HiveMetaStoreClient.java:615)


Hive Server side error:
2014-02-13 08:28:27,866 INFO  metastore.HiveMetaStore (HiveMetaStore.java:logInfo(385)) - 47: get_all_databases
2014-02-13 08:28:27,870 ERROR server.TThreadPoolServer (TThreadPoolServer.java:run(182)) - Error occurred during processing of message.
javax.jdo.JDOFatalUserException: Persistence Manager has been closed
at org.datanucleus.jdo.JDOPersistenceManager.assertIsOpen(JDOPersistenceManager.java:2088)
at org.datanucleus.jdo.JDOPersistenceManager.currentTransaction(JDOPersistenceManager.java:305)
at org.apache.hadoop.hive.metastore.ObjectStore.openTransaction(ObjectStore.java:294)
at org.apache.hadoop.hive.metastore.ObjectStore.getDatabases(ObjectStore.java:488)
at org.apache.hadoop.hive.metastore.ObjectStore.getAllDatabases(ObjectStore.java:522)
at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
at $Proxy1.getAllDatabases(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_all_databases(HiveMetaStore.java:660)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_all_databases.getResult(ThriftHiveMetastore.java:4749)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_all_databases.getResult(ThriftHiveMetastore.java:4737)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
at java.lang.Thread.run(Thread.java:738)



Thanks for your help,
Tuong