You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Rahul Singh <ra...@gmail.com> on 2010/11/29 07:07:05 UTC
Error while dropping the table.
Hi ,
We are using Postgres as Hive Meta Store. When we try to drop a table , It
hangs. Below is the small dump
Can any one help me in this regard
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.cache.level2.type value null from jpox.properties with SOFT
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.plugin.pluginRegistryBundleCheck value null from
jpox.properties with LOG
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
javax.jdo.option.ConnectionURL value null from jpox.properties with
jdbc:postgresql://10.1.0.33:5488/metastore_XXX
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.autoStartMechanismMode value null from jpox.properties with
checked
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.validateConstraints value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.autoCreateSchema value null from jpox.properties with true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.cache.level2 value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
javax.jdo.option.ConnectionUserName value null from jpox.properties with
postgres
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
javax.jdo.option.DetachAllOnCommit value null from jpox.properties with
true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.connectionPoolingType value null from jpox.properties with DBCP
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.storeManagerType value null from jpox.properties with rdbms
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
javax.jdo.option.ConnectionDriverName value null from jpox.properties with
org.postgresql.Driver
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.transactionIsolation value null from jpox.properties with
read-committed
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.validateColumns value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
javax.jdo.option.NonTransactionalRead value null from jpox.properties with
true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.validateTables value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
javax.jdo.PersistenceManagerFactoryClass value null from jpox.properties
with org.datanucleus.jdo.JDOPersistenceManagerFactory
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding
datanucleus.identifierFactory value null from jpox.properties with
datanucleus
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
datanucleus.autoStartMechanismMode = checked
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.identifierFactory
= datanucleus
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
datanucleus.transactionIsolation = read-committed
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.validateTables =
false
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
javax.jdo.option.ConnectionURL = jdbc:postgresql://
10.1.0.33:5488/metastore_yoda
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
javax.jdo.option.DetachAllOnCommit = true
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
javax.jdo.option.NonTransactionalRead = true
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
datanucleus.validateConstraints = false
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
javax.jdo.option.ConnectionDriverName = org.postgresql.Driver
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
javax.jdo.option.ConnectionUserName = postgres
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.validateColumns =
false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.cache.level2 =
false
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
datanucleus.plugin.pluginRegistryBundleCheck = LOG
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.cache.level2.type
= SOFT
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
javax.jdo.PersistenceManagerFactoryClass =
org.datanucleus.jdo.JDOPersistenceManagerFactory
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.autoCreateSchema
= true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.storeManagerType
= rdbms
10/11/29 06:02:07 DEBUG metastore.ObjectStore:
datanucleus.connectionPoolingType = DBCP
10/11/29 06:02:07 INFO metastore.ObjectStore: ObjectStore, initialize called
10/11/29 06:02:07 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core"
requires "org.eclipse.core.resources" but it cannot be resolved.
10/11/29 06:02:07 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core"
requires "org.eclipse.core.runtime" but it cannot be resolved.
10/11/29 06:02:07 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core"
requires "org.eclipse.text" but it cannot be resolved.
10/11/29 06:02:08 INFO metastore.ObjectStore: Initialized ObjectStore
10/11/29 06:02:08 INFO metastore.HiveMetaStore: 0: get_table : db=default
tbl=erer
10/11/29 06:02:09 INFO hive.log: DDL: struct erer { i32 a}
10/11/29 06:02:09 DEBUG lazy.LazySimpleSerDe: LazySimpleSerDe initialized
with: columnNames=[a] columnTypes=[int] separator=[[B@e4600c0] nullstring=\N
lastColumnTakesRest=false
10/11/29 06:02:09 INFO metastore.HiveMetaStore: 0: get_table : db=default
tbl=erer
10/11/29 06:02:09 INFO metastore.HiveMetaStore: 0: drop_table : db=default
tbl=erer
10/11/29 06:02:09 INFO metastore.HiveMetaStore: 0: get_table : db=default
tbl=erer
*10/11/29 06:02:09 DEBUG metastore.ObjectStore: Executing
listMPartitions ====> IT HANGS HERE*
Re: Error while dropping the table.
Posted by Rekha Joshi <re...@yahoo-inc.com>.
Hi Rahul,
On a quick look, it feels like your javax.jdo.Query is not able to return the list of MPartition with the passed values of default(db), erer(table).
You will need to write a small testcase and check if the below snippets work on your setup.
Query query = pm.newQuery(MPartition.class, "table.tableName == t1 && table.database.name == t2"); query.declareParameters("java.lang.String t1, java.lang.String t2"); mparts = (List<MPartition>) query.execute(erer, default); //hard-coded your values..
If getting the list of partitions is an issue, it would cause same issue in other operations other than drop as well..
That analyzed it could as well turn out to be some issue on file system or configuration..
Thanks & Regards,
/Rekha.
On 11/29/10 11:37 AM, "Rahul Singh" <ra...@gmail.com> wrote:
Hi ,
We are using Postgres as Hive Meta Store. When we try to drop a table , It hangs. Below is the small dump
Can any one help me in this regard
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2.type value null from jpox.properties with SOFT
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.plugin.pluginRegistryBundleCheck value null from jpox.properties with LOG
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionURL value null from jpox.properties with jdbc:postgresql://10.1.0.33:5488/metastore_XXX <http://10.1.0.33:5488/metastore_XXX>
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.autoStartMechanismMode value null from jpox.properties with checked
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.validateConstraints value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.autoCreateSchema value null from jpox.properties with true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2 value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionUserName value null from jpox.properties with postgres
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.DetachAllOnCommit value null from jpox.properties with true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.connectionPoolingType value null from jpox.properties with DBCP
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.storeManagerType value null from jpox.properties with rdbms
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionDriverName value null from jpox.properties with org.postgresql.Driver
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.transactionIsolation value null from jpox.properties with read-committed
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.validateColumns value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding javax.jdo.option.NonTransactionalRead value null from jpox.properties with true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.validateTables value null from jpox.properties with false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding javax.jdo.PersistenceManagerFactoryClass value null from jpox.properties with org.datanucleus.jdo.JDOPersistenceManagerFactory
10/11/29 06:02:07 DEBUG metastore.ObjectStore: Overriding datanucleus.identifierFactory value null from jpox.properties with datanucleus
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.autoStartMechanismMode = checked
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.identifierFactory = datanucleus
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.transactionIsolation = read-committed
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.validateTables = false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionURL = jdbc:postgresql://10.1.0.33:5488/metastore_yoda <http://10.1.0.33:5488/metastore_yoda>
10/11/29 06:02:07 DEBUG metastore.ObjectStore: javax.jdo.option.DetachAllOnCommit = true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: javax.jdo.option.NonTransactionalRead = true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.validateConstraints = false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionDriverName = org.postgresql.Driver
10/11/29 06:02:07 DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionUserName = postgres
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.validateColumns = false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.cache.level2 = false
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.plugin.pluginRegistryBundleCheck = LOG
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.cache.level2.type = SOFT
10/11/29 06:02:07 DEBUG metastore.ObjectStore: javax.jdo.PersistenceManagerFactoryClass = org.datanucleus.jdo.JDOPersistenceManagerFactory
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.autoCreateSchema = true
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.storeManagerType = rdbms
10/11/29 06:02:07 DEBUG metastore.ObjectStore: datanucleus.connectionPoolingType = DBCP
10/11/29 06:02:07 INFO metastore.ObjectStore: ObjectStore, initialize called
10/11/29 06:02:07 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.resources" but it cannot be resolved.
10/11/29 06:02:07 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core" requires "org.eclipse.core.runtime" but it cannot be resolved.
10/11/29 06:02:07 ERROR DataNucleus.Plugin: Bundle "org.eclipse.jdt.core" requires "org.eclipse.text" but it cannot be resolved.
10/11/29 06:02:08 INFO metastore.ObjectStore: Initialized ObjectStore
10/11/29 06:02:08 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=erer
10/11/29 06:02:09 INFO hive.log: DDL: struct erer { i32 a}
10/11/29 06:02:09 DEBUG lazy.LazySimpleSerDe: LazySimpleSerDe initialized with: columnNames=[a] columnTypes=[int] separator=[[B@e4600c0] nullstring=\N lastColumnTakesRest=false
10/11/29 06:02:09 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=erer
10/11/29 06:02:09 INFO metastore.HiveMetaStore: 0: drop_table : db=default tbl=erer
10/11/29 06:02:09 INFO metastore.HiveMetaStore: 0: get_table : db=default tbl=erer
10/11/29 06:02:09 DEBUG metastore.ObjectStore: Executing listMPartitions ====> IT HANGS HERE