thread safety (hive-30/73)

[Joydeep Sen Sarma <js...@facebook.com>]

I finally ran a multithreaded test today. Fixed the session stuff - but the MetaStore client does not appear to be thread safe. Concurrent create table commands report the exception below without fail. So as a short term measure (or at least until Prasad can look at this) - hive -e "<query>" would appear to be ur route to salvation:

Still testing without the DDLs to see if that's ok ..

Joydeep


java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\
y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
  at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:207)
  at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:209)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:174)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:210)
  at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:390)
  at org.apache.hadoop.hive.ql.QTestUtil$QTRunner.run(QTestUtil.java:681)
  at java.lang.Thread.run(Thread.java:619)
Caused by: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MTable@3bc8d400" us\
ing statement "INSERT INTO TBLS (TBL_ID,CREATE_TIME,DB_ID,RETENTION,TBL_NAME,SD_ID,OWNER,LAST_ACCESS_TIME) VALUES (?,?,?\
,?,?,?,?,?)" failed : The statement was aborted because it would have caused a duplicate key value in a unique or primar\
y key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
NestedThrowables:
java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a duplicate ke\
y value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
  at org.jpox.jdo.JPOXJDOHelper.getJDOExceptionForJPOXException(JPOXJDOHelper.java:291)
  at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:671)
  at org.jpox.jdo.AbstractPersistenceManager.makePersistent(AbstractPersistenceManager.java:691)
  at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:479)
  at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table(HiveMetaStore.java:292)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:252)
  at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:205)
  ... 7 more
Caused by: java.sql.SQLIntegrityConstraintViolationException: The statement was aborted because it would have caused a d\
uplicate key value in a unique or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
  at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
  at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
  at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
  at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
  at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
  at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
  at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
  at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
  at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
  at org.jpox.store.rdbms.SQLController.executeStatementUpdate(SQLController.java:396)
  at org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:370)
  at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertTable(RDBMSPersistenceHandler.java:157)
  at org.jpox.store.rdbms.RDBMSPersistenceHandler.insertObject(RDBMSPersistenceHandler.java:136)
  at org.jpox.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3082)
  at org.jpox.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3062)
  at org.jpox.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1231)
  at org.jpox.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1077)
  at org.jpox.jdo.AbstractPersistenceManager.jdoMakePersistent(AbstractPersistenceManager.java:666)
  ... 12 more
Caused by: java.sql.SQLException: The statement was aborted because it would have caused a duplicate key value in a uniq\
ue or primary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
  at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
  at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
  ... 30 more
Caused by: ERROR 23505: The statement was aborted because it would have caused a duplicate key value in a unique or prim\
ary key constraint or unique index identified by 'UNIQUETABLE' defined on 'TBLS'.
  at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
  at org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups(Unknown Source)
  at org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown Source)
  at org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown Source)
  at org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown Source)
  at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
  at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
  at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)