[jira] [Commented] (HIVE-14152) datanucleus.autoStartMechanismMode should set to 'Ignored' to allow rolling downgrade

["Thejas M Nair (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-14152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361772#comment-15361772 ] 

Thejas M Nair commented on HIVE-14152:
--------------------------------------

We already have safety checks in the form of hive.metastore.schema.verification, which are more advanced as it checks for schema *compatibility* instead of *exact match* .
This makes sense in case of upgrade-downgrade scenarios of hive, as newer versions of schema are compatible with older ones, and schema doesn't need to be downgraded on a downgrade of hive version.
This also allows for running of two versions of hive against same instance of metastore rdbms.
 

> datanucleus.autoStartMechanismMode should set to 'Ignored' to allow rolling downgrade 
> --------------------------------------------------------------------------------------
>
>                 Key: HIVE-14152
>                 URL: https://issues.apache.org/jira/browse/HIVE-14152
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>
> We see the following issue when downgrading metastore:
> 1. Run some query using new tables
> 2. Downgrade metastore
> 3. Restart metastore will complain the new table does not exist
> In particular, constaints tables does not exist in branch-1. If we run Hive 2 and create a constraint, then downgrade metastore to Hive 1, datanucleus will complain:
> {code}
> javax.jdo.JDOFatalUserException: Error starting up DataNucleus : a class "org.apache.hadoop.hive.metastore.model.MConstraint" was listed as being persisted previously in this datastore, yet the class wasnt found. Perhaps it is used by a different DataNucleus-enabled application in this datastore, or you have changed your class names.
> 	at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:528)
> 	at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
> 	at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
> 	at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
> 	at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
> 	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
> 	at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
> 	at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:377)
> 	at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:406)
> 	at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:299)
> 	at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:266)
> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:60)
> 	at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:69)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:650)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:628)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:677)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:484)
> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:77)
> 	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:83)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5905)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5900)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6159)
> 	at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6084)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
> Apparently datanucleus cache some trace about the new table and retry to reinstantiate later. This breaks downgrading we shall disable this behavior.
> We need to set "datanucleus.autoStartMechanismMode" to "Ignored" to disable the check since it becomes a norm in downgrading case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)