You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Elon Chen (Jira)" <ji...@apache.org> on 2022/03/09 03:38:00 UTC
[jira] [Commented] (HIVE-24493) after upgrading Hive from 2.1.1 to 2.3.7, log keeps printing "Hive Schema version 2.3.0 does not match metastore's schema version 2.1.0"
[ https://issues.apache.org/jira/browse/HIVE-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17503279#comment-17503279 ]
Elon Chen commented on HIVE-24493:
----------------------------------
I have the same issue. is there anyone?
> after upgrading Hive from 2.1.1 to 2.3.7, log keeps printing "Hive Schema version 2.3.0 does not match metastore's schema version 2.1.0"
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-24493
> URL: https://issues.apache.org/jira/browse/HIVE-24493
> Project: Hive
> Issue Type: Bug
> Environment: {code:java}
> Linux version 4.15.0-112-generic (buildd@lcy01-amd64-027) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 {code}
> {code:java}
> openjdk version "1.8.0_252"
> OpenJDK Runtime Environment (build 1.8.0_252-b09)
> OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode){code}
> Reporter: Yongle Zhang
> Priority: Major
>
> We tried to upgrade a hive single node from 2.1.1 to 2.3.7 and use derby as the database. After upgrade, Hive log keeps printing the following Error message repeatedly:
>
> {code:java}
> // code placeholder
> 2020-12-05T02:47:26,573 ERROR [main] metastore.RetryingHMSHandler: HMSHandler Fatal error: MetaException(message:Hive Schema version 2.3.0 does not match metastore's schema version 2.1.0 Metastore is not upgraded or corrupt)
> at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7825)
> at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7788)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
> at com.sun.proxy.$Proxy34.verifySchema(Unknown Source)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:595)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:588)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:655)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:431)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:79)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92)
> at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6902)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:164)
> at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1707)
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
> at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3600)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3652)
> at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3632)
> at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3894)
> at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:248)
> at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231)
> at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:388)
> at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:332)
> at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312)
> at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288)
> at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:917)
> at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:881)
> at org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1687)
> at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:130)
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:114)
> at org.apache.hive.service.CompositeService.init(CompositeService.java:59)
> at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:142)
> at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:607)
> at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:100)
> at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:855)
> at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:724)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153){code}
>
>
> We did use schematool to upgrade Hive. The log also shows that upgrade metastore is performed. Here are the steps to reproduce it:
>
> 1. Start up a standalone hive node of version 2.1.1 using embedded derby, and use schematool to initialize the schema.
> 2. Use the following shell script to generate 20k data.
>
> {code:java}
> rm -rf ./data.txt
> touch data.txt
> for((i=0;i<20000;i++))
> do
> str=',name';
> name=${i}${str}${i}
> echo $name>> data.txt
> done
> {code}
> 3. Create a table and load data by using:
> {code:java}
> create table testTable(id int,name string) row format delimited fields terminated by ',';
> LOAD DATA LOCAL INPATH '/data.txt' OVERWRITE INTO TABLE testTable;
> {code}
> 4. We stopped the nodes, and started a node running hive compiled from 2.3.7, while keeping the node of hadoop running.
> 5. Run the following command to upgrade schema.
>
> {code:java}
> schematool -dbType derby -upgradeSchema
> {code}
> Log shows that metastore is being upgraded:
> {code:java}
> LF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Metastore connection URL: jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
> Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
> Metastore connection User: APP
> Starting upgrade metastore schema from version 2.1.0 to 2.3.0
> Upgrade script upgrade-2.1.0-to-2.2.0.derby.sql
> Connecting to jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
> Connected to: Apache Derby (version 10.10.2.0 - (1582446))
> Driver: Apache Derby Embedded JDBC Driver (version 10.10.2.0 - (1582446))
> .
> .
> .
> beeline> Completed upgrade-2.1.0-to-2.2.0.derby.sql
> Upgrade script upgrade-2.2.0-to-2.3.0.derby.sql
> Connecting to jdbc:derby:;databaseName=/hive/data/metastore_db;create=true
> Connected to: Apache Derby (version 10.10.2.0 - (1582446))
> Driver: Apache Derby Embedded JDBC Driver (version 10.10.2.0 - (1582446))
> .
> .
> .
> beeline> Completed upgrade-2.2.0-to-2.3.0.derby.sql
> schemaTool completed
> {code}
> 6. However, after upgrade, the log still keeps printing the previous Error message with exception - "Hive Schema version 2.3.0 does not match metastore's schema version 2.1.0”.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)