You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Gergely Fürnstáhl (Jira)" <ji...@apache.org> on 2022/05/23 14:31:00 UTC
[jira] [Created] (IMPALA-11312) Upgrading metastore for new hive threw error
Gergely Fürnstáhl created IMPALA-11312:
------------------------------------------
Summary: Upgrading metastore for new hive threw error
Key: IMPALA-11312
URL: https://issues.apache.org/jira/browse/IMPALA-11312
Project: IMPALA
Issue Type: Bug
Reporter: Gergely Fürnstáhl
I built a new hive package from cdw-master:
git checkout cdw-master
~/bin/cdpd-patcher hive cdw-master
Create settings file: [https://cloudera.atlassian.net/wiki/spaces/ENG/pages/160597896/How+to+build+CDPD-master]
mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true -s hdpsnap.settings.xml
~/hive/packaging/target/apache-hive-3.1.3000.2022.0.9.0-18-bin/apache-hive-3.1.3000.2022.0.9.0-18-bin(cdw-master)$ cp -r ./* ~/Impala/toolchain/cdp_components-26065553/apache-hive-3.1.3000.7.2.16.0-33-bin
Got some errors from hive relating to versioning, tried to upgrade the DB.
gfurnstahl@gfurnstahl-OptiPlex-7070:~/Impala(IMPALA-11034)$ bin/create-test-configuration.sh -upgrade_metastore_db
...
Upgrading the schema of metastore db HMS_home_gfurnstahl_Impala_cdp. Check /home/gfurnstahl/Impala/logs/cluster/schematool.log for details.
awk: fatal: cannot open file `bin/create-test-configuration.sh' for reading (No such file or directory)
There were several SQL errors in schematool.log and hive still didn't work
+-------------------+
| tab_name |
+-------------------+
| hive_warm_up_tbl |
+-------------------+
1 row selected (0.058 seconds)
0: jdbc:hive2://localhost:11050/default> select * from hive_warm_up_tbl;
INFO : Compiling command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb): select * from hive_warm_up_tbl
INFO : No Stats for default@hive_warm_up_tbl, Columns: i
INFO : Semantic Analysis Completed (retrial = false)
INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:hive_warm_up_tbl.i, type:int, comment:null)], properties:null)
INFO : Completed compiling command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb); Time taken: 0.066 seconds
INFO : Executing command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb): select * from hive_warm_up_tbl
INFO : Completed executing command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb); Time taken: 0.0 seconds
INFO : OK
ERROR : FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error communicating with the metastore)
org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore
at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:568)
at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:484)
at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:448)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:788)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:526)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:520)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:232)
at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:89)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:338)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:358)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.TApplicationException: Internal error processing commit_txn
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_commit_txn(ThriftHiveMetastore.java:5995)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.commit_txn(ThriftHiveMetastore.java:5982)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.commitTxn(HiveMetaStoreClient.java:3707)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClientWithLocalCache.commitTxn(HiveMetaStoreClientWithLocalCache.java:944)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:216)
at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:4022)
at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:559)
... 20 more
*Workaround:* formatting solves the issue
--
This message was sent by Atlassian Jira
(v8.20.7#820007)