You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Gergely Fürnstáhl (Jira)" <ji...@apache.org> on 2022/05/23 14:32:00 UTC
[jira] [Updated] (IMPALA-11312) Upgrading metastore db for new hive throws error
[ https://issues.apache.org/jira/browse/IMPALA-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gergely Fürnstáhl updated IMPALA-11312:
---------------------------------------
Summary: Upgrading metastore db for new hive throws error (was: Upgrading metastore for new hive throws error)
> Upgrading metastore db for new hive throws error
> ------------------------------------------------
>
> Key: IMPALA-11312
> URL: https://issues.apache.org/jira/browse/IMPALA-11312
> Project: IMPALA
> Issue Type: Bug
> Reporter: Gergely Fürnstáhl
> Priority: Minor
>
> I built a new hive package from cdw-master:
> git checkout cdw-master
> ~/bin/cdpd-patcher hive cdw-master
> Create settings file: [https://cloudera.atlassian.net/wiki/spaces/ENG/pages/160597896/How+to+build+CDPD-master]
> mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true -s hdpsnap.settings.xml
> ~/hive/packaging/target/apache-hive-3.1.3000.2022.0.9.0-18-bin/apache-hive-3.1.3000.2022.0.9.0-18-bin(cdw-master)$ cp -r ./* ~/Impala/toolchain/cdp_components-26065553/apache-hive-3.1.3000.7.2.16.0-33-bin
> Got some errors from hive relating to versioning, tried to upgrade the DB.
> gfurnstahl@gfurnstahl-OptiPlex-7070:~/Impala(IMPALA-11034)$ bin/create-test-configuration.sh -upgrade_metastore_db
> ...
> Upgrading the schema of metastore db HMS_home_gfurnstahl_Impala_cdp. Check /home/gfurnstahl/Impala/logs/cluster/schematool.log for details.
> awk: fatal: cannot open file `bin/create-test-configuration.sh' for reading (No such file or directory)
> There were several SQL errors in schematool.log and hive still didn't work
> +-------------------+
> | tab_name |
> +-------------------+
> | hive_warm_up_tbl |
> +-------------------+
> 1 row selected (0.058 seconds)
> 0: jdbc:hive2://localhost:11050/default> select * from hive_warm_up_tbl;
> INFO : Compiling command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb): select * from hive_warm_up_tbl
> INFO : No Stats for default@hive_warm_up_tbl, Columns: i
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:hive_warm_up_tbl.i, type:int, comment:null)], properties:null)
> INFO : Completed compiling command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb); Time taken: 0.066 seconds
> INFO : Executing command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb): select * from hive_warm_up_tbl
> INFO : Completed executing command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb); Time taken: 0.0 seconds
> INFO : OK
> ERROR : FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error communicating with the metastore)
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore
> at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:568)
> at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:484)
> at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:448)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:788)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:526)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:520)
> at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:232)
> at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:89)
> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:338)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:358)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.thrift.TApplicationException: Internal error processing commit_txn
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_commit_txn(ThriftHiveMetastore.java:5995)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.commit_txn(ThriftHiveMetastore.java:5982)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.commitTxn(HiveMetaStoreClient.java:3707)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClientWithLocalCache.commitTxn(HiveMetaStoreClientWithLocalCache.java:944)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:216)
> at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:4022)
> at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
> at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:559)
> ... 20 more
> *Workaround:* formatting solves the issue
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org