You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Laszlo Gaal (Jira)" <ji...@apache.org> on 2022/11/24 14:32:00 UTC
[jira] [Closed] (IMPALA-11312) Upgrading metastore db for new hive throws error
[ https://issues.apache.org/jira/browse/IMPALA-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Laszlo Gaal closed IMPALA-11312.
--------------------------------
Assignee: Gergely Fürnstáhl
Resolution: Cannot Reproduce
This problem is not about Apache Impala, it cannot be reproduced with upstream code in any environment supported for Apache Impala code.
It is best described and tracked in a downstream environment where the original problem was encountered.
Also it is good practice to keep internal details off the public Apache Jira tracker.
> Upgrading metastore db for new hive throws error
> ------------------------------------------------
>
> Key: IMPALA-11312
> URL: https://issues.apache.org/jira/browse/IMPALA-11312
> Project: IMPALA
> Issue Type: Bug
> Reporter: Gergely Fürnstáhl
> Assignee: Gergely Fürnstáhl
> Priority: Minor
>
> I built a new hive package from cdw-master:
> git checkout cdw-master
> ~/bin/cdpd-patcher hive cdw-master
> Create settings file: [https://cloudera.atlassian.net/wiki/spaces/ENG/pages/160597896/How+to+build+CDPD-master]
> mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true -s hdpsnap.settings.xml
> ~/hive/packaging/target/apache-hive-3.1.3000.2022.0.9.0-18-bin/apache-hive-3.1.3000.2022.0.9.0-18-bin(cdw-master)$ cp -r ./* ~/Impala/toolchain/cdp_components-26065553/apache-hive-3.1.3000.7.2.16.0-33-bin
> Got some errors from hive relating to versioning, tried to upgrade the DB.
> gfurnstahl@gfurnstahl-OptiPlex-7070:~/Impala(IMPALA-11034)$ bin/create-test-configuration.sh -upgrade_metastore_db
> ...
> Upgrading the schema of metastore db HMS_home_gfurnstahl_Impala_cdp. Check /home/gfurnstahl/Impala/logs/cluster/schematool.log for details.
> awk: fatal: cannot open file `bin/create-test-configuration.sh' for reading (No such file or directory)
> There were several SQL errors in schematool.log and hive still didn't work
> +-------------------+
> | tab_name |
> +-------------------+
> | hive_warm_up_tbl |
> +-------------------+
> 1 row selected (0.058 seconds)
> 0: jdbc:hive2://localhost:11050/default> select * from hive_warm_up_tbl;
> INFO : Compiling command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb): select * from hive_warm_up_tbl
> INFO : No Stats for default@hive_warm_up_tbl, Columns: i
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:hive_warm_up_tbl.i, type:int, comment:null)], properties:null)
> INFO : Completed compiling command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb); Time taken: 0.066 seconds
> INFO : Executing command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb): select * from hive_warm_up_tbl
> INFO : Completed executing command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb); Time taken: 0.0 seconds
> INFO : OK
> ERROR : FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.lockmgr.LockException(Error communicating with the metastore)
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore
> at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:568)
> at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:484)
> at org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:448)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:788)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:526)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:520)
> at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:232)
> at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:89)
> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:338)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:358)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.thrift.TApplicationException: Internal error processing commit_txn
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_commit_txn(ThriftHiveMetastore.java:5995)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.commit_txn(ThriftHiveMetastore.java:5982)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.commitTxn(HiveMetaStoreClient.java:3707)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClientWithLocalCache.commitTxn(HiveMetaStoreClientWithLocalCache.java:944)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:216)
> at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:4022)
> at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
> at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:559)
> ... 20 more
> *Workaround:* formatting solves the issue
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org