You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Vipin (Jira)" <ji...@apache.org> on 2021/10/04 16:34:00 UTC
[jira] [Updated] (HIVE-25565) Materialized view Rebuild of Non-Txn
table issue Aws EMR
[ https://issues.apache.org/jira/browse/HIVE-25565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vipin updated HIVE-25565:
-------------------------
Summary: Materialized view Rebuild of Non-Txn table issue Aws EMR (was: Materialized view Rebuild issue Aws EMR)
> Materialized view Rebuild of Non-Txn table issue Aws EMR
> --------------------------------------------------------
>
> Key: HIVE-25565
> URL: https://issues.apache.org/jira/browse/HIVE-25565
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
> Affects Versions: 3.1.2
> Environment: Aws EMR - 6.3.0
> Spark - 3.1.1
> Hive - 3.1.2
> Reporter: Vipin
> Priority: Major
>
> We have Materialized views built on top of Hudi tables which are hive-sync'd.
> Hive uses AWS Glue for its metastore catalog.
> We are running into issue whenever we are trying to "**rebuild**" Hive materialized views.
> Please note, creation of materialized views works fine. It's only rebuild which is failing.
> However, it does seem the rebuild actually seems to work behind the scenes but its throws some exception causing EMR steps to fail.
> Can anyone please guide us here, about any config changes that we need to do or anything. Any help will be great.
>
> The stack trace of the exception -
> {quote} FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking FailureHook. hooks: java.lang.NullPointerException at org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45) at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296) at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283) at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703) at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224) at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)) > org.apache.hadoop.hive.ql.metadata.HiveException: Error while invoking FailureHook. hooks: > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)> at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616)> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386)> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011)> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709)> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703)> at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)> at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)> at java.security.AccessController.doPrivileged(Native Method)> at javax.security.auth.Subject.doAs(Subject.java:422)> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)> at java.util.concurrent.FutureTask.run(FutureTask.java:266)> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)> at java.lang.Thread.run(Thread.java:748)> > at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:302)> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616)> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386)> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011)> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709)> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703)> at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)> at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)> at java.security.AccessController.doPrivileged(Native Method)> at javax.security.auth.Subject.doAs(Subject.java:422)> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)> at java.util.concurrent.FutureTask.run(FutureTask.java:266)> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)> at java.lang.Thread.run(Thread.java:748)> Caused by: java.lang.NullPointerException> at org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)> at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)> ... 19 more > INFO : Completed executing command(queryId=hive_20210927174227_c0ccf861-e7df-4ae4-be7d-b93774d29c25); Time taken: 15.402 seconds> INFO : Concurrency mode is disabled, not creating a lock manager> Error: Error while processing statement: FAILED: Hive Internal Error: org.apache.hadoop.hive.ql.metadata.HiveException(Error while invoking FailureHook. hooks: java.lang.NullPointerException> at org.apache.hadoop.hive.ql.reexec.ReExecutionOverlayPlugin$LocalHook.run(ReExecutionOverlayPlugin.java:45)> at org.apache.hadoop.hive.ql.HookRunner.invokeGeneralHook(HookRunner.java:296)> at org.apache.hadoop.hive.ql.HookRunner.runFailureHooks(HookRunner.java:283)> at org.apache.hadoop.hive.ql.Driver.invokeFailureHooks(Driver.java:2616)> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2386)> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2011)> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1709)> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1703)> at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)> at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:224)> at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:316)> at java.security.AccessController.doPrivileged(Native Method)> at javax.security.auth.Subject.doAs(Subject.java:422)> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)> at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:330)> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)> at java.util.concurrent.FutureTask.run(FutureTask.java:266)> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)> at java.lang.Thread.run(Thread.java:748)> ) (state=08S01,code=12)>
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)