You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Vamsee Yarlagadda (JIRA)" <ji...@apache.org> on 2017/02/01 04:26:51 UTC
[jira] [Commented] (HIVE-15778) DROP INDEX (non-existent) throws
NPE when using DbNotificationListener
[ https://issues.apache.org/jira/browse/HIVE-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848003#comment-15848003 ]
Vamsee Yarlagadda commented on HIVE-15778:
------------------------------------------
Here is the [code block in HiveMetaStore.java|https://github.com/apache/hive/blob/4becd689d59ee3f75a36119fbb950c44e16c65df/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4617-L4620] which is calling the listeners.
{code}
for (MetaStoreEventListener listener : listeners) {
DropIndexEvent dropIndexEvent = new DropIndexEvent(index, success, this);
listener.onDropIndex(dropIndexEvent);
}
{code}
Rather than making it a special case for DbNotificationListener to get skipped, this has a potential to raise NPE for any future listeners that get added. So I think if the *index* variable is NULL in the finally block, we could skip the entire processing of MetaStoreEventListeners?
Suggestion:
{code}
if (index != null) {
for (MetaStoreEventListener listener : listeners) {
DropIndexEvent dropIndexEvent = new DropIndexEvent(index, success, this);
listener.onDropIndex(dropIndexEvent);
}
}
{code}
> DROP INDEX (non-existent) throws NPE when using DbNotificationListener
> -----------------------------------------------------------------------
>
> Key: HIVE-15778
> URL: https://issues.apache.org/jira/browse/HIVE-15778
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 2.1.1
> Reporter: Vamsee Yarlagadda
>
> Trying to execute a DROP INDEX operation on a non-existant index throws NPE.
> {code}
> 0: jdbc:hive2://nightly-unsecure-1.gce.cloude> DROP INDEX IF EXISTS vamsee1 ON sample_07;
> INFO : Compiling command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4): DROP INDEX IF EXISTS vamsee1 ON sample_07
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4); Time taken: 0.238 seconds
> INFO : Executing command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4): DROP INDEX IF EXISTS vamsee1 ON sample_07
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException
> INFO : Completed executing command(queryId=hive_20170131162727_663a0909-2a82-44f9-a800-f4a35abaeaa4); Time taken: 0.061 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.NullPointerException (state=08S01,code=1)
> {code}
> HMS log:
> {code}
> 2017-01-31 16:27:29,421 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler: [pool-5-thread-3]: MetaException(message:java.lang.NullPointerException)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newMetaException(HiveMetaStore.java:5823)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.rethrowException(HiveMetaStore.java:4892)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_index_by_name(HiveMetaStore.java:4403)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99)
> at com.sun.proxy.$Proxy16.drop_index_by_name(Unknown Source)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_index_by_name.getResult(ThriftHiveMetastore.java:10803)
> at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_index_by_name.getResult(ThriftHiveMetastore.java:10787)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110)
> at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
> at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.hive.hcatalog.messaging.json.JSONDropIndexMessage.<init>(JSONDropIndexMessage.java:46)
> at org.apache.hive.hcatalog.messaging.json.JSONMessageFactory.buildDropIndexMessage(JSONMessageFactory.java:159)
> at org.apache.hive.hcatalog.listener.DbNotificationListener.onDropIndex(DbNotificationListener.java:280)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_index_by_name_core(HiveMetaStore.java:4469)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_index_by_name(HiveMetaStore.java:4396)
> ... 20 more
> {code}
> Looks like if an exception is raised at [HiveMetaStore#4572|https://github.com/apache/hive/blob/4becd689d59ee3f75a36119fbb950c44e16c65df/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4572] (which gets triggered if we try to drop an index which doesn't exist) , the control directly jumps to finally block where we are trying to call drop index event on various metastore event listeners [HiveMetaStore#4619|https://github.com/apache/hive/blob/4becd689d59ee3f75a36119fbb950c44e16c65df/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L4619].
> If one of the event listeners is DbNotificationListener, then it calls the code under JSONDropIndexMessage.java and this fails with NPE during instantiation at [JSONDropIndexMessage.java#46|https://github.com/apache/hive/blob/4becd689d59ee3f75a36119fbb950c44e16c65df/hcatalog/server-extensions/src/main/java/org/apache/hive/hcatalog/messaging/json/JSONDropIndexMessage.java#L46] as the exception raised in HiveMetaStore.java wouldn't set the index variable.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)