[jira] [Comment Edited] (HIVE-13002) metastore call timing is not threadsafe

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157817#comment-15157817 ] 

Sergey Shelukhin edited comment on HIVE-13002 at 2/22/16 10:13 PM:
-------------------------------------------------------------------

There are ~12 places where Hive object is stored in a field. The most prominent are BaseSemanticAnalyzer and Task, where they are obviously used by 10000 methods/subclasses. The Task is esp. suspect since many of these would be executed on different threads. There's also one in HS2 SessionState, which as far as I know is not tied to the thread. There are also other less prominent places where it can happen. We can fix this issue and file a follow-up JIRA to clean up all this mess.

Maybe we should also rename the class Hive to HiveThreadLocal when we do the cleanup :)


was (Author: sershe):
There are ~12 places where Hive object is stored in a field. The most prominent are BaseSemanticAnalyzer and Task, where they are obviously used by 10000 methods/subclasses. The Task is esp. suspect since many of these would be executed on different threads. There's also one in HS2 SessionState, which as far as I know is not tied to the thread. There are also other less prominent places where it can happen. We can fix this issue and file a follow-up JIRA to clean up all this mess.

> metastore call timing is not threadsafe
> ---------------------------------------
>
>                 Key: HIVE-13002
>                 URL: https://issues.apache.org/jira/browse/HIVE-13002
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-13002.01.patch, HIVE-13002.patch
>
>
> Discovered in some q test run:
> {noformat}
>  TestCliDriver.testCliDriver_insert_values_orig_table:123->runTest:199 Unexpected exception java.util.ConcurrentModificationException
> 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
> 	at java.util.HashMap$EntryIterator.next(HashMap.java:966)
> 	at java.util.HashMap$EntryIterator.next(HashMap.java:964)
> 	at org.apache.hadoop.hive.ql.metadata.Hive.dumpAndClearMetaCallTiming(Hive.java:3412)
> 	at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(Driver.java:574)
> 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1722)
> 	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1342)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1113)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1101)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)