[jira] [Created] (HIVE-15936) ConcurrentModificationException in ATSHook

["Daniel Dai (JIRA)" <ji...@apache.org>]

Daniel Dai created HIVE-15936:
---------------------------------

             Summary: ConcurrentModificationException in ATSHook
                 Key: HIVE-15936
                 URL: https://issues.apache.org/jira/browse/HIVE-15936
             Project: Hive
          Issue Type: Bug
            Reporter: Daniel Dai
            Assignee: Daniel Dai
         Attachments: HIVE-15936.1.patch

See ATSHook error:

{noformat}
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) ~[?:1.8.0_112]
	at java.util.HashMap$EntryIterator.next(HashMap.java:1471) ~[?:1.8.0_112]
	at java.util.HashMap$EntryIterator.next(HashMap.java:1469) ~[?:1.8.0_112]
	at java.util.AbstractCollection.toArray(AbstractCollection.java:196) ~[?:1.8.0_112]
	at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:290) ~[guava-14.0.1.jar:?]
	at org.apache.hadoop.hive.ql.log.PerfLogger.getEndTimes(PerfLogger.java:219) ~[hive-common-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
	at org.apache.hadoop.hive.ql.hooks.ATSHook.createPostHookEvent(ATSHook.java:347) ~[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
	at org.apache.hadoop.hive.ql.hooks.ATSHook$2.run(ATSHook.java:206) [hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_112]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{noformat}

According to [~jdere], ATSHook is currently accessing the PerfLogger on a separate thread, which means the main query thread can potentially write to the PerfLogger at the same time.
The ATSHook should access the PerfLogger on the main query thread, before it sends the execution to the ATS Logger thread.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)