does hive cli support concurrency?

[Xiaomeng Huang <ar...@gmail.com>]

Hi, I found a problem when I use hive.

1. In my environment, I have many threads, and every thread fork a process
to run hive job using "hive -e sql"(not use hiveserver2, just use hive cli).
Accidentally, one hive process couldn't close when the job is fininshed. I
see the job in yarn has finished, even I see "OK" in hive log. But the
process of hive named "runjar" is still running.
When this issue occurs, I can only use "kill -9" to terminate this process.

2. I don't set the configuration "hive.support.concurrency", but I set the
configuration "hive.exec.parallel" be true.

3. My hive version is hive-0.12.0-cdh5.1.0.

4. This is the jstack of the hive process when it hang
$ cat 721783-jstack.txt
2015-06-10 19:59:30
Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode):

"Attach Listener" daemon prio=10 tid=0x00007f0ed403b800 nid=0x5d08 waiting
on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"org.apache.hadoop.hdfs.PeerCache@108e08dc" daemon prio=10
tid=0x00007f0ee8f1c800 nid=0x6188 waiting on condition [0x00007f0ed89ba000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:245)
at org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:41)
at org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:119)
at java.lang.Thread.run(Thread.java:745)

"Thread-24" daemon prio=10 tid=0x00007f0ee8d52800 nid=0x6185 runnable
[0x00007f0ed95e7000]
   java.lang.Thread.State: RUNNABLE
at org.apache.hadoop.net.unix.DomainSocketWatcher.doPoll0(Native Method)
at
org.apache.hadoop.net.unix.DomainSocketWatcher.access$800(DomainSocketWatcher.java:52)
at
org.apache.hadoop.net.unix.DomainSocketWatcher$1.run(DomainSocketWatcher.java:457)
at java.lang.Thread.run(Thread.java:745)

"Abandoned connection cleanup thread" daemon prio=10 tid=0x00007f0ee9148000
nid=0x6180 in Object.wait() [0x00007f0ed96ea000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x00000000e003f0a8> (a java.lang.ref.ReferenceQueue$Lock)
at
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:41)

"Service Thread" daemon prio=10 tid=0x00007f0ee80cc000 nid=0x616e runnable
[0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" daemon prio=10 tid=0x00007f0ee80c9800 nid=0x616d
waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" daemon prio=10 tid=0x00007f0ee80c6800 nid=0x616c
waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x00007f0ee80c5000 nid=0x616b
runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00007f0ee809d800 nid=0x6149 in
Object.wait() [0x00007f0eda9c7000]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e003f2e8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x00000000e003f2e8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" daemon prio=10 tid=0x00007f0ee8096000 nid=0x6148 in
Object.wait() [0x00007f0edaac8000]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e0041870> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0x00000000e0041870> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x00007f0ee8013800 nid=0x6134 runnable
[0x00007f0eecc86000]
   java.lang.Thread.State: RUNNABLE
at java.util.HashMap.put(HashMap.java:494)
at org.apache.hadoop.hive.ql.log.PerfLogger.PerfLogBegin(PerfLogger.java:98)
at org.apache.hadoop.hive.ql.Driver.releaseLocks(Driver.java:909)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1099)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:921)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:357)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:740)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

"VM Thread" prio=10 tid=0x00007f0ee8091800 nid=0x6147 runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f0ee8029800 nid=0x6135
runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f0ee802b000 nid=0x6136
runnable

"GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f0ee802d000 nid=0x6137
runnable

"GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f0ee802f000 nid=0x6138
runnable

"GC task thread#4 (ParallelGC)" prio=10 tid=0x00007f0ee8030800 nid=0x6139
runnable

"GC task thread#5 (ParallelGC)" prio=10 tid=0x00007f0ee8032800 nid=0x613a
runnable

"GC task thread#6 (ParallelGC)" prio=10 tid=0x00007f0ee8034800 nid=0x613b
runnable

"GC task thread#7 (ParallelGC)" prio=10 tid=0x00007f0ee8036800 nid=0x613c
runnable

"GC task thread#8 (ParallelGC)" prio=10 tid=0x00007f0ee8038000 nid=0x613d
runnable

"GC task thread#9 (ParallelGC)" prio=10 tid=0x00007f0ee803a000 nid=0x613e
runnable

"GC task thread#10 (ParallelGC)" prio=10 tid=0x00007f0ee803c000 nid=0x613f
runnable

"GC task thread#11 (ParallelGC)" prio=10 tid=0x00007f0ee803d800 nid=0x6140
runnable

"GC task thread#12 (ParallelGC)" prio=10 tid=0x00007f0ee803f800 nid=0x6141
runnable

"GC task thread#13 (ParallelGC)" prio=10 tid=0x00007f0ee8041800 nid=0x6142
runnable

"GC task thread#14 (ParallelGC)" prio=10 tid=0x00007f0ee8043800 nid=0x6143
runnable

"GC task thread#15 (ParallelGC)" prio=10 tid=0x00007f0ee8045000 nid=0x6144
runnable

"GC task thread#16 (ParallelGC)" prio=10 tid=0x00007f0ee8047000 nid=0x6145
runnable

"GC task thread#17 (ParallelGC)" prio=10 tid=0x00007f0ee8049000 nid=0x6146
runnable

"VM Periodic Task Thread" prio=10 tid=0x00007f0ee80df000 nid=0x616f waiting
on condition

JNI global references: 232


5. So does hive cli support concurrency in this scenario? And how to solve
this problem?

Thanks,
Xiaomeng