You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Dawid Weiss (JIRA)" <ji...@apache.org> on 2012/12/17 10:00:15 UTC
[jira] [Commented] (SOLR-4205) Clover runs on ASF Jenkins idle dead
without a test or any thread running in main() loop waiting for file
descriptor
[ https://issues.apache.org/jira/browse/SOLR-4205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533735#comment-13533735 ]
Dawid Weiss commented on SOLR-4205:
-----------------------------------
This definitely looks like a bug in the runner although I have no idea how to reproduce this or what may be causing it. The symptoms are: truncated events file:
{code}
[
"SUITE_FAILURE"
{code}
and the main test thread gone. This is very weird.
> Clover runs on ASF Jenkins idle dead without a test or any thread running in main() loop waiting for file descriptor
> --------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-4205
> URL: https://issues.apache.org/jira/browse/SOLR-4205
> Project: Solr
> Issue Type: Bug
> Components: Tests
> Environment: FreeBSD Jenkins
> Reporter: Uwe Schindler
> Assignee: Dawid Weiss
>
> I had to kill ASF Jenkins Clover builds two times after several 10 hours of inactivity in a random Solr test. I requested a stack trace before killing the only running JVM (clover runs with one JVM only, because clover does not like multiple processes writing the same clover metrics file).
> In both cases (4.x and trunk) the stack trace was looking identical after sending kill -3...
> https://builds.apache.org/job/Lucene-Solr-Clover-trunk/76/consoleFull (yesterday):
> {noformat}
> [junit4:junit4] HEARTBEAT J0 PID(81884@lucene.zones.apache.org): 2012-12-16T13:01:00, stalled for 28447s at: TestFunctionQuery.testBooleanFunctions
> [junit4:junit4] HEARTBEAT J0 PID(81884@lucene.zones.apache.org): 2012-12-16T13:02:00, stalled for 28507s at: TestFunctionQuery.testBooleanFunctions
> [junit4:junit4] HEARTBEAT J0 PID(81884@lucene.zones.apache.org): 2012-12-16T13:03:00, stalled for 28567s at: TestFunctionQuery.testBooleanFunctions
> [junit4:junit4] JVM J0: stdout was not empty, see: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Clover-trunk/solr/build/solr-core/test/temp/junit4-J0-20121216_044733_583.sysout
> [junit4:junit4] >>> JVM J0: stdout (verbatim) ----
> [junit4:junit4] 2012-12-16 13:03:49
> [junit4:junit4] Full thread dump OpenJDK 64-Bit Server VM (20.0-b12 mixed mode):
> [junit4:junit4]
> [junit4:junit4] "searcherExecutor-2577-thread-1" prio=5 tid=0x000000085eb67000 nid=0x61c105b waiting on condition [0x00007ffff0b0d000]
> [junit4:junit4] java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x00000008178c9c40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4] at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:386)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000840ce2800 nid=0x61c0aa2 runnable [0x00007ffff9496000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
> [junit4:junit4] at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
> [junit4:junit4] at java.net.ServerSocket.implAccept(ServerSocket.java:470)
> [junit4:junit4] at java.net.ServerSocket.accept(ServerSocket.java:438)
> [junit4:junit4] at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:387)
> [junit4:junit4] at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI Scheduler(0)" daemon prio=5 tid=0x0000000840ce1000 nid=0x61c0969 waiting on condition [0x00007ffff0f11000]
> [junit4:junit4] java.lang.Thread.State: WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000814f12f88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4] at java.util.concurrent.DelayQueue.take(DelayQueue.java:189)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "GC Daemon" daemon prio=5 tid=0x0000000840ce0000 nid=0x61c0949 in Object.wait() [0x00007ffff6d6f000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at sun.misc.GC$Daemon.run(GC.java:117)
> [junit4:junit4] - locked <0x0000000814f1c428> (a sun.misc.GC$LatencyLock)
> [junit4:junit4]
> [junit4:junit4] "RMI Reaper" prio=5 tid=0x0000000840cdf800 nid=0x61c0947 in Object.wait() [0x00007ffff7577000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
> [junit4:junit4] - locked <0x0000000814f1ad08> (a java.lang.ref.ReferenceQueue$Lock)
> [junit4:junit4] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
> [junit4:junit4] at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:350)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000840cde800 nid=0x61c0936 runnable [0x00007ffff6e70000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
> [junit4:junit4] at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
> [junit4:junit4] at java.net.ServerSocket.implAccept(ServerSocket.java:470)
> [junit4:junit4] at java.net.ServerSocket.accept(ServerSocket.java:438)
> [junit4:junit4] at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:387)
> [junit4:junit4] at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000840cde000 nid=0x61c0934 runnable [0x00007ffff494b000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.net.PlainSocketImpl.socketAccept(Native Method)
> [junit4:junit4] at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:375)
> [junit4:junit4] at java.net.ServerSocket.implAccept(ServerSocket.java:470)
> [junit4:junit4] at java.net.ServerSocket.accept(ServerSocket.java:438)
> [junit4:junit4] at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:387)
> [junit4:junit4] at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:359)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "TimeLimitedCollector timer thread" daemon prio=5 tid=0x0000000848e69800 nid=0x61c096d waiting on condition [0x00007ffff3a3c000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:267)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-2" daemon prio=5 tid=0x000000084c8c6800 nid=0x61c087d runnable [0x00007ffffdfe1000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000811568070> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-1" daemon prio=5 tid=0x000000084c8c5000 nid=0x61c087b runnable [0x00007ffffe1e3000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000811568070> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "Low Memory Detector" daemon prio=5 tid=0x00000008011c4800 nid=0x61c0876 runnable [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread1" daemon prio=5 tid=0x00000008011c3800 nid=0x61c0875 waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread0" daemon prio=5 tid=0x00000008011c3000 nid=0x61c0874 waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Signal Dispatcher" daemon prio=5 tid=0x00000008011c2000 nid=0x61c0873 waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Finalizer" daemon prio=5 tid=0x00000008011c1800 nid=0x61c0872 in Object.wait() [0x00007ffffebed000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
> [junit4:junit4] - locked <0x0000000810cec8f8> (a java.lang.ref.ReferenceQueue$Lock)
> [junit4:junit4] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
> [junit4:junit4] at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
> [junit4:junit4]
> [junit4:junit4] "Reference Handler" daemon prio=5 tid=0x00000008011c0800 nid=0x61c0871 in Object.wait() [0x00007ffffecee000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at java.lang.Object.wait(Object.java:502)
> [junit4:junit4] at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> [junit4:junit4] - locked <0x0000000810ce8b80> (a java.lang.ref.Reference$Lock)
> [junit4:junit4]
> [junit4:junit4] "main" prio=5 tid=0x00000008011c0000 nid=0x61c0672 runnable [0x00007fffffbfd000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.io.FileInputStream.readBytes(Native Method)
> [junit4:junit4] at java.io.FileInputStream.read(FileInputStream.java:236)
> [junit4:junit4] at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
> [junit4:junit4] at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> [junit4:junit4] - locked <0x0000000810ce22e0> (a java.io.BufferedInputStream)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
> [junit4:junit4] - locked <0x0000000810ce22c8> (a java.io.InputStreamReader)
> [junit4:junit4] at java.io.InputStreamReader.read(InputStreamReader.java:184)
> [junit4:junit4] at java.io.BufferedReader.fill(BufferedReader.java:153)
> [junit4:junit4] at java.io.BufferedReader.readLine(BufferedReader.java:316)
> [junit4:junit4] - locked <0x0000000810ce22c8> (a java.io.InputStreamReader)
> [junit4:junit4] at java.io.BufferedReader.readLine(BufferedReader.java:379)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:31)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:13)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.Iterators$5.hasNext(Iterators.java:539)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:150)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:255)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
> [junit4:junit4]
> [junit4:junit4] "VM Thread" prio=5 tid=0x000000080121e800 nid=0x61c0870 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#0 (ParallelGC)" prio=5 tid=0x0000000801215800 nid=0x61c0863 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#1 (ParallelGC)" prio=5 tid=0x0000000801216800 nid=0x61c0864 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#2 (ParallelGC)" prio=5 tid=0x0000000801217000 nid=0x61c0865 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#3 (ParallelGC)" prio=5 tid=0x0000000801217800 nid=0x61c0866 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#4 (ParallelGC)" prio=5 tid=0x0000000801218000 nid=0x61c0867 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#5 (ParallelGC)" prio=5 tid=0x0000000801219000 nid=0x61c0868 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#6 (ParallelGC)" prio=5 tid=0x0000000801219800 nid=0x61c0869 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#7 (ParallelGC)" prio=5 tid=0x000000080121a000 nid=0x61c086a runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#8 (ParallelGC)" prio=5 tid=0x000000080121a800 nid=0x61c086b runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#9 (ParallelGC)" prio=5 tid=0x000000080121b800 nid=0x61c086c runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#10 (ParallelGC)" prio=5 tid=0x000000080121c000 nid=0x61c086d runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#11 (ParallelGC)" prio=5 tid=0x000000080121c800 nid=0x61c086e runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#12 (ParallelGC)" prio=5 tid=0x000000080121d000 nid=0x61c086f runnable
> [junit4:junit4]
> [junit4:junit4] "VM Periodic Task Thread" prio=5 tid=0x000000080121f000 nid=0x61c0877 waiting on condition
> [junit4:junit4]
> [junit4:junit4] JNI global references: 1183
> [junit4:junit4]
> [junit4:junit4] Heap
> [junit4:junit4] PSYoungGen total 212224K, used 110789K [0x0000000830220000, 0x000000083d1c0000, 0x0000000840220000)
> [junit4:junit4] eden space 211840K, 52% used [0x0000000830220000,0x0000000836e516f0,0x000000083d100000)
> [junit4:junit4] from space 384K, 0% used [0x000000083d160000,0x000000083d160000,0x000000083d1c0000)
> [junit4:junit4] to space 384K, 0% used [0x000000083d100000,0x000000083d100000,0x000000083d160000)
> [junit4:junit4] PSOldGen total 524288K, used 127127K [0x0000000810220000, 0x0000000830220000, 0x0000000830220000)
> [junit4:junit4] object space 524288K, 24% used [0x0000000810220000,0x0000000817e45ef0,0x0000000830220000)
> [junit4:junit4] PSPermGen total 83968K, used 83967K [0x000000080b020000, 0x0000000810220000, 0x0000000810220000)
> [junit4:junit4] object space 83968K, 99% used [0x000000080b020000,0x000000081021fd98,0x0000000810220000)
> [junit4:junit4] <<< JVM J0: EOF ----
> {noformat}
> https://builds.apache.org/job/Lucene-Solr-Clover-4.x/77/consoleFull (today):
> {noformat}
> [junit4:junit4] HEARTBEAT J0 PID(2916@lucene.zones.apache.org): 2012-12-16T22:11:23, stalled for 13809s at: QueryEqualityTest (suite)
> [junit4:junit4] HEARTBEAT J0 PID(2916@lucene.zones.apache.org): 2012-12-16T22:12:23, stalled for 13869s at: QueryEqualityTest (suite)
> [junit4:junit4] HEARTBEAT J0 PID(2916@lucene.zones.apache.org): 2012-12-16T22:13:23, stalled for 13929s at: QueryEqualityTest (suite)
> [junit4:junit4] HEARTBEAT J0 PID(2916@lucene.zones.apache.org): 2012-12-16T22:14:23, stalled for 13989s at: QueryEqualityTest (suite)
> [junit4:junit4] JVM J0: stdout was not empty, see: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Clover-4.x/solr/build/solr-core/test/temp/junit4-J0-20121216_175857_638.sysout
> [junit4:junit4] >>> JVM J0: stdout (verbatim) ----
> [junit4:junit4] 2012-12-16 22:14:25
> [junit4:junit4] Full thread dump OpenJDK 64-Bit Server VM (20.0-b12 mixed mode):
> [junit4:junit4]
> [junit4:junit4] "TimeLimitedCollector timer thread" daemon prio=5 tid=0x00000008460dc800 nid=0x61c9fa3 waiting on condition [0x00007ffffd8da000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (sleeping)
> [junit4:junit4] at java.lang.Thread.sleep(Native Method)
> [junit4:junit4] at org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:267)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-2" daemon prio=5 tid=0x000000084b749800 nid=0x61c9f96 runnable [0x00007ffffe0e2000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000810dd3f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "metrics-meter-tick-thread-1" daemon prio=5 tid=0x000000084b749000 nid=0x61c9f95 runnable [0x00007ffffe1e3000]
> [junit4:junit4] java.lang.Thread.State: TIMED_WAITING (parking)
> [junit4:junit4] at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4] - parking to wait for <0x0000000810dd3f20> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> [junit4:junit4] at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> [junit4:junit4] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2081)
> [junit4:junit4] at java.util.concurrent.DelayQueue.take(DelayQueue.java:193)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:688)
> [junit4:junit4] at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:681)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4] at java.lang.Thread.run(Thread.java:679)
> [junit4:junit4]
> [junit4:junit4] "Low Memory Detector" daemon prio=5 tid=0x00000008011c4800 nid=0x61c9f90 runnable [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread1" daemon prio=5 tid=0x00000008011c3800 nid=0x61c9f8f waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "C2 CompilerThread0" daemon prio=5 tid=0x00000008011c3000 nid=0x61c9f8e waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Signal Dispatcher" daemon prio=5 tid=0x00000008011c2000 nid=0x61c9f8d waiting on condition [0x0000000000000000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4]
> [junit4:junit4] "Finalizer" daemon prio=5 tid=0x00000008011c1800 nid=0x61c9f8c in Object.wait() [0x00007ffffebed000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
> [junit4:junit4] - locked <0x00000008109642d8> (a java.lang.ref.ReferenceQueue$Lock)
> [junit4:junit4] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
> [junit4:junit4] at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
> [junit4:junit4]
> [junit4:junit4] "Reference Handler" daemon prio=5 tid=0x00000008011c0800 nid=0x61c9f8b in Object.wait() [0x00007ffffecee000]
> [junit4:junit4] java.lang.Thread.State: WAITING (on object monitor)
> [junit4:junit4] at java.lang.Object.wait(Native Method)
> [junit4:junit4] at java.lang.Object.wait(Object.java:502)
> [junit4:junit4] at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> [junit4:junit4] - locked <0x0000000810967160> (a java.lang.ref.Reference$Lock)
> [junit4:junit4]
> [junit4:junit4] "main" prio=5 tid=0x00000008011c0000 nid=0x61c9d83 runnable [0x00007fffffbfd000]
> [junit4:junit4] java.lang.Thread.State: RUNNABLE
> [junit4:junit4] at java.io.FileInputStream.readBytes(Native Method)
> [junit4:junit4] at java.io.FileInputStream.read(FileInputStream.java:236)
> [junit4:junit4] at java.io.BufferedInputStream.read1(BufferedInputStream.java:273)
> [junit4:junit4] at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
> [junit4:junit4] - locked <0x000000081098f1f8> (a java.io.BufferedInputStream)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324)
> [junit4:junit4] at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176)
> [junit4:junit4] - locked <0x000000081098f1e0> (a java.io.InputStreamReader)
> [junit4:junit4] at java.io.InputStreamReader.read(InputStreamReader.java:184)
> [junit4:junit4] at java.io.BufferedReader.fill(BufferedReader.java:153)
> [junit4:junit4] at java.io.BufferedReader.readLine(BufferedReader.java:316)
> [junit4:junit4] - locked <0x000000081098f1e0> (a java.io.InputStreamReader)
> [junit4:junit4] at java.io.BufferedReader.readLine(BufferedReader.java:379)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:31)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.StdInLineIterator.computeNext(StdInLineIterator.java:13)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.collect.Iterators$5.hasNext(Iterators.java:539)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:150)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:255)
> [junit4:junit4] at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
> [junit4:junit4]
> [junit4:junit4] "VM Thread" prio=5 tid=0x000000080121e800 nid=0x61c9f8a runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#0 (ParallelGC)" prio=5 tid=0x0000000801215800 nid=0x61c9f7d runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#1 (ParallelGC)" prio=5 tid=0x0000000801216800 nid=0x61c9f7e runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#2 (ParallelGC)" prio=5 tid=0x0000000801217000 nid=0x61c9f7f runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#3 (ParallelGC)" prio=5 tid=0x0000000801217800 nid=0x61c9f80 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#4 (ParallelGC)" prio=5 tid=0x0000000801218000 nid=0x61c9f81 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#5 (ParallelGC)" prio=5 tid=0x0000000801219000 nid=0x61c9f82 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#6 (ParallelGC)" prio=5 tid=0x0000000801219800 nid=0x61c9f83 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#7 (ParallelGC)" prio=5 tid=0x000000080121a000 nid=0x61c9f84 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#8 (ParallelGC)" prio=5 tid=0x000000080121a800 nid=0x61c9f85 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#9 (ParallelGC)" prio=5 tid=0x000000080121b800 nid=0x61c9f86 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#10 (ParallelGC)" prio=5 tid=0x000000080121c000 nid=0x61c9f87 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#11 (ParallelGC)" prio=5 tid=0x000000080121c800 nid=0x61c9f88 runnable
> [junit4:junit4]
> [junit4:junit4] "GC task thread#12 (ParallelGC)" prio=5 tid=0x000000080121d000 nid=0x61c9f89 runnable
> [junit4:junit4]
> [junit4:junit4] "VM Periodic Task Thread" prio=5 tid=0x000000080121f000 nid=0x61c9f91 waiting on condition
> [junit4:junit4]
> [junit4:junit4] JNI global references: 1271
> [junit4:junit4]
> [junit4:junit4] Heap
> [junit4:junit4] PSYoungGen total 229120K, used 170968K [0x0000000830220000, 0x000000083eb60000, 0x0000000840220000)
> [junit4:junit4] eden space 228992K, 74% used [0x0000000830220000,0x000000083a8fe360,0x000000083e1c0000)
> [junit4:junit4] from space 128K, 75% used [0x000000083eb40000,0x000000083eb58000,0x000000083eb60000)
> [junit4:junit4] to space 384K, 0% used [0x000000083eaa0000,0x000000083eaa0000,0x000000083eb00000)
> [junit4:junit4] PSOldGen total 524288K, used 179710K [0x0000000810220000, 0x0000000830220000, 0x0000000830220000)
> [junit4:junit4] object space 524288K, 34% used [0x0000000810220000,0x000000081b19fa98,0x0000000830220000)
> [junit4:junit4] PSPermGen total 83968K, used 83967K [0x000000080b020000, 0x0000000810220000, 0x0000000810220000)
> [junit4:junit4] object space 83968K, 99% used [0x000000080b020000,0x000000081021fc68,0x0000000810220000)
> [junit4:junit4] <<< JVM J0: EOF ----
> {noformat}
> To me it looks like there is something wrong with the test runner. In other cases this totally-hanging threads were caused by permgen-errors, but nothing like this in the logs. It looks like the slave is waiting forever to get some messages from the master or vice versa.
> You can imitate clover tests on ASF Jenkins by running:
> {noformat}
> ANT_OPTS="-Xmx1536M" ant -Dtests.jettyConnector=Socket jenkins-clover
> {noformat}
> (the ANT_OPTS are just for the generate-clover-task using lots of memory - totally unrelated. This is just needed after the tests ran successfully). jenkins-clover calls run-clover task which automatically gives more memory to the child JVMs, so maybe this is related and needs more memory. You can find that in the main common-build.xml file, where you will find a property definition if="run.clover" giving more heap and code cache to childs:
> {code:xml}
> <condition property="tests.heapsize" value="768M">
> <isset property="run.clover"/>
> </condition>
> <property name="tests.heapsize" value="512M"/>
>
> <condition property="tests.clover.args" value="-XX:ReservedCodeCacheSize=128m">
> <isset property="run.clover"/>
> </condition>
> <property name="tests.clover.args" value=""/>
> {code}
> (maybe tweak that if Solr needs more memory - Lucene clover passes fine).
> I disabled clover tests for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org