Thread spike every 5 minutes

[rvoliva <rv...@gmail.com>]

In our ActiveMQ server (5.10.1), the normal thread count is around 90 active
threads.  Every 5 minutes, almost to the second, we get a spike of up to
around 500-550 threads.  From JMX, all the extra threads look like this:

Name: ActiveMQ VMTransport: vm://localhost#302336-1
State: TIMED_WAITING on
java.util.concurrent.SynchronousQueue$TransferStack@3f15281
Total blocked: 0  Total waited: 6

Stack trace: 
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)


Does anyone have any tips as to what this is?

Thanks,
Robert Voliva



--
View this message in context: http://activemq.2283324.n4.nabble.com/Thread-spike-every-5-minutes-tp4699406.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Thread spike every 5 minutes

[Tim Bain <tb...@alumni.duke.edu>]

Here's a theory for what those threads are, though it doesn't explain why
they got created.  Maybe every 5 minutes, something happens that requires
lots of threads.  The thread pools create a bunch of new threads, the tasks
all end quickly, and then the threads sit around waiting for work (which is
what your stack trace shows) for a little while before the thread pool
decides most of them aren't needed and the thread pool size can shrink
again, until the next time.

Under that theory, the stack trace you're showing us won't help anyone
figure out what's going on, because that stack trace isn't the interesting
activity, it's the aftermath.  (Which is probably why you didn't get any
responses to your initial question.)  If you want to figure out what the
activity is, you need to capture it in the act.  Since you can predict it
well, you could try using a performance profiler to capture from right
before to right after the event, and use the method calls it captures to
see if you can deduce what's actually going on.

Tim

On Mon, Jul 20, 2015 at 8:05 AM, rvoliva <rv...@gmail.com> wrote:

> Is there any additional information I could provide so that someone could
> help me out?
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Thread-spike-every-5-minutes-tp4699406p4699552.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Re: Thread spike every 5 minutes

[rvoliva <rv...@gmail.com>]

Is there any additional information I could provide so that someone could
help me out?



--
View this message in context: http://activemq.2283324.n4.nabble.com/Thread-spike-every-5-minutes-tp4699406p4699552.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.