You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Maxence Dunnewind (JIRA)" <ji...@apache.org> on 2016/10/26 18:05:58 UTC
[jira] [Commented] (AMQ-6482) Possible thread leak ?
[ https://issues.apache.org/jira/browse/AMQ-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15609200#comment-15609200 ]
Maxence Dunnewind commented on AMQ-6482:
----------------------------------------
This keeps increasing , ~ 400 thread every 4/5 hours. Right now, on the same instance :
1873 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
1613 (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> Possible thread leak ?
> ----------------------
>
> Key: AMQ-6482
> URL: https://issues.apache.org/jira/browse/AMQ-6482
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.14.1
> Reporter: Maxence Dunnewind
> Labels: threading
> Attachments: thread_dump
>
>
> We have an activemq broker running for ~ 24 hours, handling ~ 100k topics and up to 8k connections (for ~ 24k consumers).
> Looking at my graph, I see that I get a big increase of threads count happended (in 4 steps, ~ 500 more threads each time) as shown here : https://snapshot.raintank.io/dashboard/snapshot/ExUb4pgNYnmBo92JgbZrvJ6fiBm3PNrI
> However, even if the traffic dropped in the night, the thread count never decreased and is still ~ 2500.
> I checked a thread dump and it looks like (full dump attached) :
> ```
> # grep Thread.State thread_dump |awk '{ print $2 }' |sort|uniq -c
> 26 RUNNABLE
> 70 TIMED_WAITING
> 2446 WAITING
> ```
> Mhh, so many WAITING threads, let's check why :
> ```
> # grep 'parking to' thread_dump |cut -d '>' -f 2 |sort |uniq -c
> 1 (a java.util.concurrent.CountDownLatch$Sync)
> 1308 (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 1159 (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> 41 (a java.util.concurrent.SynchronousQueue$TransferStack)
> ```
> I'm not a big java expert but seems a bit unexpected behavior to me.
> For various reason (big number of topics etc ) we use custom GC config, here it is :
> ```
> ACTIVEMQ_OPTS_MEMORY="-Xms128M -Xmx10G -Xloggc:/home/log/activemq/gc.log -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseG1GC -XX:InitiatingHeapOccupancyPercent=75 -XX:NewRatio=4"
> ```
> Maxence
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)