You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by SteveCherniak <ch...@us.ibm.com> on 2018/09/13 03:11:16 UTC

AMQ 5.8.0 leak on repetitive connections

AMQ 5.8.0 - persistence false

recreate leak with following:

Add connection A
Loop
    Add connection B
    Add consumer on B
    Add producer on A
    Remove producer on A
    Remove consumer on B
    Remove disconnect B

Triggered heap dumps after running test for a while. If left will run out of
storage:

Heap usage  288,896,736 bytes

Object counts:

986,690 java/lang/Object
752,581 java/lang/String
636,931 java/util/concurrent/locks/ReentrantLock
636,931 java/util/concurrent/locks/ReentrantLock$NonfairSync
636,893 java/util/concurrent/CopyOnWriteArrayList
479,962 char[]
463,256 java/util/concurrent/atomic/AtomicLong
463,183 org/apache/activemq/management/CountStatisticImpl
289,690 java/util/LinkedList
234,940 java/util/HashMap$Node
231,779 java/util/concurrent/atomic/AtomicBoolean
231,569 org/apache/activemq/usage/DefaultUsageCapacity
155,569 java/util/concurrent/ConcurrentHashMap$Node
154,979 java/util/HashMap
116,377 java/util/ArrayList
115,840 org/apache/activemq/filter/DestinationMapNode
58,276 java/util/concurrent/ConcurrentHashMap
58,162 org/apache/activemq/command/ActiveMQTopic

Leak suspects:

65,124,744 (22.54%) [131,072] 12,522 array of java/util/HashMap$Node
0xf054df70
|- 20,928 (0.01%) [24] 3 java/util/HashMap$Node 0xe1201338
|  |- 16,752 (0.01%) [24] 3 java/util/HashMap$Node 0xe44345c0
|  |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
0xe1201360
|  |- 40 (0%) [16] 1 java/lang/String 0xe1201350
|- 20,912 (0.01%) [24] 3 java/util/HashMap$Node 0xe4f77f48
   |- 16,736 (0.01%) [24] 3 java/util/HashMap$Node 0xe5158a38
   |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
0xe4f78e38
   |- 40 (0%) [16] 1 java/lang/String 0xe4f78e28
...

65,112,872 (22.54%) [131,072] 12,522 array of java/util/HashMap$Node
0xf052df68
|- 20,928 (0.01%) [24] 3 java/util/HashMap$Node 0xe1202f98
|  |- 16,752 (0.01%) [24] 3 java/util/HashMap$Node 0xe4434590
|  |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
0xe1202fb0
|  |- 40 (0%) [16] 1 java/lang/String 0xe12256c0
|- 20,912 (0.01%) [24] 3 java/util/HashMap$Node 0xe2e25340
   |- 16,736 (0.01%) [24] 3 java/util/HashMap$Node 0xe341a428
   |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
0xe2e25c88
   |- 40 (0%) [16] 1 java/lang/String 0xe2e25c78
...

53,796,568 (18.62%) [131,072] 12,522 array of java/util/HashMap$Node
0xee232140
|- 17,280 (0.01%) [24] 3 java/util/HashMap$Node 0xe2e25388
|  |- 13,824 (0%) [24] 3 java/util/HashMap$Node 0xe341a130
|  |- 3,432 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
0xe2e25d18
|  |- 40 (0%) [16] 1 java/lang/String 0xe2e25d08
|- 17,280 (0.01%) [24] 3 java/util/HashMap$Node 0xe121ee00
   |- 13,824 (0%) [24] 3 java/util/HashMap$Node 0xe44346f0
   |- 3,432 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
0xe121ee18
   |- 40 (0%) [16] 1 java/lang/String 0xe11fd368
...

Searched error database. Did not find equivalent issue. I realize that this
scenario is not typical, but leak is consistent. Is this a known issue? Was
it fixed in a later release (> 5.8.0)?

Thanks,
Steve



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: AMQ 5.8.0 leak on repetitive connections

Posted by SteveCherniak <ch...@us.ibm.com>.
OK, I will try a newer version to see if that fixes things.
Steve



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: AMQ 5.8.0 leak on repetitive connections

Posted by Christopher Shannon <ch...@gmail.com>.
Version 5.8.0 is nearly 6 years old and there have been hundreds (maybe
thousands) of commits since then and one of them could have addressed your
issue even if you didn't find an exact match in Jira.  However, no one is
going to remember that far back to see if it is a known issue so you will
just need to test against the latest release as Justin pointed out.


On Wed, Sep 12, 2018 at 11:39 PM Justin Bertram <jb...@apache.org> wrote:

> I have no idea if this is a known issue, but it seems like it would simple
> enough to run your test against the latest release to see if the issue
> still exists there.
>
>
> Justin
>
> On Wed, Sep 12, 2018 at 10:11 PM, SteveCherniak <ch...@us.ibm.com>
> wrote:
>
> > AMQ 5.8.0 - persistence false
> >
> > recreate leak with following:
> >
> > Add connection A
> > Loop
> >     Add connection B
> >     Add consumer on B
> >     Add producer on A
> >     Remove producer on A
> >     Remove consumer on B
> >     Remove disconnect B
> >
> > Triggered heap dumps after running test for a while. If left will run out
> > of
> > storage:
> >
> > Heap usage  288,896,736 bytes
> >
> > Object counts:
> >
> > 986,690 java/lang/Object
> > 752,581 java/lang/String
> > 636,931 java/util/concurrent/locks/ReentrantLock
> > 636,931 java/util/concurrent/locks/ReentrantLock$NonfairSync
> > 636,893 java/util/concurrent/CopyOnWriteArrayList
> > 479,962 char[]
> > 463,256 java/util/concurrent/atomic/AtomicLong
> > 463,183 org/apache/activemq/management/CountStatisticImpl
> > 289,690 java/util/LinkedList
> > 234,940 java/util/HashMap$Node
> > 231,779 java/util/concurrent/atomic/AtomicBoolean
> > 231,569 org/apache/activemq/usage/DefaultUsageCapacity
> > 155,569 java/util/concurrent/ConcurrentHashMap$Node
> > 154,979 java/util/HashMap
> > 116,377 java/util/ArrayList
> > 115,840 org/apache/activemq/filter/DestinationMapNode
> > 58,276 java/util/concurrent/ConcurrentHashMap
> > 58,162 org/apache/activemq/command/ActiveMQTopic
> >
> > Leak suspects:
> >
> > 65,124,744 (22.54%) [131,072] 12,522 array of java/util/HashMap$Node
> > 0xf054df70
> > |- 20,928 (0.01%) [24] 3 java/util/HashMap$Node 0xe1201338
> > |  |- 16,752 (0.01%) [24] 3 java/util/HashMap$Node 0xe44345c0
> > |  |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> > 0xe1201360
> > |  |- 40 (0%) [16] 1 java/lang/String 0xe1201350
> > |- 20,912 (0.01%) [24] 3 java/util/HashMap$Node 0xe4f77f48
> >    |- 16,736 (0.01%) [24] 3 java/util/HashMap$Node 0xe5158a38
> >    |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> > 0xe4f78e38
> >    |- 40 (0%) [16] 1 java/lang/String 0xe4f78e28
> > ...
> >
> > 65,112,872 (22.54%) [131,072] 12,522 array of java/util/HashMap$Node
> > 0xf052df68
> > |- 20,928 (0.01%) [24] 3 java/util/HashMap$Node 0xe1202f98
> > |  |- 16,752 (0.01%) [24] 3 java/util/HashMap$Node 0xe4434590
> > |  |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> > 0xe1202fb0
> > |  |- 40 (0%) [16] 1 java/lang/String 0xe12256c0
> > |- 20,912 (0.01%) [24] 3 java/util/HashMap$Node 0xe2e25340
> >    |- 16,736 (0.01%) [24] 3 java/util/HashMap$Node 0xe341a428
> >    |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> > 0xe2e25c88
> >    |- 40 (0%) [16] 1 java/lang/String 0xe2e25c78
> > ...
> >
> > 53,796,568 (18.62%) [131,072] 12,522 array of java/util/HashMap$Node
> > 0xee232140
> > |- 17,280 (0.01%) [24] 3 java/util/HashMap$Node 0xe2e25388
> > |  |- 13,824 (0%) [24] 3 java/util/HashMap$Node 0xe341a130
> > |  |- 3,432 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> > 0xe2e25d18
> > |  |- 40 (0%) [16] 1 java/lang/String 0xe2e25d08
> > |- 17,280 (0.01%) [24] 3 java/util/HashMap$Node 0xe121ee00
> >    |- 13,824 (0%) [24] 3 java/util/HashMap$Node 0xe44346f0
> >    |- 3,432 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> > 0xe121ee18
> >    |- 40 (0%) [16] 1 java/lang/String 0xe11fd368
> > ...
> >
> > Searched error database. Did not find equivalent issue. I realize that
> this
> > scenario is not typical, but leak is consistent. Is this a known issue?
> Was
> > it fixed in a later release (> 5.8.0)?
> >
> > Thanks,
> > Steve
> >
> >
> >
> > --
> > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> > f2341805.html
> >
>

Re: AMQ 5.8.0 leak on repetitive connections

Posted by Justin Bertram <jb...@apache.org>.
I have no idea if this is a known issue, but it seems like it would simple
enough to run your test against the latest release to see if the issue
still exists there.


Justin

On Wed, Sep 12, 2018 at 10:11 PM, SteveCherniak <ch...@us.ibm.com> wrote:

> AMQ 5.8.0 - persistence false
>
> recreate leak with following:
>
> Add connection A
> Loop
>     Add connection B
>     Add consumer on B
>     Add producer on A
>     Remove producer on A
>     Remove consumer on B
>     Remove disconnect B
>
> Triggered heap dumps after running test for a while. If left will run out
> of
> storage:
>
> Heap usage  288,896,736 bytes
>
> Object counts:
>
> 986,690 java/lang/Object
> 752,581 java/lang/String
> 636,931 java/util/concurrent/locks/ReentrantLock
> 636,931 java/util/concurrent/locks/ReentrantLock$NonfairSync
> 636,893 java/util/concurrent/CopyOnWriteArrayList
> 479,962 char[]
> 463,256 java/util/concurrent/atomic/AtomicLong
> 463,183 org/apache/activemq/management/CountStatisticImpl
> 289,690 java/util/LinkedList
> 234,940 java/util/HashMap$Node
> 231,779 java/util/concurrent/atomic/AtomicBoolean
> 231,569 org/apache/activemq/usage/DefaultUsageCapacity
> 155,569 java/util/concurrent/ConcurrentHashMap$Node
> 154,979 java/util/HashMap
> 116,377 java/util/ArrayList
> 115,840 org/apache/activemq/filter/DestinationMapNode
> 58,276 java/util/concurrent/ConcurrentHashMap
> 58,162 org/apache/activemq/command/ActiveMQTopic
>
> Leak suspects:
>
> 65,124,744 (22.54%) [131,072] 12,522 array of java/util/HashMap$Node
> 0xf054df70
> |- 20,928 (0.01%) [24] 3 java/util/HashMap$Node 0xe1201338
> |  |- 16,752 (0.01%) [24] 3 java/util/HashMap$Node 0xe44345c0
> |  |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> 0xe1201360
> |  |- 40 (0%) [16] 1 java/lang/String 0xe1201350
> |- 20,912 (0.01%) [24] 3 java/util/HashMap$Node 0xe4f77f48
>    |- 16,736 (0.01%) [24] 3 java/util/HashMap$Node 0xe5158a38
>    |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> 0xe4f78e38
>    |- 40 (0%) [16] 1 java/lang/String 0xe4f78e28
> ...
>
> 65,112,872 (22.54%) [131,072] 12,522 array of java/util/HashMap$Node
> 0xf052df68
> |- 20,928 (0.01%) [24] 3 java/util/HashMap$Node 0xe1202f98
> |  |- 16,752 (0.01%) [24] 3 java/util/HashMap$Node 0xe4434590
> |  |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> 0xe1202fb0
> |  |- 40 (0%) [16] 1 java/lang/String 0xe12256c0
> |- 20,912 (0.01%) [24] 3 java/util/HashMap$Node 0xe2e25340
>    |- 16,736 (0.01%) [24] 3 java/util/HashMap$Node 0xe341a428
>    |- 4,112 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> 0xe2e25c88
>    |- 40 (0%) [16] 1 java/lang/String 0xe2e25c78
> ...
>
> 53,796,568 (18.62%) [131,072] 12,522 array of java/util/HashMap$Node
> 0xee232140
> |- 17,280 (0.01%) [24] 3 java/util/HashMap$Node 0xe2e25388
> |  |- 13,824 (0%) [24] 3 java/util/HashMap$Node 0xe341a130
> |  |- 3,432 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> 0xe2e25d18
> |  |- 40 (0%) [16] 1 java/lang/String 0xe2e25d08
> |- 17,280 (0.01%) [24] 3 java/util/HashMap$Node 0xe121ee00
>    |- 13,824 (0%) [24] 3 java/util/HashMap$Node 0xe44346f0
>    |- 3,432 (0%) [32] 4 org/apache/activemq/filter/DestinationMapNode
> 0xe121ee18
>    |- 40 (0%) [16] 1 java/lang/String 0xe11fd368
> ...
>
> Searched error database. Did not find equivalent issue. I realize that this
> scenario is not typical, but leak is consistent. Is this a known issue? Was
> it fixed in a later release (> 5.8.0)?
>
> Thanks,
> Steve
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>