Another memory blow out

[Lionel van den Berg <li...@gmail.com>]

Hi all,

We had another site issue of memory blow out today in our operations system.

AMQ had been running for a month and had been stable at 1GB of memory use.
Then at 3am approximately it started rising by about 50mb every 5 minutes
until we had to restart it 4 hours later to prevent an uncontrolled outage.

I managed to get activemq.log, not much information in there, there are a
bunch of EOFException and a few of already connected messages for one
client. I also got the output of running the query argument to the startup
script. I'm not sure how to read this file completely, it's massive, 68MB.
What I do see is there are around 33K topics, a large number of those with
no subscribers, but I don't think this is necessarily any indication of an
issue, my understanding of our system is that the topic number didn't climb
rapidly.

I noticed on interesting topic which as TotalConsumerCount = 1334, this is
way higher than I expected, could this be an indication of any issue?

What else can I look for in that output? I thought it would describe
everything I would need to understand where the memory is used.

Thanks

Lionel