Memory Use

[Charles Anthony <ch...@hpdsoftware.com>]

Hi,

We're doing a stress test on our app, and we're trying to test with
thousands of concurrent users.

Each user has a connection to a (standalone) broker, and each user
(currently) has two temporary queues. There is also a central "normal"
queue.

We are getting garbage collection total freezes after a while with this
config. We are just using a tcp: transport, and to begin with we were
not using persistence. This is using activemq 4.0 (ish - a little before
the release).

We then enabled persistence (derby + journal) - and we lasted a little
longer, but the same issue arose.

I then wrote a little test harness that simply created 50 connections,
with two temp queues per session - and the heap grew by 10 Mb (non
gc-able). That works out at about 200k per connection; so, for 1000
connections, that's 200 Mb on the heap before we even send any messages
! Is there anyway to decrease this amount-per-connection ? Is there some
kind of  configureable buffer per connection somewhere ?

Our application server IS processing messages as the stress test
progresses, and JConsole is showing that the main queue rarely has more
than 30 messages, and typically only has 5 or six sitting there. Is
there any way to see how BIG these messages are ? I don't think they'll
be very big (50-60k is my guess), but I'd like to know.

I thought the introduction of the persistence would reduce the memory
usage, but it seems not.

I've just downloaded 4.0.2, and tried my how-much-space-per-connection
test - and it remains at 200k ish. I'll give the stress test using 4.02
a try in a bit - but any pointers would be helpful; I've run Jprofiler
against the broker, but nothing obvious is showing up.

Cheers,

Charles.

Re: Memory Use

[James Strachan <ja...@gmail.com>]

Am thinking this could be a good place to start...

http://issues.apache.org/activemq/browse/AMQ-1001

On 11/17/06, Charles Anthony <ch...@hpdsoftware.com> wrote:
> Hi,
>
> We're doing a stress test on our app, and we're trying to test with
> thousands of concurrent users.
>
> Each user has a connection to a (standalone) broker, and each user
> (currently) has two temporary queues. There is also a central "normal"
> queue.
>
> We are getting garbage collection total freezes after a while with this
> config. We are just using a tcp: transport, and to begin with we were
> not using persistence. This is using activemq 4.0 (ish - a little before
> the release).
>
> We then enabled persistence (derby + journal) - and we lasted a little
> longer, but the same issue arose.
>
> I then wrote a little test harness that simply created 50 connections,
> with two temp queues per session - and the heap grew by 10 Mb (non
> gc-able). That works out at about 200k per connection; so, for 1000
> connections, that's 200 Mb on the heap before we even send any messages
> ! Is there anyway to decrease this amount-per-connection ? Is there some
> kind of  configureable buffer per connection somewhere ?
>
> Our application server IS processing messages as the stress test
> progresses, and JConsole is showing that the main queue rarely has more
> than 30 messages, and typically only has 5 or six sitting there. Is
> there any way to see how BIG these messages are ? I don't think they'll
> be very big (50-60k is my guess), but I'd like to know.
>
> I thought the introduction of the persistence would reduce the memory
> usage, but it seems not.
>
> I've just downloaded 4.0.2, and tried my how-much-space-per-connection
> test - and it remains at 200k ish. I'll give the stress test using 4.02
> a try in a bit - but any pointers would be helpful; I've run Jprofiler
> against the broker, but nothing obvious is showing up.
>
> Cheers,
>
> Charles.
>


-- 

James
-------
http://radio.weblogs.com/0112098/