problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

Hi all,

I've been testing ActiveMQ under relatively high load and number of
producers and consumers (50-100 on each side) and I've attached the
code that I have been using for test purposes (it's not really clean
code since it's used for testing purposes only). What happens is when
I launch about 50-60 consumers/producers (they run on a different
boxes) and I start to stop consumer processes, producers tend to block
for a shorter period, if I repeat this process for a while I get to
state where's a whole queue blocked for 1-2h or longer (sometimes for
good).
Is this general performance problem with ActiveMQ or am I doing something wrong?

Thanks,
Igor

Re: problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

Thanks Hiram,

I'll definitely check it out!

Thanks,
Igor

On 7/3/06, Hiram Chirino <hi...@hiramchirino.com> wrote:
> I Igor,
>
> I just recently fixed a issue where a consumer could be created and not
> properly cleaned up if an async exception occured.  See
> http://issues.apache.org/activemq/browse/AMQ-724.  I have a feeling that
> this could have been the cause of the hung consumers.
>
> Could you retest against a subsequent 4.1-SNAPSHOT or 4.0.2-SNAPSHOT?
>
> Thanks!
>
> On 7/3/06, Igor Bogicevic <ca...@gmail.com> wrote:
> >
> > Hi James,
> >
> > well, it doesn't depend on the prefetch value (I have tried with 1000
> > and 2000 prefetch messages, and without prefetching at all), and what
> > happens is this:
> > 1) create about 60 producers and 60 consumers (I have attached the
> > small and ugly test I used - basically I've forked about 60 processes
> > on each side, which means they had separate sessions and connections)
> > 2) if you cut out the shutdown hook in this test, which handles
> > cleanup of connections and sessions, and you kill large amount of
> > producers/consumers this queue will be blocked in the terms that
> > neither producers or consumers will be able to either send or receive
> > messages on this queue - sometimes this happens for a period of a
> > 20-30s and very often for good (>10h, that's the longest period i've
> > left it before i've restarted broker).
> > 3) in case there is shutdown hook with properly closing sessions and
> > connections, this is very hard to repeat, and in case of 4.0.0 i've
> > managed to block the queue with repetitve starting/killing large
> > number of producers and consumers, but i've never managed to reproduce
> > this with 4.0.1
> > 4) i could see a large number of hanging connections (jmx console
> > showed much more connections than there actually were)
> > 4) after restarting broker all works like a charm
> >
> > Thanks,
> > Igor
> >
> > On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> > > By 'blocking the queue' do you just mean messages were not dispatched
> > > to consumers? If so how many messages were on the queue, how many
> > > connections & consumers did the broker think were open & what were the
> > > prefetch values? If you are having trouble of dead connections not
> > > being closed down agressively enough then your issue could just be an
> > > effect of messages being placed into prefetch buffers for clients that
> > > have not been detected as failed yet? Do you have any kind of JUnit
> > > test case that can replicate your issue?
> > >
> > > On 7/3/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > > I am using activemq 4.0.1 (I've did tests on the both 4.0.0 and 4.0.1)
> > > > and I have attached activemq.xml (basically, almost out of the box
> > > > configuration). I am using persistent queues, and I have tried with
> > > > prefetching and transactions, but that didn't really change the
> > > > behavior.
> > > >
> > > > Thanks,
> > > > Igor
> > > >
> > > > On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> > > > > On 6/30/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > > > > I also forgot to answer on this thread... this was not the problem
> > > > > > related to the lack of the memory, it's related to hanging
> > > > > > connections, i've set up the broker with the large amount of
> > memory
> > > > > > (>2GB) in both broker configuration and via jconsole and problem
> > was
> > > > > > defintely in the connections which were not cleaned up - for
> > unknown
> > > > > > reason to me. it was easy to block queue with large number of
> > > > > > un-properly closed sessions and relatively hard with properly
> > closed
> > > > > > connections in a shutdown hook.
> > > > >
> > > > > Using what version and what quality of service? e.g. are we talking
> > > > > 4.0.1 and non-persistent queues? What was your XML config file?
> > > > >
> > > > > --
> > > > >
> > > > > James
> > > > > -------
> > > > > http://radio.weblogs.com/0112098/
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > >
> > > James
> > > -------
> > > http://radio.weblogs.com/0112098/
> > >
> >
> >
> >
>
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
>

Re: problems with ActiveMQ with large number of producers/consumers

[Hiram Chirino <hi...@hiramchirino.com>]

I Igor,

I just recently fixed a issue where a consumer could be created and not
properly cleaned up if an async exception occured.  See
http://issues.apache.org/activemq/browse/AMQ-724.  I have a feeling that
this could have been the cause of the hung consumers.

Could you retest against a subsequent 4.1-SNAPSHOT or 4.0.2-SNAPSHOT?

Thanks!

On 7/3/06, Igor Bogicevic <ca...@gmail.com> wrote:
>
> Hi James,
>
> well, it doesn't depend on the prefetch value (I have tried with 1000
> and 2000 prefetch messages, and without prefetching at all), and what
> happens is this:
> 1) create about 60 producers and 60 consumers (I have attached the
> small and ugly test I used - basically I've forked about 60 processes
> on each side, which means they had separate sessions and connections)
> 2) if you cut out the shutdown hook in this test, which handles
> cleanup of connections and sessions, and you kill large amount of
> producers/consumers this queue will be blocked in the terms that
> neither producers or consumers will be able to either send or receive
> messages on this queue - sometimes this happens for a period of a
> 20-30s and very often for good (>10h, that's the longest period i've
> left it before i've restarted broker).
> 3) in case there is shutdown hook with properly closing sessions and
> connections, this is very hard to repeat, and in case of 4.0.0 i've
> managed to block the queue with repetitve starting/killing large
> number of producers and consumers, but i've never managed to reproduce
> this with 4.0.1
> 4) i could see a large number of hanging connections (jmx console
> showed much more connections than there actually were)
> 4) after restarting broker all works like a charm
>
> Thanks,
> Igor
>
> On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> > By 'blocking the queue' do you just mean messages were not dispatched
> > to consumers? If so how many messages were on the queue, how many
> > connections & consumers did the broker think were open & what were the
> > prefetch values? If you are having trouble of dead connections not
> > being closed down agressively enough then your issue could just be an
> > effect of messages being placed into prefetch buffers for clients that
> > have not been detected as failed yet? Do you have any kind of JUnit
> > test case that can replicate your issue?
> >
> > On 7/3/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > I am using activemq 4.0.1 (I've did tests on the both 4.0.0 and 4.0.1)
> > > and I have attached activemq.xml (basically, almost out of the box
> > > configuration). I am using persistent queues, and I have tried with
> > > prefetching and transactions, but that didn't really change the
> > > behavior.
> > >
> > > Thanks,
> > > Igor
> > >
> > > On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> > > > On 6/30/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > > > I also forgot to answer on this thread... this was not the problem
> > > > > related to the lack of the memory, it's related to hanging
> > > > > connections, i've set up the broker with the large amount of
> memory
> > > > > (>2GB) in both broker configuration and via jconsole and problem
> was
> > > > > defintely in the connections which were not cleaned up - for
> unknown
> > > > > reason to me. it was easy to block queue with large number of
> > > > > un-properly closed sessions and relatively hard with properly
> closed
> > > > > connections in a shutdown hook.
> > > >
> > > > Using what version and what quality of service? e.g. are we talking
> > > > 4.0.1 and non-persistent queues? What was your XML config file?
> > > >
> > > > --
> > > >
> > > > James
> > > > -------
> > > > http://radio.weblogs.com/0112098/
> > > >
> > >
> > >
> > >
> >
> >
> > --
> >
> > James
> > -------
> > http://radio.weblogs.com/0112098/
> >
>
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Re: problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

Hi James,

well, it doesn't depend on the prefetch value (I have tried with 1000
and 2000 prefetch messages, and without prefetching at all), and what
happens is this:
1) create about 60 producers and 60 consumers (I have attached the
small and ugly test I used - basically I've forked about 60 processes
on each side, which means they had separate sessions and connections)
2) if you cut out the shutdown hook in this test, which handles
cleanup of connections and sessions, and you kill large amount of
producers/consumers this queue will be blocked in the terms that
neither producers or consumers will be able to either send or receive
messages on this queue - sometimes this happens for a period of a
20-30s and very often for good (>10h, that's the longest period i've
left it before i've restarted broker).
3) in case there is shutdown hook with properly closing sessions and
connections, this is very hard to repeat, and in case of 4.0.0 i've
managed to block the queue with repetitve starting/killing large
number of producers and consumers, but i've never managed to reproduce
this with 4.0.1
4) i could see a large number of hanging connections (jmx console
showed much more connections than there actually were)
4) after restarting broker all works like a charm

Thanks,
Igor

On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> By 'blocking the queue' do you just mean messages were not dispatched
> to consumers? If so how many messages were on the queue, how many
> connections & consumers did the broker think were open & what were the
> prefetch values? If you are having trouble of dead connections not
> being closed down agressively enough then your issue could just be an
> effect of messages being placed into prefetch buffers for clients that
> have not been detected as failed yet? Do you have any kind of JUnit
> test case that can replicate your issue?
>
> On 7/3/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > I am using activemq 4.0.1 (I've did tests on the both 4.0.0 and 4.0.1)
> > and I have attached activemq.xml (basically, almost out of the box
> > configuration). I am using persistent queues, and I have tried with
> > prefetching and transactions, but that didn't really change the
> > behavior.
> >
> > Thanks,
> > Igor
> >
> > On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> > > On 6/30/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > > I also forgot to answer on this thread... this was not the problem
> > > > related to the lack of the memory, it's related to hanging
> > > > connections, i've set up the broker with the large amount of memory
> > > > (>2GB) in both broker configuration and via jconsole and problem was
> > > > defintely in the connections which were not cleaned up - for unknown
> > > > reason to me. it was easy to block queue with large number of
> > > > un-properly closed sessions and relatively hard with properly closed
> > > > connections in a shutdown hook.
> > >
> > > Using what version and what quality of service? e.g. are we talking
> > > 4.0.1 and non-persistent queues? What was your XML config file?
> > >
> > > --
> > >
> > > James
> > > -------
> > > http://radio.weblogs.com/0112098/
> > >
> >
> >
> >
>
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>

Re: problems with ActiveMQ with large number of producers/consumers

[James Strachan <ja...@gmail.com>]

By 'blocking the queue' do you just mean messages were not dispatched
to consumers? If so how many messages were on the queue, how many
connections & consumers did the broker think were open & what were the
prefetch values? If you are having trouble of dead connections not
being closed down agressively enough then your issue could just be an
effect of messages being placed into prefetch buffers for clients that
have not been detected as failed yet? Do you have any kind of JUnit
test case that can replicate your issue?

On 7/3/06, Igor Bogicevic <ca...@gmail.com> wrote:
> I am using activemq 4.0.1 (I've did tests on the both 4.0.0 and 4.0.1)
> and I have attached activemq.xml (basically, almost out of the box
> configuration). I am using persistent queues, and I have tried with
> prefetching and transactions, but that didn't really change the
> behavior.
>
> Thanks,
> Igor
>
> On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> > On 6/30/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > I also forgot to answer on this thread... this was not the problem
> > > related to the lack of the memory, it's related to hanging
> > > connections, i've set up the broker with the large amount of memory
> > > (>2GB) in both broker configuration and via jconsole and problem was
> > > defintely in the connections which were not cleaned up - for unknown
> > > reason to me. it was easy to block queue with large number of
> > > un-properly closed sessions and relatively hard with properly closed
> > > connections in a shutdown hook.
> >
> > Using what version and what quality of service? e.g. are we talking
> > 4.0.1 and non-persistent queues? What was your XML config file?
> >
> > --
> >
> > James
> > -------
> > http://radio.weblogs.com/0112098/
> >
>
>
>


-- 

James
-------
http://radio.weblogs.com/0112098/

Re: problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

I am using activemq 4.0.1 (I've did tests on the both 4.0.0 and 4.0.1)
and I have attached activemq.xml (basically, almost out of the box
configuration). I am using persistent queues, and I have tried with
prefetching and transactions, but that didn't really change the
behavior.

Thanks,
Igor

On 7/3/06, James Strachan <ja...@gmail.com> wrote:
> On 6/30/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > I also forgot to answer on this thread... this was not the problem
> > related to the lack of the memory, it's related to hanging
> > connections, i've set up the broker with the large amount of memory
> > (>2GB) in both broker configuration and via jconsole and problem was
> > defintely in the connections which were not cleaned up - for unknown
> > reason to me. it was easy to block queue with large number of
> > un-properly closed sessions and relatively hard with properly closed
> > connections in a shutdown hook.
>
> Using what version and what quality of service? e.g. are we talking
> 4.0.1 and non-persistent queues? What was your XML config file?
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>

Re: problems with ActiveMQ with large number of producers/consumers

[James Strachan <ja...@gmail.com>]

On 6/30/06, Igor Bogicevic <ca...@gmail.com> wrote:
> I also forgot to answer on this thread... this was not the problem
> related to the lack of the memory, it's related to hanging
> connections, i've set up the broker with the large amount of memory
> (>2GB) in both broker configuration and via jconsole and problem was
> defintely in the connections which were not cleaned up - for unknown
> reason to me. it was easy to block queue with large number of
> un-properly closed sessions and relatively hard with properly closed
> connections in a shutdown hook.

Using what version and what quality of service? e.g. are we talking
4.0.1 and non-persistent queues? What was your XML config file?

-- 

James
-------
http://radio.weblogs.com/0112098/

Re: problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

I also forgot to answer on this thread... this was not the problem
related to the lack of the memory, it's related to hanging
connections, i've set up the broker with the large amount of memory
(>2GB) in both broker configuration and via jconsole and problem was
defintely in the connections which were not cleaned up - for unknown
reason to me. it was easy to block queue with large number of
un-properly closed sessions and relatively hard with properly closed
connections in a shutdown hook.

thanks,
igor

On 6/30/06, skarthik <ka...@necam.com> wrote:
>
> Hi Hiram,
>
> I was wondering what is the relationship between the Broker memory limit set
> in the broker XML and the JVM heap size specified in when launching the
> broker. For example should the heap size be some extra percentage of the
> configured broker memory? I guess the broker memory can nenver be greater
> than the -Xmx value.
>
> thanks,
> karthik
> --
> View this message in context: http://www.nabble.com/problems-with-ActiveMQ-with-large-number-of-producers-consumers-tf1823629.html#a5126960
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>

Re: problems with ActiveMQ with large number of producers/consumers

[skarthik <ka...@necam.com>]

Hi Hiram,

I was wondering what is the relationship between the Broker memory limit set
in the broker XML and the JVM heap size specified in when launching the
broker. For example should the heap size be some extra percentage of the
configured broker memory? I guess the broker memory can nenver be greater
than the -Xmx value.

thanks,
karthik
-- 
View this message in context: http://www.nabble.com/problems-with-ActiveMQ-with-large-number-of-producers-consumers-tf1823629.html#a5126960
Sent from the ActiveMQ - User forum at Nabble.com.

Re: problems with ActiveMQ with large number of producers/consumers

[Hiram Chirino <hi...@hiramchirino.com>]

I would attach a JMX console to the broker and look at the broker's
memory percentage used attribute.  If it's 100% then that's the reason
that the broker is blocked.  This memory used has nothing to do with
the JVM memory configuration BTW.  It's a configurable setting in the
broker XML and can be changed on the fly using the jmx console by
setting the memory limit attribute on the broker JMX object.

On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> I actually have one more detail that might be interesting... During
> some experiments I've when the queue is blocked I received this output
> from bstat:
>
>
> ACTIVEMQ_HOME: /data1/activemq/incubator-activemq-4.0.1
> ERROR: java.lang.RuntimeException: Failed to execute query task.
> Reason: java.io.IOException: Failed to retrieve RMIServer stub:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectExc
> eption: Connection refused to host: localhost; nested exception is:
>         java.net.ConnectException: Connection refused]
> ERROR: java.lang.Exception: java.io.IOException: Failed to retrieve
> RMIServer stub: javax.naming.ServiceUnavailableException [Root
> exception is java.rmi.ConnectException: Connection refused to host:
> localhost
> ; nested exception is:
>         java.net.ConnectException: Connection refused]
>
> also, tests are performed on 3 different boxes on same switch
> (full-duplex 1Gbit) and producers, consumers and broker are on
> separate boxes with plenty of memory (-Xmx8192M -Xms1024M for broker),
> with no visible trail of memory lack or I/O blocking on the each of
> the boxes).
>
> Thanks,
> Igor
>
> On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > Sorry, I forgot to specify the details, it's incubator-activemq-4.0
> > and i am using Sun JDK 1.5_06
> >
> > Thanks,
> > Igor
> >
> > On 6/21/06, James Strachan <ja...@gmail.com> wrote:
> > > Which version of ActiveMQ are you using?
> > >
> > > On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > > Hi all,
> > > >
> > > > I've been testing ActiveMQ under relatively high load and number of
> > > > producers and consumers (50-100 on each side) and I've attached the
> > > > code that I have been using for test purposes (it's not really clean
> > > > code since it's used for testing purposes only). What happens is when
> > > > I launch about 50-60 consumers/producers (they run on a different
> > > > boxes) and I start to stop consumer processes, producers tend to block
> > > > for a shorter period, if I repeat this process for a while I get to
> > > > state where's a whole queue blocked for 1-2h or longer (sometimes for
> > > > good).
> > > > Is this general performance problem with ActiveMQ or am I doing something wrong?
> > > >
> > > > Thanks,
> > > > Igor
> > > >
> > > >
> > > >
> > >
> > >
> > > --
> > >
> > > James
> > > -------
> > > http://radio.weblogs.com/0112098/
> > >
> >
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Re: problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

I actually have one more detail that might be interesting... During
some experiments I've when the queue is blocked I received this output
from bstat:


ACTIVEMQ_HOME: /data1/activemq/incubator-activemq-4.0.1
ERROR: java.lang.RuntimeException: Failed to execute query task.
Reason: java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectExc
eption: Connection refused to host: localhost; nested exception is:
        java.net.ConnectException: Connection refused]
ERROR: java.lang.Exception: java.io.IOException: Failed to retrieve
RMIServer stub: javax.naming.ServiceUnavailableException [Root
exception is java.rmi.ConnectException: Connection refused to host:
localhost
; nested exception is:
        java.net.ConnectException: Connection refused]

also, tests are performed on 3 different boxes on same switch
(full-duplex 1Gbit) and producers, consumers and broker are on
separate boxes with plenty of memory (-Xmx8192M -Xms1024M for broker),
with no visible trail of memory lack or I/O blocking on the each of
the boxes).

Thanks,
Igor

On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> Sorry, I forgot to specify the details, it's incubator-activemq-4.0
> and i am using Sun JDK 1.5_06
>
> Thanks,
> Igor
>
> On 6/21/06, James Strachan <ja...@gmail.com> wrote:
> > Which version of ActiveMQ are you using?
> >
> > On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > > Hi all,
> > >
> > > I've been testing ActiveMQ under relatively high load and number of
> > > producers and consumers (50-100 on each side) and I've attached the
> > > code that I have been using for test purposes (it's not really clean
> > > code since it's used for testing purposes only). What happens is when
> > > I launch about 50-60 consumers/producers (they run on a different
> > > boxes) and I start to stop consumer processes, producers tend to block
> > > for a shorter period, if I repeat this process for a while I get to
> > > state where's a whole queue blocked for 1-2h or longer (sometimes for
> > > good).
> > > Is this general performance problem with ActiveMQ or am I doing something wrong?
> > >
> > > Thanks,
> > > Igor
> > >
> > >
> > >
> >
> >
> > --
> >
> > James
> > -------
> > http://radio.weblogs.com/0112098/
> >
>

Re: problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

Sorry, I forgot to specify the details, it's incubator-activemq-4.0
and i am using Sun JDK 1.5_06

Thanks,
Igor

On 6/21/06, James Strachan <ja...@gmail.com> wrote:
> Which version of ActiveMQ are you using?
>
> On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > Hi all,
> >
> > I've been testing ActiveMQ under relatively high load and number of
> > producers and consumers (50-100 on each side) and I've attached the
> > code that I have been using for test purposes (it's not really clean
> > code since it's used for testing purposes only). What happens is when
> > I launch about 50-60 consumers/producers (they run on a different
> > boxes) and I start to stop consumer processes, producers tend to block
> > for a shorter period, if I repeat this process for a while I get to
> > state where's a whole queue blocked for 1-2h or longer (sometimes for
> > good).
> > Is this general performance problem with ActiveMQ or am I doing something wrong?
> >
> > Thanks,
> > Igor
> >
> >
> >
>
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>

Re: problems with ActiveMQ with large number of producers/consumers

[James Strachan <ja...@gmail.com>]

Which version of ActiveMQ are you using?

On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> Hi all,
>
> I've been testing ActiveMQ under relatively high load and number of
> producers and consumers (50-100 on each side) and I've attached the
> code that I have been using for test purposes (it's not really clean
> code since it's used for testing purposes only). What happens is when
> I launch about 50-60 consumers/producers (they run on a different
> boxes) and I start to stop consumer processes, producers tend to block
> for a shorter period, if I repeat this process for a while I get to
> state where's a whole queue blocked for 1-2h or longer (sometimes for
> good).
> Is this general performance problem with ActiveMQ or am I doing something wrong?
>
> Thanks,
> Igor
>
>
>


-- 

James
-------
http://radio.weblogs.com/0112098/

Re: problems with ActiveMQ with large number of producers/consumers

[Igor Bogicevic <ca...@gmail.com>]

hmm... could be but I am definitely not sure what is actually causing
it, what is interesting and I've forgot to mention is that i have
large number of still hanging connections and basically it blocks
producers for some period as rule of thumb. However, I was not able to
reproduce this problem on a small number of producers/consumers. Just
stopping the producers doesn't help and it can be reproduced on
relatively small number of messages being sent to the queue
(1000-2000) and I don't experience this problem if I connect consumers
after some time (about 10k messages), but what really blocks it is
situation when I kill the consumers... I assume that it could be
related to hanging connections, but the problem is that sometimes
broken connections stay present after a very long period of time (at
least with bstat I can see large number of Type = Subscription
elements, that exceeds the connected producers/subscribers if that's
of any relevance).

Thanks,
Igor

On 6/21/06, Hiram Chirino <hi...@hiramchirino.com> wrote:
> When you stop a consumer I wonder if your causing a slow consumer
> situation... see:
> http://activemq.com/site/slow-consumers.html
>
> On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> > Hi all,
> >
> > I've been testing ActiveMQ under relatively high load and number of
> > producers and consumers (50-100 on each side) and I've attached the
> > code that I have been using for test purposes (it's not really clean
> > code since it's used for testing purposes only). What happens is when
> > I launch about 50-60 consumers/producers (they run on a different
> > boxes) and I start to stop consumer processes, producers tend to block
> > for a shorter period, if I repeat this process for a while I get to
> > state where's a whole queue blocked for 1-2h or longer (sometimes for
> > good).
> > Is this general performance problem with ActiveMQ or am I doing something wrong?
> >
> > Thanks,
> > Igor
> >
> >
> >
>
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>

Re: problems with ActiveMQ with large number of producers/consumers

[Hiram Chirino <hi...@hiramchirino.com>]

When you stop a consumer I wonder if your causing a slow consumer
situation... see:
http://activemq.com/site/slow-consumers.html

On 6/21/06, Igor Bogicevic <ca...@gmail.com> wrote:
> Hi all,
>
> I've been testing ActiveMQ under relatively high load and number of
> producers and consumers (50-100 on each side) and I've attached the
> code that I have been using for test purposes (it's not really clean
> code since it's used for testing purposes only). What happens is when
> I launch about 50-60 consumers/producers (they run on a different
> boxes) and I start to stop consumer processes, producers tend to block
> for a shorter period, if I repeat this process for a while I get to
> state where's a whole queue blocked for 1-2h or longer (sometimes for
> good).
> Is this general performance problem with ActiveMQ or am I doing something wrong?
>
> Thanks,
> Igor
>
>
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com