You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by KumarR <ry...@gmail.com> on 2009/05/14 12:33:04 UTC
Publishing to Topic is very slow in higher loads ActiveMQ5.2.
Hi All ,
I am having problem with publishing messages to Topic. It's taking too much
time to publish a message. Below are the details.
Test Case:
The Client is publishing messages to RequestQueue. and waiting responses on
ResponseTopic using a co-relation id.
Server program consuming messages from RequestQueue using Spring Message
POJOs. These POJOs after consuming and processing the messages,
forms the response message and publishes to the ResponseTopic using
co-relationid.
The client using blocked receive(TIMEOUT) receive responses from
ResponseTopic using Message selector with co-relation id.
Observations: at higher loads, around 3000 conc users, the publishing
response messages back to Response Topic becoming slow;
taking around 5 secs to 10 secs. otherwise in 2000 conc users load, it was
taking around 100 ms-120 ms.
Setup Configuration:
We are using ActiveMQ 5.2 on Sun Solaris 10 - SPARC JVM 1.6_16;
Topic has Non-Durable Consumer; we use Non-Transactional, Non-Persistent
Messages
Client program is a WebApplication running on JBoss-JVM instance on server
machine 1
ActiveMQ is a standalone broker jvm running on server machine 2
Server program is another backend application (not web-application) on
another Jboss-JVM instance on server machine2.
Can any one help in solving this problem ?
Thanks
Kumar
-----
Thanks,
Kumar Ravella
--
View this message in context: http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23538009.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Publishing to Topic is very slow in higher loads ActiveMQ5.2.
Posted by Yaswanth <ya...@gmail.com>.
Hi,
I have tried disabling producer flow control. It has shown some improvement.
But still at higher loads, around 3000 conc users, publishing response
messages is taking around 2.5 secs to 4.5 secs.
Any clue ?
Thanks,
Kumar
rajdavies wrote:
>
> Looks like you've heat a flow contorl problem - try disabling it - see
> http://activemq.apache.org/producer-flow-control.html
>
> cheers,
>
> Rob
>
>
> On 14 May 2009, at 11:33, KumarR wrote:
>
>>
>> Hi All ,
>>
>> I am having problem with publishing messages to Topic. It's taking
>> too much
>> time to publish a message. Below are the details.
>>
>> Test Case:
>>
>> The Client is publishing messages to RequestQueue. and waiting
>> responses on
>> ResponseTopic using a co-relation id.
>> Server program consuming messages from RequestQueue using Spring
>> Message
>> POJOs. These POJOs after consuming and processing the messages,
>> forms the response message and publishes to the ResponseTopic using
>> co-relationid.
>> The client using blocked receive(TIMEOUT) receive responses from
>> ResponseTopic using Message selector with co-relation id.
>>
>>
>> Observations: at higher loads, around 3000 conc users, the publishing
>> response messages back to Response Topic becoming slow;
>> taking around 5 secs to 10 secs. otherwise in 2000 conc users load,
>> it was
>> taking around 100 ms-120 ms.
>>
>> Setup Configuration:
>> We are using ActiveMQ 5.2 on Sun Solaris 10 - SPARC JVM 1.6_16;
>> Topic has Non-Durable Consumer; we use Non-Transactional, Non-
>> Persistent
>> Messages
>>
>> Client program is a WebApplication running on JBoss-JVM instance on
>> server
>> machine 1
>> ActiveMQ is a standalone broker jvm running on server machine 2
>> Server program is another backend application (not web-application) on
>> another Jboss-JVM instance on server machine2.
>>
>>
>> Can any one help in solving this problem ?
>>
>> Thanks
>> Kumar
>>
>> -----
>> Thanks,
>> Kumar Ravella
>> --
>> View this message in context:
>> http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23538009.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
--
View this message in context: http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23555106.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Publishing to Topic is very slow in higher loads ActiveMQ5.2.
Posted by Rob Davies <ra...@gmail.com>.
Looks like you've heat a flow contorl problem - try disabling it - see http://activemq.apache.org/producer-flow-control.html
cheers,
Rob
On 14 May 2009, at 11:33, KumarR wrote:
>
> Hi All ,
>
> I am having problem with publishing messages to Topic. It's taking
> too much
> time to publish a message. Below are the details.
>
> Test Case:
>
> The Client is publishing messages to RequestQueue. and waiting
> responses on
> ResponseTopic using a co-relation id.
> Server program consuming messages from RequestQueue using Spring
> Message
> POJOs. These POJOs after consuming and processing the messages,
> forms the response message and publishes to the ResponseTopic using
> co-relationid.
> The client using blocked receive(TIMEOUT) receive responses from
> ResponseTopic using Message selector with co-relation id.
>
>
> Observations: at higher loads, around 3000 conc users, the publishing
> response messages back to Response Topic becoming slow;
> taking around 5 secs to 10 secs. otherwise in 2000 conc users load,
> it was
> taking around 100 ms-120 ms.
>
> Setup Configuration:
> We are using ActiveMQ 5.2 on Sun Solaris 10 - SPARC JVM 1.6_16;
> Topic has Non-Durable Consumer; we use Non-Transactional, Non-
> Persistent
> Messages
>
> Client program is a WebApplication running on JBoss-JVM instance on
> server
> machine 1
> ActiveMQ is a standalone broker jvm running on server machine 2
> Server program is another backend application (not web-application) on
> another Jboss-JVM instance on server machine2.
>
>
> Can any one help in solving this problem ?
>
> Thanks
> Kumar
>
> -----
> Thanks,
> Kumar Ravella
> --
> View this message in context: http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23538009.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Publishing to Topic is very slow in higher loads ActiveMQ5.2.
Posted by KumarR <ry...@gmail.com>.
Hi,
In continuation with previous email, more information regarding the issue:
we have analysed the logs and observed that there is some pattern of varying
publishing times, for every few seconds.
example:
At the full load,
the event published at 14:49:24,352 took TotalTime : 55 ms
and then gradually increased..and
the event published at 14:49:26,445 took TotalTime : 4475 ms
Hence within this approx, 2 minutes duration, there were 190 response
messages published to
ResponseTopic; starting at 55 ms and increasing upto 4475 ms..
and after that,
the next event published at
14:49:26,498 took only 62 ms;
from thereon, again the same pattern is observed...
Is there any JMS resources contention happening in this duration between the
publisher and the ActiveMQ standalone broker ?
as known, send API() uses tcp connection from the application instance to
the standalone broker instance.
is there any TCP tuning to be done ?
or
is there any Active MQ tuning to be done ?
Both Active MQ JVM heap and the application JVM heap are quite stable. there
are minor collections
happening every few seconds.
Note: JMeter test plan is being run for 4000 conc users with think time of 5
seconds; and jmeter throughput arround 450 req/sec
please let us know if anybody has clues ?
regards
Y wrote:
>
>
> Hi,
>
> Thanks for the reply .
>
> When observed in Broker Statistics in Jconsole it has enough memory.
>
> Here is my ActiveMQ.xml file ..
> http://www.nabble.com/file/p23539624/activemq.xml activemq.xml
>
> Here is the Memory Snapshot of ActiveMQ Old Generation.
> http://www.nabble.com/file/p23539624/OldGen.JPG OldGen.JPG
>
>
>
>
> Gary Tully wrote:
>>
>> can you give more memory resources to the broker? I imagine there is
>> some spooling to disk under high load. It may make sense to post your
>> activemq.xml.
>>
>> 2009/5/14 KumarR <ry...@gmail.com>:
>>>
>>> Hi All ,
>>>
>>> I am having problem with publishing messages to Topic. It's taking too
>>> much
>>> time to publish a message. Below are the details.
>>>
>>> Test Case:
>>>
>>> The Client is publishing messages to RequestQueue. and waiting
>>> responses on
>>> ResponseTopic using a co-relation id.
>>> Server program consuming messages from RequestQueue using Spring Message
>>> POJOs. These POJOs after consuming and processing the messages,
>>> forms the response message and publishes to the ResponseTopic using
>>> co-relationid.
>>> The client using blocked receive(TIMEOUT) receive responses from
>>> ResponseTopic using Message selector with co-relation id.
>>>
>>>
>>> Observations: at higher loads, around 3000 conc users, the publishing
>>> response messages back to Response Topic becoming slow;
>>> taking around 5 secs to 10 secs. otherwise in 2000 conc users load, it
>>> was
>>> taking around 100 ms-120 ms.
>>>
>>> Setup Configuration:
>>> We are using ActiveMQ 5.2 on Sun Solaris 10 - SPARC JVM 1.6_16;
>>> Topic has Non-Durable Consumer; we use Non-Transactional,
>>> Non-Persistent
>>> Messages
>>>
>>> Client program is a WebApplication running on JBoss-JVM instance on
>>> server
>>> machine 1
>>> ActiveMQ is a standalone broker jvm running on server machine 2
>>> Server program is another backend application (not web-application) on
>>> another Jboss-JVM instance on server machine2.
>>>
>>>
>>> Can any one help in solving this problem ?
>>>
>>> Thanks
>>> Kumar
>>>
>>> -----
>>> Thanks,
>>> Kumar Ravella
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23538009.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> http://blog.garytully.com
>>
>> Open Source SOA
>> http://FUSESource.com
>>
>>
>
>
-----
Thanks,
Kumar Ravella
--
View this message in context: http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23596007.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Publishing to Topic is very slow in higher loads ActiveMQ5.2.
Posted by Yaswanth <ya...@gmail.com>.
Hi,
Thanks for the reply .
When observed in Broker Statistics in Jconsole it has enough memory.
Here is my ActiveMQ.xml file ..
http://www.nabble.com/file/p23539624/activemq.xml activemq.xml
Here is the Memory Snapshot of ActiveMQ Old Generation.
http://www.nabble.com/file/p23539624/OldGen.JPG OldGen.JPG
Gary Tully wrote:
>
> can you give more memory resources to the broker? I imagine there is
> some spooling to disk under high load. It may make sense to post your
> activemq.xml.
>
> 2009/5/14 KumarR <ry...@gmail.com>:
>>
>> Hi All ,
>>
>> I am having problem with publishing messages to Topic. It's taking too
>> much
>> time to publish a message. Below are the details.
>>
>> Test Case:
>>
>> The Client is publishing messages to RequestQueue. and waiting responses
>> on
>> ResponseTopic using a co-relation id.
>> Server program consuming messages from RequestQueue using Spring Message
>> POJOs. These POJOs after consuming and processing the messages,
>> forms the response message and publishes to the ResponseTopic using
>> co-relationid.
>> The client using blocked receive(TIMEOUT) receive responses from
>> ResponseTopic using Message selector with co-relation id.
>>
>>
>> Observations: at higher loads, around 3000 conc users, the publishing
>> response messages back to Response Topic becoming slow;
>> taking around 5 secs to 10 secs. otherwise in 2000 conc users load, it
>> was
>> taking around 100 ms-120 ms.
>>
>> Setup Configuration:
>> We are using ActiveMQ 5.2 on Sun Solaris 10 - SPARC JVM 1.6_16;
>> Topic has Non-Durable Consumer; we use Non-Transactional, Non-Persistent
>> Messages
>>
>> Client program is a WebApplication running on JBoss-JVM instance on
>> server
>> machine 1
>> ActiveMQ is a standalone broker jvm running on server machine 2
>> Server program is another backend application (not web-application) on
>> another Jboss-JVM instance on server machine2.
>>
>>
>> Can any one help in solving this problem ?
>>
>> Thanks
>> Kumar
>>
>> -----
>> Thanks,
>> Kumar Ravella
>> --
>> View this message in context:
>> http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23538009.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> http://blog.garytully.com
>
> Open Source SOA
> http://FUSESource.com
>
>
--
View this message in context: http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23539624.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Publishing to Topic is very slow in higher loads ActiveMQ5.2.
Posted by Gary Tully <ga...@gmail.com>.
can you give more memory resources to the broker? I imagine there is
some spooling to disk under high load. It may make sense to post your
activemq.xml.
2009/5/14 KumarR <ry...@gmail.com>:
>
> Hi All ,
>
> I am having problem with publishing messages to Topic. It's taking too much
> time to publish a message. Below are the details.
>
> Test Case:
>
> The Client is publishing messages to RequestQueue. and waiting responses on
> ResponseTopic using a co-relation id.
> Server program consuming messages from RequestQueue using Spring Message
> POJOs. These POJOs after consuming and processing the messages,
> forms the response message and publishes to the ResponseTopic using
> co-relationid.
> The client using blocked receive(TIMEOUT) receive responses from
> ResponseTopic using Message selector with co-relation id.
>
>
> Observations: at higher loads, around 3000 conc users, the publishing
> response messages back to Response Topic becoming slow;
> taking around 5 secs to 10 secs. otherwise in 2000 conc users load, it was
> taking around 100 ms-120 ms.
>
> Setup Configuration:
> We are using ActiveMQ 5.2 on Sun Solaris 10 - SPARC JVM 1.6_16;
> Topic has Non-Durable Consumer; we use Non-Transactional, Non-Persistent
> Messages
>
> Client program is a WebApplication running on JBoss-JVM instance on server
> machine 1
> ActiveMQ is a standalone broker jvm running on server machine 2
> Server program is another backend application (not web-application) on
> another Jboss-JVM instance on server machine2.
>
>
> Can any one help in solving this problem ?
>
> Thanks
> Kumar
>
> -----
> Thanks,
> Kumar Ravella
> --
> View this message in context: http://www.nabble.com/Publishing-to-Topic-is-very-slow-in-higher-loads-ActiveMQ5.2.-tp23538009p23538009.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source SOA
http://FUSESource.com