You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by VJ22 <Vi...@iflexsolutions.com> on 2007/10/11 19:27:00 UTC
Spring JMS Scalability - Newbie questions
Hi All,
I have just implemented a messaging system using Spring JMS with Active MQ
4.1.1...I do have some questions...Would be appreciable if someone could
answer them
We are looking at processing close to 100000 transactions in a
minute...Heres my settings
-> 1 Message Listener Container with 300 concurrent consumers of a
particular class configured via a MessageListenerAdapter
-> Connection Factory is "org.apache.activemq.pool.PooledConnectionFact ory"
My question is whether the connection factory usage is OK? I have read at
many places that the connection factory usage should be ideally via JCA due
to better resource management. Also, the PooledConnectionFactory should be
ideally used for producing and not consuming. I do not want to use Jencks as
we will be migrating to Spring 2.1 which has got the requisite JCA Features.
Also is there any other alternative ?
Secondly, I presume when I mention 300 consumers, 300 parallel threads are
automatically spawned....is there a way to mention if each thread should
automatically prefetch a set of messages and process them ? Also is 300
consumers a lot ?
Any help would be highly appreciated
Thanks...Vijay
--
View this message in context: http://www.nabble.com/Spring-JMS-Scalability---Newbie-questions-tf4608817s2354.html#a13160710
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Spring JMS Scalability - Newbie questions
Posted by VJ22 <Vi...@iflexsolutions.com>.
Hello James,
Thanks a ton for the answers...Much appreciated considering your
workload.....Just some final clarifications if you do not mind answering
them....
- By setting the value of the concurrent consumers I assume that many
consumers are ready for message consumption.....By setting the value of
cache to CACHE_CONSUMER does it imply that consumers are re-used across
listener threads or once a consumer has been created for a listener thread
it is re-used for message consumption for every subsequent messages passed
to that listener thread ? I use the Default Message Listener Container along
with native ActiveMQ settings using the amq namespace...This has a default
cache of CACHE CONSUMER so I presume it is OK for now
- I do use MessageListener Adapter as my listeners with the default response
destination set for response messages....It has been working fine for
now....I also use JMSTransactionManager within the Container definition
- I presume that when you meant re-use the same JMS Session across JMS
Message Listener Container and JMS Template (which the container would be
using for response messages), a setting of CACHE_CONSUMER should be enough
- Yes, the message listener container is good but the ActiveMQ Java Doc
clearly states that PooledConnectionFactory should be used only for
producers and not consumers so was just clarifying
- Finally, do you think using a ServerSessionMessageListenerContainer would
be better than a DefaultMessageListenerContainer ?
Thanks a ton again for the answers
Regards...VJ
James.Strachan wrote:
>
> On 11/10/2007, VJ22 <Vi...@iflexsolutions.com> wrote:
>>
>> Hi All,
>>
>> I have just implemented a messaging system using Spring JMS with Active
>> MQ
>> 4.1.1...I do have some questions...Would be appreciable if someone could
>> answer them
>>
>> We are looking at processing close to 100000 transactions in a
>> minute...Heres my settings
>>
>> -> 1 Message Listener Container with 300 concurrent consumers of a
>> particular class configured via a MessageListenerAdapter
>
> If you are using the MessageListenerContainer and want high
> performance you should ideally set the cache level to "CONSUMER" -
> then consumers are reused (rather than being created & destroyed each
> time along with the session (and connection too if you don't use
> CONNECTION caching or the PooledConnectionFactory).
>
> Note though that there's currently a bug in Spring if you're expecting
> to use JMS transactions to consume then send in the same session...
> http://opensource.atlassian.com/projects/spring/browse/SPR-3890
>
> Also if you want to use a single JMS session (to avoid XA) make sure
> you reuse the same JMS session on both the JmsTemplate and
> MessageListenerContainer.
>
>
>> -> Connection Factory is "org.apache.activemq.pool.PooledConnectionFact
>> ory"
>>
>> My question is whether the connection factory usage is OK? I have read at
>> many places that the connection factory usage should be ideally via JCA
>> due
>> to better resource management. Also, the PooledConnectionFactory should
>> be
>> ideally used for producing and not consuming. I do not want to use Jencks
>> as
>> we will be migrating to Spring 2.1 which has got the requisite JCA
>> Features.
>> Also is there any other alternative ?
>
> The Spring MessageListenerContainer stuff is fine - provided you set
> it up and configure it correctly. FWIW we use it as the JMS consuming
> component in Camel to provide EIP integration...
>
> http://activemq.apache.org/enterprise-integration-patterns.html
>
>
>> Secondly, I presume when I mention 300 consumers, 300 parallel threads
>> are
>> automatically spawned....is there a way to mention if each thread should
>> automatically prefetch a set of messages and process them ?
>
> Yes - ActiveMQ client does that automatically.
> http://activemq.apache.org/what-is-the-prefetch-limit-for.html
>
>> Also is 300
>> consumers a lot ?
>
> It depends on your platform and what the consumer does; i'd experiment
> with the number and see which one performs best. Sometimes on some
> platforms (particularly solaris) the thread context switching often
> outweighs the extra parallelism. Sometimes if your consumer is just
> doing database stuff, your database often dictates how parallel you
> should be for optimum performance.
>
> If you have a high number of consumers; with a high prefetch - you may
> use up too much RAM BTW (and sometimes starve the queue) so you might
> want to experiment lowering the prefetch or consumer count if RAM is
> tight.
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source SOA
> http://open.iona.com
>
>
--
View this message in context: http://www.nabble.com/Spring-JMS-Scalability---Newbie-questions-tf4608817s2354.html#a13187200
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Spring JMS Scalability - Newbie questions
Posted by James Strachan <ja...@gmail.com>.
On 11/10/2007, VJ22 <Vi...@iflexsolutions.com> wrote:
>
> Hi All,
>
> I have just implemented a messaging system using Spring JMS with Active MQ
> 4.1.1...I do have some questions...Would be appreciable if someone could
> answer them
>
> We are looking at processing close to 100000 transactions in a
> minute...Heres my settings
>
> -> 1 Message Listener Container with 300 concurrent consumers of a
> particular class configured via a MessageListenerAdapter
If you are using the MessageListenerContainer and want high
performance you should ideally set the cache level to "CONSUMER" -
then consumers are reused (rather than being created & destroyed each
time along with the session (and connection too if you don't use
CONNECTION caching or the PooledConnectionFactory).
Note though that there's currently a bug in Spring if you're expecting
to use JMS transactions to consume then send in the same session...
http://opensource.atlassian.com/projects/spring/browse/SPR-3890
Also if you want to use a single JMS session (to avoid XA) make sure
you reuse the same JMS session on both the JmsTemplate and
MessageListenerContainer.
> -> Connection Factory is "org.apache.activemq.pool.PooledConnectionFact ory"
>
> My question is whether the connection factory usage is OK? I have read at
> many places that the connection factory usage should be ideally via JCA due
> to better resource management. Also, the PooledConnectionFactory should be
> ideally used for producing and not consuming. I do not want to use Jencks as
> we will be migrating to Spring 2.1 which has got the requisite JCA Features.
> Also is there any other alternative ?
The Spring MessageListenerContainer stuff is fine - provided you set
it up and configure it correctly. FWIW we use it as the JMS consuming
component in Camel to provide EIP integration...
http://activemq.apache.org/enterprise-integration-patterns.html
> Secondly, I presume when I mention 300 consumers, 300 parallel threads are
> automatically spawned....is there a way to mention if each thread should
> automatically prefetch a set of messages and process them ?
Yes - ActiveMQ client does that automatically.
http://activemq.apache.org/what-is-the-prefetch-limit-for.html
> Also is 300
> consumers a lot ?
It depends on your platform and what the consumer does; i'd experiment
with the number and see which one performs best. Sometimes on some
platforms (particularly solaris) the thread context switching often
outweighs the extra parallelism. Sometimes if your consumer is just
doing database stuff, your database often dictates how parallel you
should be for optimum performance.
If you have a high number of consumers; with a high prefetch - you may
use up too much RAM BTW (and sometimes starve the queue) so you might
want to experiment lowering the prefetch or consumer count if RAM is
tight.
--
James
-------
http://macstrac.blogspot.com/
Open Source SOA
http://open.iona.com