Poor design, poor performance. Better design, better performance?

[jongraf <jo...@gmail.com>]

In an effort to keep all of the configuration within the code, my design for
implementing asynchronous messaging failed our load tests.  Our application
was able to withstand 1400 hits per second before implementing ActiveMQ with
the following model:
* embedded JVM broker
* non-persistent messaging
* consumer took messages from producer with the same app and made httpClient
sends to the remote machine using REST

After tweaking lots of settings we never made it above 800 hits/second.  I
then decided to RTFM and educate myself on Enterprise Integration Patterns.

The new design is much simpler and my hope is that it will increase our
application performance because...
1) The processing of the queue will now be outside of my application's JVM
2) My application will not function as both a producer and a consumer
3) The consumer will not be making httpClient (hence using more threads)

Proposed design:
* client-server topology with standalone broker
* non-persistent messaging
* producer client sends to queue, consumer client on remote machine listens
to queue

Does my thought process make sense?  I am not necessarily looking to
increase the number of messages sent through the queue, but to allow my
application to have better response times.
-- 
View this message in context: http://old.nabble.com/Poor-design%2C-poor-performance.--Better-design%2C-better-performance--tp27168341p27168341.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Poor design, poor performance. Better design, better performance?

[Rob Davies <ra...@gmail.com>]

Hey - that's great to hear!

On 19 Jan 2010, at 23:13, jongraf wrote:

>
> Our load tests have completed and I am proud to report that reading  
> the
> documentation, educating the entire technical team and documenting the
> various ActiveMQ settings in our properties files has led to a  
> significant
> performance increase.
>
> Our web app hit 1750 hits per second with 1,000 virtual users.  This  
> is way
> better than what we were getting with embedded brokers in each JVM  
> with
> consumers talking to each other via REST.
>
> Lessons: simple architecture, less layers and reading, reading,  
> reading.
> READ!
>
> Read the various enterprise integration patterns and pick one that  
> suits
> your needs.
> In my case: http://www.enterpriseintegrationpatterns.com/Message.html
>
> Learn about the various ActiveMQ Topologies:
> http://activemq.apache.org/topologies.html
> (this should probably be the first page of the documentation)
>
> Many of the Spring examples show the Producer beans and the Consumer  
> beans
> in the same Spring XML.  In a client-server topology, this is a highly
> unlikely scenario.  Your producer and consumer will live in  
> separate .war
> distributions.
>
>
> jongraf wrote:
>>
>>
>> Proposed design:
>> * client-server topology with standalone broker
>> * non-persistent messaging
>> * producer client sends to queue, consumer client on remote machine
>> listens to queue
>>
>> Does my thought process make sense?  I am not necessarily looking to
>> increase the number of messages sent through the queue, but to  
>> allow my
>> application to have better response times.
>>
>
> -- 
> View this message in context: http://old.nabble.com/Poor-design%2C-poor-performance.--Better-design%2C-better-performance--tp27168341p27234416.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/

Re: Poor design, poor performance. Better design, better performance?

[James Strachan <ja...@gmail.com>]

2010/1/19 jongraf <jo...@hbo.com>:
>
> Our load tests have completed and I am proud to report that reading the
> documentation, educating the entire technical team and documenting the
> various ActiveMQ settings in our properties files has led to a significant
> performance increase.
>
> Our web app hit 1750 hits per second with 1,000 virtual users.  This is way
> better than what we were getting with embedded brokers in each JVM with
> consumers talking to each other via REST.
>
> Lessons: simple architecture, less layers and reading, reading, reading.
> READ!
>
> Read the various enterprise integration patterns and pick one that suits
> your needs.
> In my case: http://www.enterpriseintegrationpatterns.com/Message.html

Also worth a read is how to use EIP with ActiveMQ...
http://activemq.apache.org/enterprise-integration-patterns.html

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/

Re: Poor design, poor performance. Better design, better performance?

[jongraf <jo...@hbo.com>]

Our load tests have completed and I am proud to report that reading the
documentation, educating the entire technical team and documenting the
various ActiveMQ settings in our properties files has led to a significant
performance increase.

Our web app hit 1750 hits per second with 1,000 virtual users.  This is way
better than what we were getting with embedded brokers in each JVM with
consumers talking to each other via REST.  

Lessons: simple architecture, less layers and reading, reading, reading. 
READ!

Read the various enterprise integration patterns and pick one that suits
your needs.
In my case: http://www.enterpriseintegrationpatterns.com/Message.html

Learn about the various ActiveMQ Topologies:
http://activemq.apache.org/topologies.html
(this should probably be the first page of the documentation)

Many of the Spring examples show the Producer beans and the Consumer beans
in the same Spring XML.  In a client-server topology, this is a highly
unlikely scenario.  Your producer and consumer will live in separate .war
distributions.  


jongraf wrote:
> 
> 
> Proposed design:
> * client-server topology with standalone broker
> * non-persistent messaging
> * producer client sends to queue, consumer client on remote machine
> listens to queue
> 
> Does my thought process make sense?  I am not necessarily looking to
> increase the number of messages sent through the queue, but to allow my
> application to have better response times.
> 

-- 
View this message in context: http://old.nabble.com/Poor-design%2C-poor-performance.--Better-design%2C-better-performance--tp27168341p27234416.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.