You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Patrik Nordwall <pa...@gmail.com> on 2009/10/12 22:13:47 UTC
Performance of persistent messages
Hi, I'm trying to find a message broker that has the performance for
persistent messages that we need.
We have defined the following scenario:
- 6 producers (standalone java applications) that send persistent messages.
Each producer send 20 msg/s. The messages have xml payload, 3 kB. No
transactions in producer.
- 1 message driven bean (running in jboss, it is 15 instances in the mdb
pool) receive the messages and send new messages (same type of payload) to 6
queues, which are consumed by 6 different message driven beans.
We are only using queues.
With this load, the average time to send a message in the producer should be
less than 10 ms.
I'm running broker and jboss on a fast linux server. Producers are running
on separate machine.
I have tried this with 5.3.0 RC4 (which I think will be final today). It
can't handle the load at all. Throttling. Latency in producers about 100ms.
I have tried settings for the persistence adapter kahaDB but without
success.
Can ActiveMQ handle this load with low latency? What configuration should be
used? We can't afford to lose any messages.
I also tried the kahaPersistenceAdapter and it is amazingly fast. It handles
the load without any problems and the average latency in producers to send a
message is 2.5 ms. Is it persisting messages? Is it reliable?
Regards,
Patrik
--
View this message in context: http://www.nabble.com/Performance-of-persistent-messages-tp25862425p25862425.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Performance of persistent messages
Posted by Rob Davies <ra...@gmail.com>.
On 13 Oct 2009, at 10:28, Patrik Nordwall wrote:
>
> I think (hope) I have found the root of problem.
> I found a test program in SonicMQ documentation:
> http://communities.progress.com/pcom/servlet/JiveServlet/download/10720-3-10090/mq_tuning_76.pdf
>
> It tests performance of RandomAccessFile.getFD().sync()
> Is that relevant for ActiveMQ persistence performance?
>
> When running this on the test server I got 145 syncs/second. It
> should be
> mid-range SAN, but it must be something wrong with it.
> I tested on high-end SAN and got 2300 syncs/second.
>
> /Patrik
> --
> View this message in context: http://www.nabble.com/Performance-of-persistent-messages-tp25862425p25869764.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Yes - that would certainly have an impact on performance for
persistent messages :)
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Performance of persistent messages
Posted by Patrik Nordwall <pa...@gmail.com>.
I think (hope) I have found the root of problem.
I found a test program in SonicMQ documentation:
http://communities.progress.com/pcom/servlet/JiveServlet/download/10720-3-10090/mq_tuning_76.pdf
It tests performance of RandomAccessFile.getFD().sync()
Is that relevant for ActiveMQ persistence performance?
When running this on the test server I got 145 syncs/second. It should be
mid-range SAN, but it must be something wrong with it.
I tested on high-end SAN and got 2300 syncs/second.
/Patrik
--
View this message in context: http://www.nabble.com/Performance-of-persistent-messages-tp25862425p25869764.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Performance of persistent messages
Posted by Patrik Nordwall <pa...@gmail.com>.
useAsyncSend gives a great initial boost in the producers, but number of
pending messages are increasing when running at sustained load and after a
while throttling starts and everything slows down.
/Patrik
Joe Fernandez wrote:
>
> FWIW, I have gotten throughput rates of ~3000 msg/s under the following
> conditions.
>
> Used persistent messages, default msg store, message size = 1k, queues,
> and producer had jms.useAsyncSend=true
>
> Broker ran on an openSUSE 11.0 laptop (2GB memory, Intel Core 2 Duo T5550,
> 1.83 GHz, JDK 1.6) and didn't compete with much of anything else on that
> machine. Broker cpu utilization was ~30%.
>
> Producer and consumer ran on their own Windoze XP machines as follows:
>
> producer: Windows XP Pro SP2, 2GB memory, Pentium 4, 3.0GHz, JDK 1.5
> consumer: Windows XP Pro SP2, 2GB memory, Intel Core 2 Duo T7500, 2.0
> GHz, JDK 1.5
>
> Joe
> http://www.ttmsolutions.com
>
>
>
>
> Patrik Nordwall wrote:
>>
>> Hi, I'm trying to find a message broker that has the performance for
>> persistent messages that we need.
>>
>> We have defined the following scenario:
>> - 6 producers (standalone java applications) that send persistent
>> messages. Each producer send 20 msg/s. The messages have xml payload, 3
>> kB. No transactions in producer.
>> - 1 message driven bean (running in jboss, it is 15 instances in the mdb
>> pool) receive the messages and send new messages (same type of payload)
>> to 6 queues, which are consumed by 6 different message driven beans.
>>
>> We are only using queues.
>>
>> With this load, the average time to send a message in the producer should
>> be less than 10 ms.
>>
>> I'm running broker and jboss on a fast linux server. Producers are
>> running on separate machine.
>>
>> I have tried this with 5.3.0 RC4 (which I think will be final today). It
>> can't handle the load at all. Throttling. Latency in producers about
>> 100ms. I have tried settings for the persistence adapter kahaDB but
>> without success.
>>
>> Can ActiveMQ handle this load with low latency? What configuration should
>> be used? We can't afford to lose any messages.
>>
>> I also tried the kahaPersistenceAdapter and it is amazingly fast. It
>> handles the load without any problems and the average latency in
>> producers to send a message is 2.5 ms. Is it persisting messages? Is it
>> reliable?
>>
>> Regards,
>> Patrik
>>
>
>
--
View this message in context: http://www.nabble.com/Performance-of-persistent-messages-tp25862425p25868321.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Performance of persistent messages
Posted by Joe Fernandez <jo...@ttmsolutions.com>.
FWIW, I have gotten throughput rates of ~3000 msg/s under the following
conditions.
Used persistent messages, default msg store, message size = 1k, queues, and
producer had jms.useAsyncSend=true
Broker ran on an openSUSE 11.0 laptop (2GB memory, Intel Core 2 Duo T5550,
1.83 GHz, JDK 1.6) and didn't compete with much of anything else on that
machine. Broker cpu utilization was ~30%.
Producer and consumer ran on their own Windoze XP machines as follows:
producer: Windows XP Pro SP2, 2GB memory, Pentium 4, 3.0GHz, JDK 1.5
consumer: Windows XP Pro SP2, 2GB memory, Intel Core 2 Duo T7500, 2.0 GHz,
JDK 1.5
Joe
http://www.ttmsolutions.com
Patrik Nordwall wrote:
>
> Hi, I'm trying to find a message broker that has the performance for
> persistent messages that we need.
>
> We have defined the following scenario:
> - 6 producers (standalone java applications) that send persistent
> messages. Each producer send 20 msg/s. The messages have xml payload, 3
> kB. No transactions in producer.
> - 1 message driven bean (running in jboss, it is 15 instances in the mdb
> pool) receive the messages and send new messages (same type of payload) to
> 6 queues, which are consumed by 6 different message driven beans.
>
> We are only using queues.
>
> With this load, the average time to send a message in the producer should
> be less than 10 ms.
>
> I'm running broker and jboss on a fast linux server. Producers are running
> on separate machine.
>
> I have tried this with 5.3.0 RC4 (which I think will be final today). It
> can't handle the load at all. Throttling. Latency in producers about
> 100ms. I have tried settings for the persistence adapter kahaDB but
> without success.
>
> Can ActiveMQ handle this load with low latency? What configuration should
> be used? We can't afford to lose any messages.
>
> I also tried the kahaPersistenceAdapter and it is amazingly fast. It
> handles the load without any problems and the average latency in producers
> to send a message is 2.5 ms. Is it persisting messages? Is it reliable?
>
> Regards,
> Patrik
>
--
View this message in context: http://www.nabble.com/Performance-of-persistent-messages-tp25862425p25862740.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.