You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by cityjms <de...@gmail.com> on 2011/05/07 05:37:42 UTC
disk IO rate peak?
- using 5.5.0
- using KahaDB store
- publishing to a topic with 10 durables (offline at the time of the publish
rate test)
- message size 500 byte
cannot get over 2.5MB per second disk write (approx 5K messages per second).
tried local disk, ssd, tmpfs ram mount and Windows PC - nothing helps.
the rate is stable whether I publish 25K messages or 40 million messages :(
KahaDB message store is kept in memory, so I guess it is only the journals
that are being written.
I doubt this is TCP issue. Have plenty of CPU (16 core box with 128GB of
RAM).
ActiveMQ disk benchmark tells me I should get 2600MB per second write, which
is a bit optimistic but certainly better than 2.5 mb :(
help!
--
View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3504958.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: disk IO rate peak?
Posted by cityjms <de...@gmail.com>.
done
https://issues.apache.org/jira/browse/AMQ-3309
On May 8, 2011, at 4:16 PM, Gary Tully [via ActiveMQ] wrote:
> http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3507749.html
--
View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3508399.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: disk IO rate peak?
Posted by Gary Tully <ga...@gmail.com>.
There was some recent work on message priority support for durable
subs which puts more pressure on the index for each message send. My
suspicion is, that is the cause of the slow down. Unfortunately it is
not possible to disable priority ordering at the moment.
Can you open a jira issue to track this, we should be able to attain
parity with the amqPersistenceAdapter for this use case. I will try
and get some time over the next few weeks to investigate further.
On 8 May 2011 06:04, cityjms <de...@gmail.com> wrote:
> I switched to amqPersistenceAdapter and got 5X the speed.
>
> any ideas?
>
> I was under the impression that KahaDB is the latest/greatest...
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3506515.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
http://blog.garytully.com
http://fusesource.com
Re: disk IO rate peak?
Posted by cityjms <de...@gmail.com>.
I switched to amqPersistenceAdapter and got 5X the speed.
any ideas?
I was under the impression that KahaDB is the latest/greatest...
--
View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3506515.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: disk IO rate peak?
Posted by cityjms <de...@gmail.com>.
Hi Stan
Yes, I have this on, plus optimized dispatches and async send on the publisher side....
Also tried the sample activemq code to make sure it isn't my code.
On May 7, 2011, at 8:25, "Stan Lewis [via ActiveMQ]"<ml...@n4.nabble.com> wrote:
> Have you tried the settings from the example configuration file
> conf/activemq-throughput.xml:
>
> <persistenceAdapter>
> <kahaDB directory="${activemq.base}/data/kahadb"
> enableJournalDiskSyncs="false" indexWriteBatchSize="10000"
> indexCacheSize="1000"/>
> </persistenceAdapter>
>
> turning off journal disk syncs should have the biggest impact, however
> the tradeoff is some reliability since the default (true) is to do a
> blocking fsync on every message.
>
> On Fri, May 6, 2011 at 11:39 PM, cityjms <[hidden email]> wrote:
> > - GC doesn't seem to be an issue
> > - threads are under 100
> > - publishing connection ranging from 1 to 40 - doesn't affect the result
> >
> > --
> > View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3504962.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
>
>
> If you reply to this email, your message will be added to the discussion below:
> http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3505401.html
> To unsubscribe from disk IO rate peak?, click here.
--
View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3505584.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: disk IO rate peak?
Posted by Stan Lewis <sl...@fusesource.com>.
Have you tried the settings from the example configuration file
conf/activemq-throughput.xml:
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb"
enableJournalDiskSyncs="false" indexWriteBatchSize="10000"
indexCacheSize="1000"/>
</persistenceAdapter>
turning off journal disk syncs should have the biggest impact, however
the tradeoff is some reliability since the default (true) is to do a
blocking fsync on every message.
On Fri, May 6, 2011 at 11:39 PM, cityjms <de...@gmail.com> wrote:
> - GC doesn't seem to be an issue
> - threads are under 100
> - publishing connection ranging from 1 to 40 - doesn't affect the result
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3504962.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: disk IO rate peak?
Posted by cityjms <de...@gmail.com>.
- GC doesn't seem to be an issue
- threads are under 100
- publishing connection ranging from 1 to 40 - doesn't affect the result
--
View this message in context: http://activemq.2283324.n4.nabble.com/disk-IO-rate-peak-tp3504958p3504962.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.