You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Jamie McCrindle <ja...@gmail.com> on 2006/10/11 12:58:29 UTC
surprising performance tuning result with activemq 4.0.1
hi,
just thought i'd mention, i switched from a journaledJDBC backed by a
datastore to a jdbcPersistenceAdapter without the journalling and got
a 30% speed up in our application. obviously this is going to be very
dependent on the type of messaging we're doing but I thought I'd
mention it in the event that someone else is looking for something to
try out to speed things up.
cheers,
j.
Re: surprising performance tuning result with activemq 4.0.1
Posted by yaussy <ya...@cboe.com>.
Just to add - I switched from using the default (journal+derbydb) to Kaha.
Wow - what a difference. The derbydb, once the checkpointing starts, very
much gets in the way. I think we probably have fairly slow disks in the dev
env we were testing on, but nevertheless, switching to Kaha was
night-and-day. On a fairly slow machine, 8 cpu's at 1.349ghz, we could get
a little over 1000/sec (not sure of message size off the top of my head, but
not big/not little - maybe 1k?) between one producer and one consumer.
James.Strachan wrote:
>
> On 10/11/06, Jamie McCrindle <ja...@gmail.com> wrote:
>> Hi James,
>>
>> > Am wondering if you're hitting a slightly different issue though - of
>> > the RAM / gc impact of using the journal causing pauses versus the
>> > much simpler write to JDBC
>>
>> Could be. The test cases run from pretty small to pretty large and they
>> all
>> seem to have the same results. I'm doing the tests on my dev box but
>> hopefully we'll get a chance to do more on a box that matches our
>> production
>> scenario a little better (remote sql server, dedicated box, lots of
>> memory)
>> and I'll see what the numbers look like between them and get them back to
>> you.
>>
>> > BTW it'd be interesting to see the performance if you tried kaha
>> > instead of JDBC, just to get an idea of relative speed
>>
>> Will do as soon as we move to 4.1 (I assume it's only available in 4.1)
>> and
>> get back to you with the results. That said, as soon as 4.1 goes live,
>> we're
>> going to move to Shared JDBC Master / Slave which is why I was trying to
>> get
>> an idea of what the performance of pure JDBC was like.
>
> Yeah - its a very cool option :)
>
> Thanks for the heads up - I was just nosy to find out kaha v JDBC with
> SQLServer :)
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
>
--
View this message in context: http://www.nabble.com/surprising-performance-tuning-result-with-activemq-4.0.1-tf2422666.html#a6783898
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: surprising performance tuning result with activemq 4.0.1
Posted by James Strachan <ja...@gmail.com>.
On 10/11/06, Jamie McCrindle <ja...@gmail.com> wrote:
> Hi James,
>
> > Am wondering if you're hitting a slightly different issue though - of
> > the RAM / gc impact of using the journal causing pauses versus the
> > much simpler write to JDBC
>
> Could be. The test cases run from pretty small to pretty large and they all
> seem to have the same results. I'm doing the tests on my dev box but
> hopefully we'll get a chance to do more on a box that matches our production
> scenario a little better (remote sql server, dedicated box, lots of memory)
> and I'll see what the numbers look like between them and get them back to
> you.
>
> > BTW it'd be interesting to see the performance if you tried kaha
> > instead of JDBC, just to get an idea of relative speed
>
> Will do as soon as we move to 4.1 (I assume it's only available in 4.1) and
> get back to you with the results. That said, as soon as 4.1 goes live, we're
> going to move to Shared JDBC Master / Slave which is why I was trying to get
> an idea of what the performance of pure JDBC was like.
Yeah - its a very cool option :)
Thanks for the heads up - I was just nosy to find out kaha v JDBC with
SQLServer :)
--
James
-------
http://radio.weblogs.com/0112098/
Re: surprising performance tuning result with activemq 4.0.1
Posted by Jamie McCrindle <ja...@gmail.com>.
Hi James,
> Am wondering if you're hitting a slightly different issue though - of
> the RAM / gc impact of using the journal causing pauses versus the
> much simpler write to JDBC
Could be. The test cases run from pretty small to pretty large and they all
seem to have the same results. I'm doing the tests on my dev box but
hopefully we'll get a chance to do more on a box that matches our production
scenario a little better (remote sql server, dedicated box, lots of memory)
and I'll see what the numbers look like between them and get them back to
you.
> BTW it'd be interesting to see the performance if you tried kaha
> instead of JDBC, just to get an idea of relative speed
Will do as soon as we move to 4.1 (I assume it's only available in 4.1) and
get back to you with the results. That said, as soon as 4.1 goes live, we're
going to move to Shared JDBC Master / Slave which is why I was trying to get
an idea of what the performance of pure JDBC was like.
cheers,
j.
On 10/11/06, James Strachan <ja...@gmail.com> wrote:
>
> On 10/11/06, Jamie McCrindle <ja...@gmail.com> wrote:
> > in the test, yes it is.
>
> OK - I just wondered that maybe the journal was on a slower/heavily
> used disk and SQLServer was using a faster one :0
>
>
> > would the journal and sqlserver be queing behind
> > each other for io?
>
> The ideal scenario for using the journal is for it to use a dedicated
> disk and be the only writer, so the disk head doesn't need to do
> seeks, it can just keep writing to the end of a file etc.
>
> Am wondering if you're hitting a slightly different issue though - of
> the RAM / gc impact of using the journal causing pauses versus the
> much simpler write to JDBC
>
> BTW it'd be interesting to see the performance if you tried kaha
> instead of JDBC, just to get an idea of relative speed
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
Re: surprising performance tuning result with activemq 4.0.1
Posted by James Strachan <ja...@gmail.com>.
On 10/11/06, Jamie McCrindle <ja...@gmail.com> wrote:
> in the test, yes it is.
OK - I just wondered that maybe the journal was on a slower/heavily
used disk and SQLServer was using a faster one :0
> would the journal and sqlserver be queing behind
> each other for io?
The ideal scenario for using the journal is for it to use a dedicated
disk and be the only writer, so the disk head doesn't need to do
seeks, it can just keep writing to the end of a file etc.
Am wondering if you're hitting a slightly different issue though - of
the RAM / gc impact of using the journal causing pauses versus the
much simpler write to JDBC
BTW it'd be interesting to see the performance if you tried kaha
instead of JDBC, just to get an idea of relative speed
--
James
-------
http://radio.weblogs.com/0112098/
Re: surprising performance tuning result with activemq 4.0.1
Posted by Jamie McCrindle <ja...@gmail.com>.
in the test, yes it is. would the journal and sqlserver be queing behind
each other for io?
On 10/11/06, James Strachan <ja...@gmail.com> wrote:
>
> On 10/11/06, Jamie McCrindle <ja...@gmail.com> wrote:
> > Hi Rob,
> >
> > The database we're using is SQL Server 2005. The messaging we're doing
> > tends to be relatively slow, i.e. about 40 messages per second off a
> > number of durable queues. Two of the queues are exclusive consumers.
>
> I wonder, was the journal on the same disk as is being used by SQL Server?
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>
Re: surprising performance tuning result with activemq 4.0.1
Posted by James Strachan <ja...@gmail.com>.
On 10/11/06, Jamie McCrindle <ja...@gmail.com> wrote:
> Hi Rob,
>
> The database we're using is SQL Server 2005. The messaging we're doing
> tends to be relatively slow, i.e. about 40 messages per second off a
> number of durable queues. Two of the queues are exclusive consumers.
I wonder, was the journal on the same disk as is being used by SQL Server?
--
James
-------
http://radio.weblogs.com/0112098/
Re: surprising performance tuning result with activemq 4.0.1
Posted by Jamie McCrindle <ja...@gmail.com>.
Hi Rob,
The database we're using is SQL Server 2005. The messaging we're doing
tends to be relatively slow, i.e. about 40 messages per second off a
number of durable queues. Two of the queues are exclusive consumers.
cheers,
j.
On 10/11/06, Rob Davies <ra...@gmail.com> wrote:
> Hi Jamie,
>
> could you share some details about the type of messaging you were
> using, and which database you were using to achieve the speed
> improvement ?
>
> cheers,
>
> Rob
> On 11 Oct 2006, at 11:58, Jamie McCrindle wrote:
>
> > hi,
> >
> > just thought i'd mention, i switched from a journaledJDBC backed by a
> > datastore to a jdbcPersistenceAdapter without the journalling and got
> > a 30% speed up in our application. obviously this is going to be very
> > dependent on the type of messaging we're doing but I thought I'd
> > mention it in the event that someone else is looking for something to
> > try out to speed things up.
> >
> > cheers,
> > j.
>
>
Re: surprising performance tuning result with activemq 4.0.1
Posted by Rob Davies <ra...@gmail.com>.
Hi Jamie,
could you share some details about the type of messaging you were
using, and which database you were using to achieve the speed
improvement ?
cheers,
Rob
On 11 Oct 2006, at 11:58, Jamie McCrindle wrote:
> hi,
>
> just thought i'd mention, i switched from a journaledJDBC backed by a
> datastore to a jdbcPersistenceAdapter without the journalling and got
> a 30% speed up in our application. obviously this is going to be very
> dependent on the type of messaging we're doing but I thought I'd
> mention it in the event that someone else is looking for something to
> try out to speed things up.
>
> cheers,
> j.