You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Alan Conway <ac...@redhat.com> on 2006/10/25 21:23:21 UTC
Re: Performance comparison for C++ and Java.
OK, some slightly saner performance results but still not seeing C++
slowing down.
First note I've got client and server on the same box, and the box has 2
cpus, not 8 (dual core with hyper threading so linux reports each one as
4 cpus.) so slower times than the JPM test is normal.
I built C++ optimized (-O3 -DNDEBUG), used java 1.6.0-beta2 with the
flags below and ran the test 3 times against each server.
Results:
Java broker
java -server -Xmx1024m -Damqj.logging.level=fatal
-DQPID_HOME=/home/aconway/svn/qpid/java/build -DQPID_WORK=/home/aconway
-Xmx1024M
Broker started: qpid-server
20 batches completed. avg=3169, max=6140, min=2473
20 batches completed. avg=2672, max=3094, min=2270
20 batches completed. avg=2672, max=3137, min=2335
C++ broker
Broker started: qpidd
20 batches completed. avg=1973, max=3340, min=1701
20 batches completed. avg=2028, max=2445, min=1891
20 batches completed. avg=2141, max=2613, min=1908
Still lots slower than JPM but that's reasonable given 2 vs. 8 CPUs and
clients+server on same box. The Java broker does indeed have a warm-up
issue so we'll ignore the first test run against it.
C++ broker remains faster by about 20%.
Robert, Martin can either of you try this on the JPM machine? I've
attached my test scripts, put them in your path. Add qpid/cpp/bin and
qpid/cpp/test/client to the path then run topicall.
Cheers,
Alan.
On Mon, 2006-10-23 at 13:48 +0100, Martin Ritchie wrote:
> I have run the test as described above. The only deviation may have
> been setting -Xmx1024M. Although this is the default set by the
> scripts.
>
> The broker was restarted between test and the SVN revision was: 466043.
>
> I ran the tests twice:
> 1) All processes on a single box.
> 2) Broker on 8 way opteron, Clients on 4 way opteron.
>
> Unfortunately we cannot get exclusive access to these boxes however
> the loading on the machines are quite low.
>
> In both cases a significant performance increase was shown. I believe
> this is down to memory usage. The test utilises around 2.2G of RAM
> which may have caused a problem on the test box at Redhat.
>
Re: Fwd: Performance comparison for C++ and Java.
Posted by Gordon Sim <gs...@redhat.com>.
Robert Greig wrote:
> Incidentally, how do I get usage?
> I tried everything I could think of --help, -h /?, /help and it didn't
> seem to work.
Yes, I discovered a wee regression bug there! Sorry, will fix it shortly.
Re: Fwd: Performance comparison for C++ and Java.
Posted by Robert Greig <ro...@gmail.com>.
On 26/10/06, Gordon Sim <gs...@redhat.com> wrote:
> Interesting. Did you alter the number of threads for the c++ broker
> (--worker-threads)? The default is 5. Last time round you noticed an
> improvement with a higher number I believe.
Ah, I had forgotten about that flag. Incidentally, how do I get usage?
I tried everything I could think of --help, -h /?, /help and it didn't
seem to work.
I will try that and get back to you.
> > What is your threading model?
>
> A separate thread for accepting connections, then a pool of workers used
> in a leader-follower pattern processing io events for each connection.
So does a single thread process the entire thing - decoding of bytes
into protocol frame, processing of frame (i.e. routing) then encoding
etc?
RG
Re: Fwd: Performance comparison for C++ and Java.
Posted by Alan Conway <ac...@redhat.com>.
Thanks. In both 5 and 8 thread cases there's a steep linear climb after
2 clients, which is not good. Java is starting to climb after 4 but not
nearly as steep.
I'm investigating!
On Thu, 2006-10-26 at 16:58 +0100, Robert Greig wrote:
> On 26/10/06, Alan Conway <ac...@redhat.com> wrote:
>
> > > With 10 threads, the 8 client test takes 1589ms. With 20 it takes 1190
> > > and increasing to 30 threads did not improve that any further.
> >
> > Do you have 2 and 4 client results with 20 threads?
>
> 2 clients: 376
> 4 clients: 632
>
> RG
Re: Fwd: Performance comparison for C++ and Java.
Posted by Robert Greig <ro...@gmail.com>.
On 26/10/06, Alan Conway <ac...@redhat.com> wrote:
> > With 10 threads, the 8 client test takes 1589ms. With 20 it takes 1190
> > and increasing to 30 threads did not improve that any further.
>
> Do you have 2 and 4 client results with 20 threads?
2 clients: 376
4 clients: 632
RG
Re: Fwd: Performance comparison for C++ and Java.
Posted by Alan Conway <ac...@redhat.com>.
On Thu, 2006-10-26 at 13:30 +0100, Robert Greig wrote:
> On 26/10/06, Gordon Sim <gs...@redhat.com> wrote:
> > Interesting. Did you alter the number of threads for the c++ broker
> > (--worker-threads)? The default is 5. Last time round you noticed an
> > improvement with a higher number I believe.
>
> With 10 threads, the 8 client test takes 1589ms. With 20 it takes 1190
> and increasing to 30 threads did not improve that any further.
Do you have 2 and 4 client results with 20 threads?
Thanks,
Alan.
Re: Fwd: Performance comparison for C++ and Java.
Posted by Robert Greig <ro...@gmail.com>.
On 26/10/06, Gordon Sim <gs...@redhat.com> wrote:
> Interesting. Did you alter the number of threads for the c++ broker
> (--worker-threads)? The default is 5. Last time round you noticed an
> improvement with a higher number I believe.
With 10 threads, the 8 client test takes 1589ms. With 20 it takes 1190
and increasing to 30 threads did not improve that any further.
RG
Re: Fwd: Performance comparison for C++ and Java.
Posted by Gordon Sim <gs...@redhat.com>.
Robert Greig wrote:
> I wanted to see how both solutions scaled so did tests with increasing
> numbers of clients, starting with a single client. The results were:
>
> No clients | CPP | Java
> 1 | 320 | 324
> 2 | 395 | 359
> 4 | 979 | 396
> 8 | 1912 | 625
Interesting. Did you alter the number of threads for the c++ broker
(--worker-threads)? The default is 5. Last time round you noticed an
improvement with a higher number I believe.
> What is your threading model?
A separate thread for accepting connections, then a pool of workers used
in a leader-follower pattern processing io events for each connection.
Fwd: Performance comparison for C++ and Java.
Posted by Robert Greig <ro...@gmail.com>.
On 25/10/06, Alan Conway <ac...@redhat.com> wrote:
> OK, some slightly saner performance results but still not seeing C++
> slowing down.
I have run the test and see very similar results to those I saw before.
> I built C++ optimized (-O3 -DNDEBUG), used java 1.6.0-beta2 with the
> flags below and ran the test 3 times against each server.
I built debug and optimised (just to make sure I saw a difference) and
debug was thankfully significantly slower than optimised.
I was seeing times of about 2400ms for 10 clients compared with about
750ms with Java.
I am running the Sun 1.6 build 102.
I wanted to see how both solutions scaled so did tests with increasing
numbers of clients, starting with a single client. The results were:
No clients | CPP | Java
1 | 320 | 324
2 | 395 | 359
4 | 979 | 396
8 | 1912 | 625
What is your threading model? I know it will certainly be different
from the Java broker just because MINA uses what I believe (?) is an
unusual model for I/O - at least I haven't seen that model before. I
tried putting the number of I/O processor threads down to 1 on the
java and it slowed down by just under 50% - 8 clients was taking
890ms.
RG