C++ and Java brokers

[Fraser Adams <fr...@blueyonder.co.uk>]

At the risk of being controversial :-)

So qpid has both C++ and Java brokers available. Out of curiosity (and 
definitely not wishing to start a flame war!!!!) what's the benefit of 
that strategy, particularly because AMQP is language neutral and the C++ 
broker acts as a very nice JMS message provider.

Most of my qpid work has been done using Java clients and the C++ 
broker, so I'm curious as to what the advantages of the Java broker may 
be. As far as I could imagine from general experience of C++ versus Java 
I'd be surprise if the Java broker performs anything like as well as the 
C++ broker (but I'm prepared to be proven wrong!!).

I'm genuinely curious, I'm really not trying to offend anyone working on 
the Java broker so please don't take it that way. I've only ever used 
the C++ broker so I'm coming from a position of total ignorance with 
respect to the Java broker.

I'd appreciate thoughts/opinions.

Cheers,
Frase

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: C++ and Java brokers

[Fraser Adams <fr...@blueyonder.co.uk>]

Thanks Rob.
I really need to have a play with the Java broker some time - I do like the C++ broker though :-)

Re "

Having spent the best part of an afternoon just trying to get the C++
broker and store to build on my Ubuntu 11.10 machine... that is certainly a
huge convenience :-)
"

I don't suppose that you came across my post on a related subject?

http://qpid.2158936.n2.nabble.com/qpid-0-12-make-fails-on-Linux-Mint-12-I-think-derived-from-Ubuntu-11-10-td7140482.html

Were the issues you were seeing related to that (e.g. linker issues caused by ld changes).

I've raised a Jira

https://issues.apache.org/jira/browse/QPID-3723

With a patch.

Have you seen any other issues beyond that? I've not tried the store yet, but I'm expecting similar. If you've got any additions that would help Ubuntu/Mint folks then could you add them to the Jira to make sure they get tracked.

"that is certainly a huge convenience :-)" is an understatement :-D boy was I cursing when I upgraded my Linux box after Xmas to find I couldn't build qpid - bang went my "list of things to do" :-)

Frase.



---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: C++ and Java brokers

[Rob Godfrey <ro...@gmail.com>]

On 5 January 2012 22:56, Daryoush Mehrtash <dm...@gmail.com> wrote:

> >
> > For persistent messaging the solutions have been tuned for very different
> > use cases it seems, with the Java performing better when the clients are
> > operating in a highly synchronous manner with high levels of concurrent
> > connections.  The C++ (on Linux) is obviously much better tuned for the
> > asynchronous persistent messaging case where it can achieve throughputs
> > much higher than the Java could aspire to due to the lack of an
> > asynchronous IO API.
>
>
> Would you please explain what you mean by "highly synchronous manner with
> high levels of concurrent
> connections" and "asynchronous persistent messaging".   An example would
>  help.
>

For the synchronous case I mean something like: Producers and consumers
using transactions for each send or receive.  Multiple such connections
running in parallel (particularly if each connection may be operating on a
different queue).

For the asynchronous case what I mean is that you are not waiting
synchronously after you send/receive each message.  In the producer case
this means that there are some number of messages which it knows it sent
but it doesn't know that the broker has persisted.  On the consumer side it
means that the client may delay acking messages, and/or does not wait to
hear confirmation that the ack has been processed.  The consequence being
overall that there are more messages "in doubt" at any one time, however
your overall throughput is much higher.

Hope this helps,
Rob

>
> Thanks
>
> Daryoush
>
>
> On Thu, Jan 5, 2012 at 1:39 PM, Rob Godfrey <ro...@gmail.com>
> wrote:
>
> > On 5 January 2012 20:39, William Henry <wh...@redhat.com> wrote:
> >
> > > Hi Rob,
> > >
> > > I'm sure others on this list, like myself, would be keen to hear of use
> > > cases where the Java broker outperforms the C++ broker.
> >
> >
> > Don't get me wrong, both brokers are extremely performant... and for
> > transient messaging it's almost always going to be the network that is
> the
> > bottleneck. (And clearly if you have InfiniBand available, then C++ is
> the
> > way to go, since this is not supported in the Java).
> >
> > Moreover any given benchmark test result is going to be completely
> > dependent on environment and you may get completely contradictory results
> > on different hardware.  As I said above, when evaluating performance one
> > must always define the use cases that are actually going to be in play in
> > the proposed deployment, and write performance testing code that tests
> > those cases on your proposed hardware.  It's almost always going to be
> the
> > case between any two Brokers A and B that for some benchmarks A >> B and
> on
> > others B >> A.
> >
> > In general where the broker is not CPU bound I've found that the two
> > brokers have reasonably comparable transient messaging performance with
> the
> > Java Broker normally requiring a little more CPU in order to achieve the
> > same task, but sometimes (especially with smaller messages) seemingly
> able
> > to better eke out a little higher throughput.  The C++ broker seems to
> have
> > an advantage with large message size (32Kb or so) - again though for
> > transient under load the network is probably going to be your bottleneck
> > unless you have a couple of 10G ethernet cards in your machine :-)
> >
> > For persistent messaging the solutions have been tuned for very different
> > use cases it seems, with the Java performing better when the clients are
> > operating in a highly synchronous manner with high levels of concurrent
> > connections.  The C++ (on Linux) is obviously much better tuned for the
> > asynchronous persistent messaging case where it can achieve throughputs
> > much higher than the Java could aspire to due to the lack of an
> > asynchronous IO API. I've not looked at the performance on Windows of the
> > C++ broker with the SQL store.
> >
> > Cheers,
> > Rob
> >
>
>
>
> --
> Daryoush
>
> Weblog:  http://onfp.blogspot.com/
>

Re: C++ and Java brokers

[Daryoush Mehrtash <dm...@gmail.com>]

>
> For persistent messaging the solutions have been tuned for very different
> use cases it seems, with the Java performing better when the clients are
> operating in a highly synchronous manner with high levels of concurrent
> connections.  The C++ (on Linux) is obviously much better tuned for the
> asynchronous persistent messaging case where it can achieve throughputs
> much higher than the Java could aspire to due to the lack of an
> asynchronous IO API.


Would you please explain what you mean by "highly synchronous manner with
high levels of concurrent
connections" and "asynchronous persistent messaging".   An example would
 help.

Thanks

Daryoush


On Thu, Jan 5, 2012 at 1:39 PM, Rob Godfrey <ro...@gmail.com> wrote:

> On 5 January 2012 20:39, William Henry <wh...@redhat.com> wrote:
>
> > Hi Rob,
> >
> > I'm sure others on this list, like myself, would be keen to hear of use
> > cases where the Java broker outperforms the C++ broker.
>
>
> Don't get me wrong, both brokers are extremely performant... and for
> transient messaging it's almost always going to be the network that is the
> bottleneck. (And clearly if you have InfiniBand available, then C++ is the
> way to go, since this is not supported in the Java).
>
> Moreover any given benchmark test result is going to be completely
> dependent on environment and you may get completely contradictory results
> on different hardware.  As I said above, when evaluating performance one
> must always define the use cases that are actually going to be in play in
> the proposed deployment, and write performance testing code that tests
> those cases on your proposed hardware.  It's almost always going to be the
> case between any two Brokers A and B that for some benchmarks A >> B and on
> others B >> A.
>
> In general where the broker is not CPU bound I've found that the two
> brokers have reasonably comparable transient messaging performance with the
> Java Broker normally requiring a little more CPU in order to achieve the
> same task, but sometimes (especially with smaller messages) seemingly able
> to better eke out a little higher throughput.  The C++ broker seems to have
> an advantage with large message size (32Kb or so) - again though for
> transient under load the network is probably going to be your bottleneck
> unless you have a couple of 10G ethernet cards in your machine :-)
>
> For persistent messaging the solutions have been tuned for very different
> use cases it seems, with the Java performing better when the clients are
> operating in a highly synchronous manner with high levels of concurrent
> connections.  The C++ (on Linux) is obviously much better tuned for the
> asynchronous persistent messaging case where it can achieve throughputs
> much higher than the Java could aspire to due to the lack of an
> asynchronous IO API. I've not looked at the performance on Windows of the
> C++ broker with the SQL store.
>
> Cheers,
> Rob
>



-- 
Daryoush

Weblog:  http://onfp.blogspot.com/

Re: C++ and Java brokers

[Rob Godfrey <ro...@gmail.com>]

On 5 January 2012 20:39, William Henry <wh...@redhat.com> wrote:

> Hi Rob,
>
> I'm sure others on this list, like myself, would be keen to hear of use
> cases where the Java broker outperforms the C++ broker.


Don't get me wrong, both brokers are extremely performant... and for
transient messaging it's almost always going to be the network that is the
bottleneck. (And clearly if you have InfiniBand available, then C++ is the
way to go, since this is not supported in the Java).

Moreover any given benchmark test result is going to be completely
dependent on environment and you may get completely contradictory results
on different hardware.  As I said above, when evaluating performance one
must always define the use cases that are actually going to be in play in
the proposed deployment, and write performance testing code that tests
those cases on your proposed hardware.  It's almost always going to be the
case between any two Brokers A and B that for some benchmarks A >> B and on
others B >> A.

In general where the broker is not CPU bound I've found that the two
brokers have reasonably comparable transient messaging performance with the
Java Broker normally requiring a little more CPU in order to achieve the
same task, but sometimes (especially with smaller messages) seemingly able
to better eke out a little higher throughput.  The C++ broker seems to have
an advantage with large message size (32Kb or so) - again though for
transient under load the network is probably going to be your bottleneck
unless you have a couple of 10G ethernet cards in your machine :-)

For persistent messaging the solutions have been tuned for very different
use cases it seems, with the Java performing better when the clients are
operating in a highly synchronous manner with high levels of concurrent
connections.  The C++ (on Linux) is obviously much better tuned for the
asynchronous persistent messaging case where it can achieve throughputs
much higher than the Java could aspire to due to the lack of an
asynchronous IO API. I've not looked at the performance on Windows of the
C++ broker with the SQL store.

Cheers,
Rob

Re: C++ and Java brokers

[William Henry <wh...@redhat.com>]

----- Original Message -----
> On 5 January 2012 19:46, Fraser Adams <fr...@blueyonder.co.uk>
> wrote:
> 
> > At the risk of being controversial :-)
> >
> > So qpid has both C++ and Java brokers available. Out of curiosity
> > (and
> > definitely not wishing to start a flame war!!!!) what's the benefit
> > of that
> > strategy, particularly because AMQP is language neutral and the C++
> > broker
> > acts as a very nice JMS message provider.
> >
> >
> So the most obvious difference is that the Java Broker runs pretty
> much
> anywhere and has the same functionality across all those platforms...
> Having spent the best part of an afternoon just trying to get the C++
> broker and store to build on my Ubuntu 11.10 machine... that is
> certainly a
> huge convenience :-) For a lot of organisations, including the one I
> work
> at, having the ability to deploy exactly the same binaries and
> configurations on Windows, Linux (all flavours), Solaris, OS X,
> etc... does
> turn out to be surprisingly convenient.
> 
> The Java Broker also allows communication using older versions of
> AMQP with
> automatic conversion (i.e. you can mix AMQP 0-8,0-9,0-9-1 and 0-10
> clients
> on the same broker, using the same queues).
> 
> Most of my qpid work has been done using Java clients and the C++
> broker,
> > so I'm curious as to what the advantages of the Java broker may be.
> > As far
> > as I could imagine from general experience of C++ versus Java I'd
> > be
> > surprise if the Java broker performs anything like as well as the
> > C++
> > broker (but I'm prepared to be proven wrong!!).
> >
> >
> There are use cases where the C++ Broker handily out performs the
> Java
> Broker, certainly... but there are others where you'd be surprised
> that the
> reverse is true.

Hi Rob,

I'm sure others on this list, like myself, would be keen to hear of use cases where the Java broker outperforms the C++ broker. What does that look like?

Thanks,
William

> 
> 
> > I'm genuinely curious, I'm really not trying to offend anyone
> > working on
> > the Java broker so please don't take it that way. I've only ever
> > used the
> > C++ broker so I'm coming from a position of total ignorance with
> > respect to
> > the Java broker.
> >
> > I'd appreciate thoughts/opinions.
> >
> >
> From a historical point of view, the Java Broker was written first,
> and the
> C++ Broker was developed with the aim of better exploiting
> performance
> opportunities that could be gained by using lower level C APIs
> (particularly from Linux).
> 
> Historically we've not done as good a job as we could have in keeping
> the
> brokers in sync in terms of functionality, but hopefully we will
> improve on
> that in the future.
> 
> Cheers,
> Rob
> 

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: C++ and Java brokers

[Rob Godfrey <ro...@gmail.com>]

On 5 January 2012 19:46, Fraser Adams <fr...@blueyonder.co.uk> wrote:

> At the risk of being controversial :-)
>
> So qpid has both C++ and Java brokers available. Out of curiosity (and
> definitely not wishing to start a flame war!!!!) what's the benefit of that
> strategy, particularly because AMQP is language neutral and the C++ broker
> acts as a very nice JMS message provider.
>
>
So the most obvious difference is that the Java Broker runs pretty much
anywhere and has the same functionality across all those platforms...
Having spent the best part of an afternoon just trying to get the C++
broker and store to build on my Ubuntu 11.10 machine... that is certainly a
huge convenience :-) For a lot of organisations, including the one I work
at, having the ability to deploy exactly the same binaries and
configurations on Windows, Linux (all flavours), Solaris, OS X, etc... does
turn out to be surprisingly convenient.

The Java Broker also allows communication using older versions of AMQP with
automatic conversion (i.e. you can mix AMQP 0-8,0-9,0-9-1 and 0-10 clients
on the same broker, using the same queues).

Most of my qpid work has been done using Java clients and the C++ broker,
> so I'm curious as to what the advantages of the Java broker may be. As far
> as I could imagine from general experience of C++ versus Java I'd be
> surprise if the Java broker performs anything like as well as the C++
> broker (but I'm prepared to be proven wrong!!).
>
>
There are use cases where the C++ Broker handily out performs the Java
Broker, certainly... but there are others where you'd be surprised that the
reverse is true.


> I'm genuinely curious, I'm really not trying to offend anyone working on
> the Java broker so please don't take it that way. I've only ever used the
> C++ broker so I'm coming from a position of total ignorance with respect to
> the Java broker.
>
> I'd appreciate thoughts/opinions.
>
>
>From a historical point of view, the Java Broker was written first, and the
C++ Broker was developed with the aim of better exploiting performance
opportunities that could be gained by using lower level C APIs
(particularly from Linux).

Historically we've not done as good a job as we could have in keeping the
brokers in sync in terms of functionality, but hopefully we will improve on
that in the future.

Cheers,
Rob