You are viewing a plain text version of this content. The canonical link for it is here.
Posted to proton@qpid.apache.org by Ted Ross <tr...@redhat.com> on 2013/01/02 20:14:37 UTC
Proton Messenger and the Request/Response pattern
I'd like to start a discussion on how, from an API perspective,
applications can use the request/response pattern. If we get this
right, we will remove a significant barrier to adoption of AMQP.
Middleware messaging systems typically do a poor job of supporting this
pattern. The Qpid APIs are quite lacking in this regard (requester
creates and subscribes to a temporary queue with a _unique_ name and
places this name in the reply-to field).
Proton Messenger supports request/reply (see
examples/messenger/$LANG/{client,server}) as follows:
The requester (client) has to put _something_ into the request message's
reply_to field. It also sets the correlation_id field if it needs to
dispatch multiple responses. The responder (server) must copy the
request message's reply_to field to the response message's address field
and also copy the correlation_id.
This API is good for the case where the client wants the response to go
to a third party. In this case the reply_to is well understood to be
the address of the third party receiver. However in the more common
case where the response is intended to come back to the client, it's not
clear at all what to put in the reply_to field.
I propose that we allow the client to simply say
request_msg.reply_expected(cid)
(I added the correlation_id argument because it's almost always going to
be needed). Further, the server could use
reply_msg.in_reply_to(request_msg)
which would take care of the addresses and the correlation_id. It also
provides a place to report an error if a request cannot be replied to
(absent or invalid reply_to address) rather than force each server
implementer to code this check.
Thoughts?
-Ted
Re: Proton Messenger and the Request/Response pattern
Posted by Jakub Scholz <ja...@scholz.cz>.
Hi Ted,
>From my experience I have to agree with you, that the way the
request/response is handled in the current API is a bit complicated and
caused some troubles to some of our customers connecting to our brokers.
But at the same time I like the flexibility of the current approach, where
you have a high level of control about the queues / exchanges used to
deliver the responses. In our use case, when we have a lot of 3rd party
customers connecting to our broker, it is the security and the operability
what is important for us. For example having the response queues to follow
some naming schema based on the user name gives you a quick overview about
which queues belong to which customers and at the same time makes it really
easy to ensure the security using ACLs. Also, when you have a broker where
multiple different clients are connecting, you have to validate the
reply_to address to avoid sending the response(s) to a different customer
(either by mistake or by intention).
Of course the best would be if we can have both - the simplicity as well as
the flexibility. But I'm not sure whether it is possible to add the
flexibility to your proposal without making it too complicated again.
Having some default behavior and some configuration options to affect the
default behavior might be one of the possibilities.
Regards
Jakub
On Wed, Jan 2, 2013 at 8:14 PM, Ted Ross <tr...@redhat.com> wrote:
> I'd like to start a discussion on how, from an API perspective,
> applications can use the request/response pattern. If we get this right,
> we will remove a significant barrier to adoption of AMQP.
>
> Middleware messaging systems typically do a poor job of supporting this
> pattern. The Qpid APIs are quite lacking in this regard (requester creates
> and subscribes to a temporary queue with a _unique_ name and places this
> name in the reply-to field).
>
> Proton Messenger supports request/reply (see examples/messenger/$LANG/{**client,server})
> as follows:
>
> The requester (client) has to put _something_ into the request message's
> reply_to field. It also sets the correlation_id field if it needs to
> dispatch multiple responses. The responder (server) must copy the request
> message's reply_to field to the response message's address field and also
> copy the correlation_id.
>
> This API is good for the case where the client wants the response to go to
> a third party. In this case the reply_to is well understood to be the
> address of the third party receiver. However in the more common case where
> the response is intended to come back to the client, it's not clear at all
> what to put in the reply_to field.
>
> I propose that we allow the client to simply say
>
> request_msg.reply_expected(**cid)
>
> (I added the correlation_id argument because it's almost always going to
> be needed). Further, the server could use
>
> reply_msg.in_reply_to(request_**msg)
>
> which would take care of the addresses and the correlation_id. It also
> provides a place to report an error if a request cannot be replied to
> (absent or invalid reply_to address) rather than force each server
> implementer to code this check.
>
> Thoughts?
>
> -Ted
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Proton Messenger and the Request/Response pattern
Posted by Jakub Scholz <ja...@scholz.cz>.
Hi Ted,
>From my experience I have to agree with you, that the way the
request/response is handled in the current API is a bit complicated and
caused some troubles to some of our customers connecting to our brokers.
But at the same time I like the flexibility of the current approach, where
you have a high level of control about the queues / exchanges used to
deliver the responses. In our use case, when we have a lot of 3rd party
customers connecting to our broker, it is the security and the operability
what is important for us. For example having the response queues to follow
some naming schema based on the user name gives you a quick overview about
which queues belong to which customers and at the same time makes it really
easy to ensure the security using ACLs. Also, when you have a broker where
multiple different clients are connecting, you have to validate the
reply_to address to avoid sending the response(s) to a different customer
(either by mistake or by intention).
Of course the best would be if we can have both - the simplicity as well as
the flexibility. But I'm not sure whether it is possible to add the
flexibility to your proposal without making it too complicated again.
Having some default behavior and some configuration options to affect the
default behavior might be one of the possibilities.
Regards
Jakub
On Wed, Jan 2, 2013 at 8:14 PM, Ted Ross <tr...@redhat.com> wrote:
> I'd like to start a discussion on how, from an API perspective,
> applications can use the request/response pattern. If we get this right,
> we will remove a significant barrier to adoption of AMQP.
>
> Middleware messaging systems typically do a poor job of supporting this
> pattern. The Qpid APIs are quite lacking in this regard (requester creates
> and subscribes to a temporary queue with a _unique_ name and places this
> name in the reply-to field).
>
> Proton Messenger supports request/reply (see examples/messenger/$LANG/{**client,server})
> as follows:
>
> The requester (client) has to put _something_ into the request message's
> reply_to field. It also sets the correlation_id field if it needs to
> dispatch multiple responses. The responder (server) must copy the request
> message's reply_to field to the response message's address field and also
> copy the correlation_id.
>
> This API is good for the case where the client wants the response to go to
> a third party. In this case the reply_to is well understood to be the
> address of the third party receiver. However in the more common case where
> the response is intended to come back to the client, it's not clear at all
> what to put in the reply_to field.
>
> I propose that we allow the client to simply say
>
> request_msg.reply_expected(**cid)
>
> (I added the correlation_id argument because it's almost always going to
> be needed). Further, the server could use
>
> reply_msg.in_reply_to(request_**msg)
>
> which would take care of the addresses and the correlation_id. It also
> provides a place to report an error if a request cannot be replied to
> (absent or invalid reply_to address) rather than force each server
> implementer to code this check.
>
> Thoughts?
>
> -Ted
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Proton Messenger and the Request/Response pattern
Posted by Rajith Attapattu <ra...@gmail.com>.
I'd agree with Gordon.
1. We should keep the Message as a pure value object without any sort
of coupling to Messenger or other objects.
2. I'm in favor of layering features on top of a generic flexible core
component rather than putting them all in the same layer.
This allows us the freedom to add "extra features" without
sacrificing the "core features" that makes messenger attractive in the
first place.
So things like Request/Response, A Synchronous version of
Messenger, Message Listener/Callback features etc.. can be implemented
as separate layers on top of the Messenger instead of passing flags
for blocking etc.. The code is going to be more complicated and error
prone and less maintainable in the long run.
Rajith
On Fri, Jan 18, 2013 at 6:24 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 01/02/2013 07:14 PM, Ted Ross wrote:
>>
>> I'd like to start a discussion on how, from an API perspective,
>> applications can use the request/response pattern. If we get this
>> right, we will remove a significant barrier to adoption of AMQP.
>>
>> Middleware messaging systems typically do a poor job of supporting this
>> pattern. The Qpid APIs are quite lacking in this regard (requester
>> creates and subscribes to a temporary queue with a _unique_ name and
>> places this name in the reply-to field).
>>
>> Proton Messenger supports request/reply (see
>> examples/messenger/$LANG/{client,server}) as follows:
>>
>> The requester (client) has to put _something_ into the request message's
>> reply_to field. It also sets the correlation_id field if it needs to
>> dispatch multiple responses. The responder (server) must copy the
>> request message's reply_to field to the response message's address field
>> and also copy the correlation_id.
>>
>> This API is good for the case where the client wants the response to go
>> to a third party. In this case the reply_to is well understood to be
>> the address of the third party receiver. However in the more common
>> case where the response is intended to come back to the client, it's not
>> clear at all what to put in the reply_to field.
>>
>> I propose that we allow the client to simply say
>>
>> request_msg.reply_expected(cid)
>
>
> A message has no association with a messenger (until it is put onto the
> messengers outgoing queue at least). It is not clear how a method on message
> would therefore be in any better position to determine the reply-address to
> use.
>
> At present, if a message on the outgoing queue has no reply-to set, then the
> messenger will automatically set the reply-to to be an address corresponding
> to itself.
>
> You can also use the (in my view slightly clumsy) syntax ~/xyz, which will
> be 'munged' into a form that prepends the messengers name to form a full
> address. This form I presume would be used if you want to distinguish
> different categories of replies.
>
> I'm not exactly enamoured with this, but it does seem at least to address
> the key concern expressed here, namely what to put in the reply-to for the
> simple case. (The answer being, nothing!)
>
>
>> (I added the correlation_id argument because it's almost always going to
>> be needed). Further, the server could use
>>
>> reply_msg.in_reply_to(request_msg)
>>
>> which would take care of the addresses and the correlation_id. It also
>> provides a place to report an error if a request cannot be replied to
>> (absent or invalid reply_to address) rather than force each server
>> implementer to code this check.
>
>
> This is really just a short hand for:
>
> response.address = request.reply_to
> response.correlation_id = request.correlation_id
>
> plus some code or exception indicating whether the address has been set. I
> have no objection to such a utility, though I'm not entirely convinced that
> a method on message is the best place. It also seems of fairly marginal
> benefit rather than removing a 'significant barrier to adoption'.
>
> Have you seen the JMS solution to simplifying the request-response pattern?
> There it is done by specifying a distinct class that uses a supplied session
> and destination and provides a request() method that takes a request message
> as input and blocks until it can return the response message. (See
> http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html).
> Details of creating temporary response queues etc are hidden behind this. I
> like that general approach as it layers the functionality cleanly on top of
> the more flexible base, and would also allow different configurations or
> conventions to be used.
>
> --Gordon.
>
Re: Proton Messenger and the Request/Response pattern
Posted by Michael Goulish <mg...@redhat.com>.
----- Original Message -----
>
>
> ----- Original Message -----
> > From: "Michael Goulish" <mg...@redhat.com>
> > To: proton@qpid.apache.org
> > Sent: Thursday, March 7, 2013 9:48:24 AM
> > Subject: Re: Proton Messenger and the Request/Response pattern
> >
> >
> >
> > ----- Original Message -----
> > > Hi All,
> > >
> > > I wanted to re-raise Ted's suggestion regarding the need for a
> > > better
> > > Messenger API for handling reply-to's.
> > >
> > > I've could've used this recently while designing a "echo" stress
> > > test
> > > for Messenger. For my client, I have two requirements:
> > >
> > > 1) I don't want to have to assign a publicly visible address to
> > > the
> > > client in order to receive replies.
> > >
> > > 2) I only want a reply for particular messages.
> > >
> > > I think these two requirements are common enough to merit some
> > > Messenger API lovin'.
> > >
> > > In that context, Ted's API recommendation makes my life as the
> > > client
> > > writer easier: It doesn't require me to obtain a reply-to value
> > > up-front (ie. before calling pn_messenger_put()), and it doesn't
> > > cause every message to be sent with a reply-to (which, I believe
> > > is
> > > the current case - messenger always adds a reply-to if not set).
> > >
> > > I think there's value in Ted's proposal, but I'd recommend a
> > > slight
> > > modification to it: rather than mark the message directly,
> > > instead
> > > indicate to Messenger that a particular message needs its
> > > reply-to
> > > set correctly prior to transmitting it. For example, we could
> > > add
> > > a
> > > flag to pn_messenger_put():
> > >
> > > msg = pn_message();
> > > pn_message_set_address( msg, "peer's remote address" );
> > > .... set message contents, cid, etc ...
> > > pn_messenger_put( messenger, msg, PN_SET_REPLY_TO );
> > >
> >
> >
> > I think this requires little or no change to the user's mental
> > model,
> > which is a _good thing_.
> >
> > But wouldn't the following be more natural? :
> >
> > pn_message_set_address( msg, peer_addr, PN_SET_REPLY_TO );
> >
> > The address setting is done when your code is thinking about
> > addresses,
> > and put() only puts.
> >
> >
>
> This would require the message object to contain some extra state,
> since the reply-to address may not be known at the point the
> pn_message_set_address() is invoked. That extra state - a flag to
> tell Messenger to fill in the reply-to - is kinda sorta Messenger
> specific information that would be piggybacked on the message. I
> seem to recall the consensus was against that.
>
> /my memory ain't what is once was....
Ah, good point. The code that the app is talking to when it is
setting up msg addresses doesn't know how to set up that address.
Dang. OK, _nolo contendere_.
>
> >
> >
> >
> > >
> > > Setting the PN_SET_REPLY_TO would cause Messenger to set the
> > > reply-to
> > > properly prior to sending, or signal an error if unable to do so
> > > (reported via a tracker, as usual).
> > >
> > > This API change would satisfy my two requirements, while keeping
> > > the
> > > whole address management stuff out of my client.
> > >
> > > Thoughts?
> > >
> > > -K
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Rafael Schloming" <rh...@alum.mit.edu>
> > > > To: proton@qpid.apache.org
> > > > Sent: Friday, January 18, 2013 9:13:34 AM
> > > > Subject: Re: Proton Messenger and the Request/Response pattern
> > > >
> > > > Gordon makes some good points. I'd like to add that I think
> > > > historically a
> > > > big part of the hassle isn't actually necessarily solely the
> > > > API
> > > > but
> > > > also
> > > > having to configure and manage the intermediary, and I think we
> > > > need
> > > > to
> > > > look there as well if we want to simplify the overall pattern.
> > > >
> > > > It's also worth remembering that there are significant
> > > > performance
> > > > benefits
> > > > to keeping Message as a pure value object (i.e. a content
> > > > holder)
> > > > that can
> > > > be reused many times even across multiple Messengers. Some of
> > > > the
> > > > suggestions really stray from this pattern. If I recall
> > > > correctly,
> > > > I
> > > > think
> > > > we said in prior threads that we wanted to maintain the notion
> > > > of
> > > > Message
> > > > being a reusable content holder as an architectural invariant.
> > > > Perhaps it's
> > > > worth re-examining what we could do for request/response while
> > > > keeping this
> > > > invariant in mind.
> > > >
> > > > --Rafael
> > > >
> > > > On Fri, Jan 18, 2013 at 6:24 AM, Gordon Sim <gs...@redhat.com>
> > > > wrote:
> > > >
> > > > > On 01/02/2013 07:14 PM, Ted Ross wrote:
> > > > >
> > > > >> I'd like to start a discussion on how, from an API
> > > > >> perspective,
> > > > >> applications can use the request/response pattern. If we
> > > > >> get
> > > > >> this
> > > > >> right, we will remove a significant barrier to adoption of
> > > > >> AMQP.
> > > > >>
> > > > >> Middleware messaging systems typically do a poor job of
> > > > >> supporting
> > > > >> this
> > > > >> pattern. The Qpid APIs are quite lacking in this regard
> > > > >> (requester
> > > > >> creates and subscribes to a temporary queue with a _unique_
> > > > >> name
> > > > >> and
> > > > >> places this name in the reply-to field).
> > > > >>
> > > > >> Proton Messenger supports request/reply (see
> > > > >> examples/messenger/$LANG/{**client,server}) as follows:
> > > > >>
> > > > >> The requester (client) has to put _something_ into the
> > > > >> request
> > > > >> message's
> > > > >> reply_to field. It also sets the correlation_id field if it
> > > > >> needs
> > > > >> to
> > > > >> dispatch multiple responses. The responder (server) must
> > > > >> copy
> > > > >> the
> > > > >> request message's reply_to field to the response message's
> > > > >> address
> > > > >> field
> > > > >> and also copy the correlation_id.
> > > > >>
> > > > >> This API is good for the case where the client wants the
> > > > >> response
> > > > >> to go
> > > > >> to a third party. In this case the reply_to is well
> > > > >> understood
> > > > >> to
> > > > >> be
> > > > >> the address of the third party receiver. However in the
> > > > >> more
> > > > >> common
> > > > >> case where the response is intended to come back to the
> > > > >> client,
> > > > >> it's not
> > > > >> clear at all what to put in the reply_to field.
> > > > >>
> > > > >> I propose that we allow the client to simply say
> > > > >>
> > > > >> request_msg.reply_expected(**cid)
> > > > >>
> > > > >
> > > > > A message has no association with a messenger (until it is
> > > > > put
> > > > > onto
> > > > > the
> > > > > messengers outgoing queue at least). It is not clear how a
> > > > > method
> > > > > on
> > > > > message would therefore be in any better position to
> > > > > determine
> > > > > the
> > > > > reply-address to use.
> > > > >
> > > > > At present, if a message on the outgoing queue has no
> > > > > reply-to
> > > > > set,
> > > > > then
> > > > > the messenger will automatically set the reply-to to be an
> > > > > address
> > > > > corresponding to itself.
> > > > >
> > > > > You can also use the (in my view slightly clumsy) syntax
> > > > > ~/xyz,
> > > > > which will
> > > > > be 'munged' into a form that prepends the messengers name to
> > > > > form
> > > > > a
> > > > > full
> > > > > address. This form I presume would be used if you want to
> > > > > distinguish
> > > > > different categories of replies.
> > > > >
> > > > > I'm not exactly enamoured with this, but it does seem at
> > > > > least
> > > > > to
> > > > > address
> > > > > the key concern expressed here, namely what to put in the
> > > > > reply-to
> > > > > for the
> > > > > simple case. (The answer being, nothing!)
> > > > >
> > > > >
> > > > > (I added the correlation_id argument because it's almost
> > > > > always
> > > > > going to
> > > > >> be needed). Further, the server could use
> > > > >>
> > > > >> reply_msg.in_reply_to(request_**msg)
> > > > >>
> > > > >> which would take care of the addresses and the
> > > > >> correlation_id.
> > > > >> It
> > > > >> also
> > > > >> provides a place to report an error if a request cannot be
> > > > >> replied
> > > > >> to
> > > > >> (absent or invalid reply_to address) rather than force each
> > > > >> server
> > > > >> implementer to code this check.
> > > > >>
> > > > >
> > > > > This is really just a short hand for:
> > > > >
> > > > > response.address = request.reply_to
> > > > > response.correlation_id = request.correlation_id
> > > > >
> > > > > plus some code or exception indicating whether the address
> > > > > has
> > > > > been
> > > > > set. I
> > > > > have no objection to such a utility, though I'm not entirely
> > > > > convinced that
> > > > > a method on message is the best place. It also seems of
> > > > > fairly
> > > > > marginal
> > > > > benefit rather than removing a 'significant barrier to
> > > > > adoption'.
> > > > >
> > > > > Have you seen the JMS solution to simplifying the
> > > > > request-response
> > > > > pattern? There it is done by specifying a distinct class that
> > > > > uses
> > > > > a
> > > > > supplied session and destination and provides a request()
> > > > > method
> > > > > that takes
> > > > > a request message as input and blocks until it can return the
> > > > > response
> > > > > message. (See
> > > > > http://docs.oracle.com/javaee/**1.4/api/javax/jms/**
> > > > > QueueRequestor.html<http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html>).
> > > > > Details of creating temporary response queues etc are hidden
> > > > > behind
> > > > > this. I
> > > > > like that general approach as it layers the functionality
> > > > > cleanly
> > > > > on top of
> > > > > the more flexible base, and would also allow different
> > > > > configurations or
> > > > > conventions to be used.
> > > > >
> > > > > --Gordon.
> > > > >
> > > > >
> > > >
> > >
> > > --
> > > -K
> > >
> >
>
> --
> -K
>
Re: Proton Messenger and the Request/Response pattern
Posted by Ken Giusti <kg...@redhat.com>.
----- Original Message -----
> From: "Michael Goulish" <mg...@redhat.com>
> To: proton@qpid.apache.org
> Sent: Thursday, March 7, 2013 9:48:24 AM
> Subject: Re: Proton Messenger and the Request/Response pattern
>
>
>
> ----- Original Message -----
> > Hi All,
> >
> > I wanted to re-raise Ted's suggestion regarding the need for a
> > better
> > Messenger API for handling reply-to's.
> >
> > I've could've used this recently while designing a "echo" stress
> > test
> > for Messenger. For my client, I have two requirements:
> >
> > 1) I don't want to have to assign a publicly visible address to the
> > client in order to receive replies.
> >
> > 2) I only want a reply for particular messages.
> >
> > I think these two requirements are common enough to merit some
> > Messenger API lovin'.
> >
> > In that context, Ted's API recommendation makes my life as the
> > client
> > writer easier: It doesn't require me to obtain a reply-to value
> > up-front (ie. before calling pn_messenger_put()), and it doesn't
> > cause every message to be sent with a reply-to (which, I believe is
> > the current case - messenger always adds a reply-to if not set).
> >
> > I think there's value in Ted's proposal, but I'd recommend a slight
> > modification to it: rather than mark the message directly, instead
> > indicate to Messenger that a particular message needs its reply-to
> > set correctly prior to transmitting it. For example, we could add
> > a
> > flag to pn_messenger_put():
> >
> > msg = pn_message();
> > pn_message_set_address( msg, "peer's remote address" );
> > .... set message contents, cid, etc ...
> > pn_messenger_put( messenger, msg, PN_SET_REPLY_TO );
> >
>
>
> I think this requires little or no change to the user's mental model,
> which is a _good thing_.
>
> But wouldn't the following be more natural? :
>
> pn_message_set_address( msg, peer_addr, PN_SET_REPLY_TO );
>
> The address setting is done when your code is thinking about
> addresses,
> and put() only puts.
>
>
This would require the message object to contain some extra state, since the reply-to address may not be known at the point the pn_message_set_address() is invoked. That extra state - a flag to tell Messenger to fill in the reply-to - is kinda sorta Messenger specific information that would be piggybacked on the message. I seem to recall the consensus was against that.
/my memory ain't what is once was....
>
>
>
> >
> > Setting the PN_SET_REPLY_TO would cause Messenger to set the
> > reply-to
> > properly prior to sending, or signal an error if unable to do so
> > (reported via a tracker, as usual).
> >
> > This API change would satisfy my two requirements, while keeping
> > the
> > whole address management stuff out of my client.
> >
> > Thoughts?
> >
> > -K
> >
> >
> > ----- Original Message -----
> > > From: "Rafael Schloming" <rh...@alum.mit.edu>
> > > To: proton@qpid.apache.org
> > > Sent: Friday, January 18, 2013 9:13:34 AM
> > > Subject: Re: Proton Messenger and the Request/Response pattern
> > >
> > > Gordon makes some good points. I'd like to add that I think
> > > historically a
> > > big part of the hassle isn't actually necessarily solely the API
> > > but
> > > also
> > > having to configure and manage the intermediary, and I think we
> > > need
> > > to
> > > look there as well if we want to simplify the overall pattern.
> > >
> > > It's also worth remembering that there are significant
> > > performance
> > > benefits
> > > to keeping Message as a pure value object (i.e. a content holder)
> > > that can
> > > be reused many times even across multiple Messengers. Some of the
> > > suggestions really stray from this pattern. If I recall
> > > correctly,
> > > I
> > > think
> > > we said in prior threads that we wanted to maintain the notion of
> > > Message
> > > being a reusable content holder as an architectural invariant.
> > > Perhaps it's
> > > worth re-examining what we could do for request/response while
> > > keeping this
> > > invariant in mind.
> > >
> > > --Rafael
> > >
> > > On Fri, Jan 18, 2013 at 6:24 AM, Gordon Sim <gs...@redhat.com>
> > > wrote:
> > >
> > > > On 01/02/2013 07:14 PM, Ted Ross wrote:
> > > >
> > > >> I'd like to start a discussion on how, from an API
> > > >> perspective,
> > > >> applications can use the request/response pattern. If we get
> > > >> this
> > > >> right, we will remove a significant barrier to adoption of
> > > >> AMQP.
> > > >>
> > > >> Middleware messaging systems typically do a poor job of
> > > >> supporting
> > > >> this
> > > >> pattern. The Qpid APIs are quite lacking in this regard
> > > >> (requester
> > > >> creates and subscribes to a temporary queue with a _unique_
> > > >> name
> > > >> and
> > > >> places this name in the reply-to field).
> > > >>
> > > >> Proton Messenger supports request/reply (see
> > > >> examples/messenger/$LANG/{**client,server}) as follows:
> > > >>
> > > >> The requester (client) has to put _something_ into the request
> > > >> message's
> > > >> reply_to field. It also sets the correlation_id field if it
> > > >> needs
> > > >> to
> > > >> dispatch multiple responses. The responder (server) must copy
> > > >> the
> > > >> request message's reply_to field to the response message's
> > > >> address
> > > >> field
> > > >> and also copy the correlation_id.
> > > >>
> > > >> This API is good for the case where the client wants the
> > > >> response
> > > >> to go
> > > >> to a third party. In this case the reply_to is well
> > > >> understood
> > > >> to
> > > >> be
> > > >> the address of the third party receiver. However in the more
> > > >> common
> > > >> case where the response is intended to come back to the
> > > >> client,
> > > >> it's not
> > > >> clear at all what to put in the reply_to field.
> > > >>
> > > >> I propose that we allow the client to simply say
> > > >>
> > > >> request_msg.reply_expected(**cid)
> > > >>
> > > >
> > > > A message has no association with a messenger (until it is put
> > > > onto
> > > > the
> > > > messengers outgoing queue at least). It is not clear how a
> > > > method
> > > > on
> > > > message would therefore be in any better position to determine
> > > > the
> > > > reply-address to use.
> > > >
> > > > At present, if a message on the outgoing queue has no reply-to
> > > > set,
> > > > then
> > > > the messenger will automatically set the reply-to to be an
> > > > address
> > > > corresponding to itself.
> > > >
> > > > You can also use the (in my view slightly clumsy) syntax ~/xyz,
> > > > which will
> > > > be 'munged' into a form that prepends the messengers name to
> > > > form
> > > > a
> > > > full
> > > > address. This form I presume would be used if you want to
> > > > distinguish
> > > > different categories of replies.
> > > >
> > > > I'm not exactly enamoured with this, but it does seem at least
> > > > to
> > > > address
> > > > the key concern expressed here, namely what to put in the
> > > > reply-to
> > > > for the
> > > > simple case. (The answer being, nothing!)
> > > >
> > > >
> > > > (I added the correlation_id argument because it's almost
> > > > always
> > > > going to
> > > >> be needed). Further, the server could use
> > > >>
> > > >> reply_msg.in_reply_to(request_**msg)
> > > >>
> > > >> which would take care of the addresses and the correlation_id.
> > > >> It
> > > >> also
> > > >> provides a place to report an error if a request cannot be
> > > >> replied
> > > >> to
> > > >> (absent or invalid reply_to address) rather than force each
> > > >> server
> > > >> implementer to code this check.
> > > >>
> > > >
> > > > This is really just a short hand for:
> > > >
> > > > response.address = request.reply_to
> > > > response.correlation_id = request.correlation_id
> > > >
> > > > plus some code or exception indicating whether the address has
> > > > been
> > > > set. I
> > > > have no objection to such a utility, though I'm not entirely
> > > > convinced that
> > > > a method on message is the best place. It also seems of fairly
> > > > marginal
> > > > benefit rather than removing a 'significant barrier to
> > > > adoption'.
> > > >
> > > > Have you seen the JMS solution to simplifying the
> > > > request-response
> > > > pattern? There it is done by specifying a distinct class that
> > > > uses
> > > > a
> > > > supplied session and destination and provides a request()
> > > > method
> > > > that takes
> > > > a request message as input and blocks until it can return the
> > > > response
> > > > message. (See
> > > > http://docs.oracle.com/javaee/**1.4/api/javax/jms/**
> > > > QueueRequestor.html<http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html>).
> > > > Details of creating temporary response queues etc are hidden
> > > > behind
> > > > this. I
> > > > like that general approach as it layers the functionality
> > > > cleanly
> > > > on top of
> > > > the more flexible base, and would also allow different
> > > > configurations or
> > > > conventions to be used.
> > > >
> > > > --Gordon.
> > > >
> > > >
> > >
> >
> > --
> > -K
> >
>
--
-K
Re: Proton Messenger and the Request/Response pattern
Posted by Michael Goulish <mg...@redhat.com>.
----- Original Message -----
> Hi All,
>
> I wanted to re-raise Ted's suggestion regarding the need for a better
> Messenger API for handling reply-to's.
>
> I've could've used this recently while designing a "echo" stress test
> for Messenger. For my client, I have two requirements:
>
> 1) I don't want to have to assign a publicly visible address to the
> client in order to receive replies.
>
> 2) I only want a reply for particular messages.
>
> I think these two requirements are common enough to merit some
> Messenger API lovin'.
>
> In that context, Ted's API recommendation makes my life as the client
> writer easier: It doesn't require me to obtain a reply-to value
> up-front (ie. before calling pn_messenger_put()), and it doesn't
> cause every message to be sent with a reply-to (which, I believe is
> the current case - messenger always adds a reply-to if not set).
>
> I think there's value in Ted's proposal, but I'd recommend a slight
> modification to it: rather than mark the message directly, instead
> indicate to Messenger that a particular message needs its reply-to
> set correctly prior to transmitting it. For example, we could add a
> flag to pn_messenger_put():
>
> msg = pn_message();
> pn_message_set_address( msg, "peer's remote address" );
> .... set message contents, cid, etc ...
> pn_messenger_put( messenger, msg, PN_SET_REPLY_TO );
>
I think this requires little or no change to the user's mental model,
which is a _good thing_.
But wouldn't the following be more natural? :
pn_message_set_address( msg, peer_addr, PN_SET_REPLY_TO );
The address setting is done when your code is thinking about addresses,
and put() only puts.
>
> Setting the PN_SET_REPLY_TO would cause Messenger to set the reply-to
> properly prior to sending, or signal an error if unable to do so
> (reported via a tracker, as usual).
>
> This API change would satisfy my two requirements, while keeping the
> whole address management stuff out of my client.
>
> Thoughts?
>
> -K
>
>
> ----- Original Message -----
> > From: "Rafael Schloming" <rh...@alum.mit.edu>
> > To: proton@qpid.apache.org
> > Sent: Friday, January 18, 2013 9:13:34 AM
> > Subject: Re: Proton Messenger and the Request/Response pattern
> >
> > Gordon makes some good points. I'd like to add that I think
> > historically a
> > big part of the hassle isn't actually necessarily solely the API
> > but
> > also
> > having to configure and manage the intermediary, and I think we
> > need
> > to
> > look there as well if we want to simplify the overall pattern.
> >
> > It's also worth remembering that there are significant performance
> > benefits
> > to keeping Message as a pure value object (i.e. a content holder)
> > that can
> > be reused many times even across multiple Messengers. Some of the
> > suggestions really stray from this pattern. If I recall correctly,
> > I
> > think
> > we said in prior threads that we wanted to maintain the notion of
> > Message
> > being a reusable content holder as an architectural invariant.
> > Perhaps it's
> > worth re-examining what we could do for request/response while
> > keeping this
> > invariant in mind.
> >
> > --Rafael
> >
> > On Fri, Jan 18, 2013 at 6:24 AM, Gordon Sim <gs...@redhat.com>
> > wrote:
> >
> > > On 01/02/2013 07:14 PM, Ted Ross wrote:
> > >
> > >> I'd like to start a discussion on how, from an API perspective,
> > >> applications can use the request/response pattern. If we get
> > >> this
> > >> right, we will remove a significant barrier to adoption of AMQP.
> > >>
> > >> Middleware messaging systems typically do a poor job of
> > >> supporting
> > >> this
> > >> pattern. The Qpid APIs are quite lacking in this regard
> > >> (requester
> > >> creates and subscribes to a temporary queue with a _unique_ name
> > >> and
> > >> places this name in the reply-to field).
> > >>
> > >> Proton Messenger supports request/reply (see
> > >> examples/messenger/$LANG/{**client,server}) as follows:
> > >>
> > >> The requester (client) has to put _something_ into the request
> > >> message's
> > >> reply_to field. It also sets the correlation_id field if it
> > >> needs
> > >> to
> > >> dispatch multiple responses. The responder (server) must copy
> > >> the
> > >> request message's reply_to field to the response message's
> > >> address
> > >> field
> > >> and also copy the correlation_id.
> > >>
> > >> This API is good for the case where the client wants the
> > >> response
> > >> to go
> > >> to a third party. In this case the reply_to is well understood
> > >> to
> > >> be
> > >> the address of the third party receiver. However in the more
> > >> common
> > >> case where the response is intended to come back to the client,
> > >> it's not
> > >> clear at all what to put in the reply_to field.
> > >>
> > >> I propose that we allow the client to simply say
> > >>
> > >> request_msg.reply_expected(**cid)
> > >>
> > >
> > > A message has no association with a messenger (until it is put
> > > onto
> > > the
> > > messengers outgoing queue at least). It is not clear how a method
> > > on
> > > message would therefore be in any better position to determine
> > > the
> > > reply-address to use.
> > >
> > > At present, if a message on the outgoing queue has no reply-to
> > > set,
> > > then
> > > the messenger will automatically set the reply-to to be an
> > > address
> > > corresponding to itself.
> > >
> > > You can also use the (in my view slightly clumsy) syntax ~/xyz,
> > > which will
> > > be 'munged' into a form that prepends the messengers name to form
> > > a
> > > full
> > > address. This form I presume would be used if you want to
> > > distinguish
> > > different categories of replies.
> > >
> > > I'm not exactly enamoured with this, but it does seem at least to
> > > address
> > > the key concern expressed here, namely what to put in the
> > > reply-to
> > > for the
> > > simple case. (The answer being, nothing!)
> > >
> > >
> > > (I added the correlation_id argument because it's almost always
> > > going to
> > >> be needed). Further, the server could use
> > >>
> > >> reply_msg.in_reply_to(request_**msg)
> > >>
> > >> which would take care of the addresses and the correlation_id.
> > >> It
> > >> also
> > >> provides a place to report an error if a request cannot be
> > >> replied
> > >> to
> > >> (absent or invalid reply_to address) rather than force each
> > >> server
> > >> implementer to code this check.
> > >>
> > >
> > > This is really just a short hand for:
> > >
> > > response.address = request.reply_to
> > > response.correlation_id = request.correlation_id
> > >
> > > plus some code or exception indicating whether the address has
> > > been
> > > set. I
> > > have no objection to such a utility, though I'm not entirely
> > > convinced that
> > > a method on message is the best place. It also seems of fairly
> > > marginal
> > > benefit rather than removing a 'significant barrier to adoption'.
> > >
> > > Have you seen the JMS solution to simplifying the
> > > request-response
> > > pattern? There it is done by specifying a distinct class that
> > > uses
> > > a
> > > supplied session and destination and provides a request() method
> > > that takes
> > > a request message as input and blocks until it can return the
> > > response
> > > message. (See
> > > http://docs.oracle.com/javaee/**1.4/api/javax/jms/**
> > > QueueRequestor.html<http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html>).
> > > Details of creating temporary response queues etc are hidden
> > > behind
> > > this. I
> > > like that general approach as it layers the functionality cleanly
> > > on top of
> > > the more flexible base, and would also allow different
> > > configurations or
> > > conventions to be used.
> > >
> > > --Gordon.
> > >
> > >
> >
>
> --
> -K
>
Re: Proton Messenger and the Request/Response pattern
Posted by Ken Giusti <kg...@redhat.com>.
Hi All,
I wanted to re-raise Ted's suggestion regarding the need for a better Messenger API for handling reply-to's.
I've could've used this recently while designing a "echo" stress test for Messenger. For my client, I have two requirements:
1) I don't want to have to assign a publicly visible address to the client in order to receive replies.
2) I only want a reply for particular messages.
I think these two requirements are common enough to merit some Messenger API lovin'.
In that context, Ted's API recommendation makes my life as the client writer easier: It doesn't require me to obtain a reply-to value up-front (ie. before calling pn_messenger_put()), and it doesn't cause every message to be sent with a reply-to (which, I believe is the current case - messenger always adds a reply-to if not set).
I think there's value in Ted's proposal, but I'd recommend a slight modification to it: rather than mark the message directly, instead indicate to Messenger that a particular message needs its reply-to set correctly prior to transmitting it. For example, we could add a flag to pn_messenger_put():
msg = pn_message();
pn_message_set_address( msg, "peer's remote address" );
.... set message contents, cid, etc ...
pn_messenger_put( messenger, msg, PN_SET_REPLY_TO );
Setting the PN_SET_REPLY_TO would cause Messenger to set the reply-to properly prior to sending, or signal an error if unable to do so (reported via a tracker, as usual).
This API change would satisfy my two requirements, while keeping the whole address management stuff out of my client.
Thoughts?
-K
----- Original Message -----
> From: "Rafael Schloming" <rh...@alum.mit.edu>
> To: proton@qpid.apache.org
> Sent: Friday, January 18, 2013 9:13:34 AM
> Subject: Re: Proton Messenger and the Request/Response pattern
>
> Gordon makes some good points. I'd like to add that I think
> historically a
> big part of the hassle isn't actually necessarily solely the API but
> also
> having to configure and manage the intermediary, and I think we need
> to
> look there as well if we want to simplify the overall pattern.
>
> It's also worth remembering that there are significant performance
> benefits
> to keeping Message as a pure value object (i.e. a content holder)
> that can
> be reused many times even across multiple Messengers. Some of the
> suggestions really stray from this pattern. If I recall correctly, I
> think
> we said in prior threads that we wanted to maintain the notion of
> Message
> being a reusable content holder as an architectural invariant.
> Perhaps it's
> worth re-examining what we could do for request/response while
> keeping this
> invariant in mind.
>
> --Rafael
>
> On Fri, Jan 18, 2013 at 6:24 AM, Gordon Sim <gs...@redhat.com> wrote:
>
> > On 01/02/2013 07:14 PM, Ted Ross wrote:
> >
> >> I'd like to start a discussion on how, from an API perspective,
> >> applications can use the request/response pattern. If we get this
> >> right, we will remove a significant barrier to adoption of AMQP.
> >>
> >> Middleware messaging systems typically do a poor job of supporting
> >> this
> >> pattern. The Qpid APIs are quite lacking in this regard
> >> (requester
> >> creates and subscribes to a temporary queue with a _unique_ name
> >> and
> >> places this name in the reply-to field).
> >>
> >> Proton Messenger supports request/reply (see
> >> examples/messenger/$LANG/{**client,server}) as follows:
> >>
> >> The requester (client) has to put _something_ into the request
> >> message's
> >> reply_to field. It also sets the correlation_id field if it needs
> >> to
> >> dispatch multiple responses. The responder (server) must copy the
> >> request message's reply_to field to the response message's address
> >> field
> >> and also copy the correlation_id.
> >>
> >> This API is good for the case where the client wants the response
> >> to go
> >> to a third party. In this case the reply_to is well understood to
> >> be
> >> the address of the third party receiver. However in the more
> >> common
> >> case where the response is intended to come back to the client,
> >> it's not
> >> clear at all what to put in the reply_to field.
> >>
> >> I propose that we allow the client to simply say
> >>
> >> request_msg.reply_expected(**cid)
> >>
> >
> > A message has no association with a messenger (until it is put onto
> > the
> > messengers outgoing queue at least). It is not clear how a method
> > on
> > message would therefore be in any better position to determine the
> > reply-address to use.
> >
> > At present, if a message on the outgoing queue has no reply-to set,
> > then
> > the messenger will automatically set the reply-to to be an address
> > corresponding to itself.
> >
> > You can also use the (in my view slightly clumsy) syntax ~/xyz,
> > which will
> > be 'munged' into a form that prepends the messengers name to form a
> > full
> > address. This form I presume would be used if you want to
> > distinguish
> > different categories of replies.
> >
> > I'm not exactly enamoured with this, but it does seem at least to
> > address
> > the key concern expressed here, namely what to put in the reply-to
> > for the
> > simple case. (The answer being, nothing!)
> >
> >
> > (I added the correlation_id argument because it's almost always
> > going to
> >> be needed). Further, the server could use
> >>
> >> reply_msg.in_reply_to(request_**msg)
> >>
> >> which would take care of the addresses and the correlation_id. It
> >> also
> >> provides a place to report an error if a request cannot be replied
> >> to
> >> (absent or invalid reply_to address) rather than force each server
> >> implementer to code this check.
> >>
> >
> > This is really just a short hand for:
> >
> > response.address = request.reply_to
> > response.correlation_id = request.correlation_id
> >
> > plus some code or exception indicating whether the address has been
> > set. I
> > have no objection to such a utility, though I'm not entirely
> > convinced that
> > a method on message is the best place. It also seems of fairly
> > marginal
> > benefit rather than removing a 'significant barrier to adoption'.
> >
> > Have you seen the JMS solution to simplifying the request-response
> > pattern? There it is done by specifying a distinct class that uses
> > a
> > supplied session and destination and provides a request() method
> > that takes
> > a request message as input and blocks until it can return the
> > response
> > message. (See http://docs.oracle.com/javaee/**1.4/api/javax/jms/**
> > QueueRequestor.html<http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html>).
> > Details of creating temporary response queues etc are hidden behind
> > this. I
> > like that general approach as it layers the functionality cleanly
> > on top of
> > the more flexible base, and would also allow different
> > configurations or
> > conventions to be used.
> >
> > --Gordon.
> >
> >
>
--
-K
Re: Proton Messenger and the Request/Response pattern
Posted by Rafael Schloming <rh...@alum.mit.edu>.
Gordon makes some good points. I'd like to add that I think historically a
big part of the hassle isn't actually necessarily solely the API but also
having to configure and manage the intermediary, and I think we need to
look there as well if we want to simplify the overall pattern.
It's also worth remembering that there are significant performance benefits
to keeping Message as a pure value object (i.e. a content holder) that can
be reused many times even across multiple Messengers. Some of the
suggestions really stray from this pattern. If I recall correctly, I think
we said in prior threads that we wanted to maintain the notion of Message
being a reusable content holder as an architectural invariant. Perhaps it's
worth re-examining what we could do for request/response while keeping this
invariant in mind.
--Rafael
On Fri, Jan 18, 2013 at 6:24 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 01/02/2013 07:14 PM, Ted Ross wrote:
>
>> I'd like to start a discussion on how, from an API perspective,
>> applications can use the request/response pattern. If we get this
>> right, we will remove a significant barrier to adoption of AMQP.
>>
>> Middleware messaging systems typically do a poor job of supporting this
>> pattern. The Qpid APIs are quite lacking in this regard (requester
>> creates and subscribes to a temporary queue with a _unique_ name and
>> places this name in the reply-to field).
>>
>> Proton Messenger supports request/reply (see
>> examples/messenger/$LANG/{**client,server}) as follows:
>>
>> The requester (client) has to put _something_ into the request message's
>> reply_to field. It also sets the correlation_id field if it needs to
>> dispatch multiple responses. The responder (server) must copy the
>> request message's reply_to field to the response message's address field
>> and also copy the correlation_id.
>>
>> This API is good for the case where the client wants the response to go
>> to a third party. In this case the reply_to is well understood to be
>> the address of the third party receiver. However in the more common
>> case where the response is intended to come back to the client, it's not
>> clear at all what to put in the reply_to field.
>>
>> I propose that we allow the client to simply say
>>
>> request_msg.reply_expected(**cid)
>>
>
> A message has no association with a messenger (until it is put onto the
> messengers outgoing queue at least). It is not clear how a method on
> message would therefore be in any better position to determine the
> reply-address to use.
>
> At present, if a message on the outgoing queue has no reply-to set, then
> the messenger will automatically set the reply-to to be an address
> corresponding to itself.
>
> You can also use the (in my view slightly clumsy) syntax ~/xyz, which will
> be 'munged' into a form that prepends the messengers name to form a full
> address. This form I presume would be used if you want to distinguish
> different categories of replies.
>
> I'm not exactly enamoured with this, but it does seem at least to address
> the key concern expressed here, namely what to put in the reply-to for the
> simple case. (The answer being, nothing!)
>
>
> (I added the correlation_id argument because it's almost always going to
>> be needed). Further, the server could use
>>
>> reply_msg.in_reply_to(request_**msg)
>>
>> which would take care of the addresses and the correlation_id. It also
>> provides a place to report an error if a request cannot be replied to
>> (absent or invalid reply_to address) rather than force each server
>> implementer to code this check.
>>
>
> This is really just a short hand for:
>
> response.address = request.reply_to
> response.correlation_id = request.correlation_id
>
> plus some code or exception indicating whether the address has been set. I
> have no objection to such a utility, though I'm not entirely convinced that
> a method on message is the best place. It also seems of fairly marginal
> benefit rather than removing a 'significant barrier to adoption'.
>
> Have you seen the JMS solution to simplifying the request-response
> pattern? There it is done by specifying a distinct class that uses a
> supplied session and destination and provides a request() method that takes
> a request message as input and blocks until it can return the response
> message. (See http://docs.oracle.com/javaee/**1.4/api/javax/jms/**
> QueueRequestor.html<http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html>).
> Details of creating temporary response queues etc are hidden behind this. I
> like that general approach as it layers the functionality cleanly on top of
> the more flexible base, and would also allow different configurations or
> conventions to be used.
>
> --Gordon.
>
>
Re: Proton Messenger and the Request/Response pattern
Posted by Gordon Sim <gs...@redhat.com>.
On 01/02/2013 07:14 PM, Ted Ross wrote:
> I'd like to start a discussion on how, from an API perspective,
> applications can use the request/response pattern. If we get this
> right, we will remove a significant barrier to adoption of AMQP.
>
> Middleware messaging systems typically do a poor job of supporting this
> pattern. The Qpid APIs are quite lacking in this regard (requester
> creates and subscribes to a temporary queue with a _unique_ name and
> places this name in the reply-to field).
>
> Proton Messenger supports request/reply (see
> examples/messenger/$LANG/{client,server}) as follows:
>
> The requester (client) has to put _something_ into the request message's
> reply_to field. It also sets the correlation_id field if it needs to
> dispatch multiple responses. The responder (server) must copy the
> request message's reply_to field to the response message's address field
> and also copy the correlation_id.
>
> This API is good for the case where the client wants the response to go
> to a third party. In this case the reply_to is well understood to be
> the address of the third party receiver. However in the more common
> case where the response is intended to come back to the client, it's not
> clear at all what to put in the reply_to field.
>
> I propose that we allow the client to simply say
>
> request_msg.reply_expected(cid)
A message has no association with a messenger (until it is put onto the
messengers outgoing queue at least). It is not clear how a method on
message would therefore be in any better position to determine the
reply-address to use.
At present, if a message on the outgoing queue has no reply-to set, then
the messenger will automatically set the reply-to to be an address
corresponding to itself.
You can also use the (in my view slightly clumsy) syntax ~/xyz, which
will be 'munged' into a form that prepends the messengers name to form a
full address. This form I presume would be used if you want to
distinguish different categories of replies.
I'm not exactly enamoured with this, but it does seem at least to
address the key concern expressed here, namely what to put in the
reply-to for the simple case. (The answer being, nothing!)
> (I added the correlation_id argument because it's almost always going to
> be needed). Further, the server could use
>
> reply_msg.in_reply_to(request_msg)
>
> which would take care of the addresses and the correlation_id. It also
> provides a place to report an error if a request cannot be replied to
> (absent or invalid reply_to address) rather than force each server
> implementer to code this check.
This is really just a short hand for:
response.address = request.reply_to
response.correlation_id = request.correlation_id
plus some code or exception indicating whether the address has been set.
I have no objection to such a utility, though I'm not entirely convinced
that a method on message is the best place. It also seems of fairly
marginal benefit rather than removing a 'significant barrier to adoption'.
Have you seen the JMS solution to simplifying the request-response
pattern? There it is done by specifying a distinct class that uses a
supplied session and destination and provides a request() method that
takes a request message as input and blocks until it can return the
response message. (See
http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html).
Details of creating temporary response queues etc are hidden behind
this. I like that general approach as it layers the functionality
cleanly on top of the more flexible base, and would also allow different
configurations or conventions to be used.
--Gordon.
Re: Proton Messenger and the Request/Response pattern
Posted by Gordon Sim <gs...@redhat.com>.
On 01/02/2013 07:14 PM, Ted Ross wrote:
> I'd like to start a discussion on how, from an API perspective,
> applications can use the request/response pattern. If we get this
> right, we will remove a significant barrier to adoption of AMQP.
>
> Middleware messaging systems typically do a poor job of supporting this
> pattern. The Qpid APIs are quite lacking in this regard (requester
> creates and subscribes to a temporary queue with a _unique_ name and
> places this name in the reply-to field).
>
> Proton Messenger supports request/reply (see
> examples/messenger/$LANG/{client,server}) as follows:
>
> The requester (client) has to put _something_ into the request message's
> reply_to field. It also sets the correlation_id field if it needs to
> dispatch multiple responses. The responder (server) must copy the
> request message's reply_to field to the response message's address field
> and also copy the correlation_id.
>
> This API is good for the case where the client wants the response to go
> to a third party. In this case the reply_to is well understood to be
> the address of the third party receiver. However in the more common
> case where the response is intended to come back to the client, it's not
> clear at all what to put in the reply_to field.
>
> I propose that we allow the client to simply say
>
> request_msg.reply_expected(cid)
A message has no association with a messenger (until it is put onto the
messengers outgoing queue at least). It is not clear how a method on
message would therefore be in any better position to determine the
reply-address to use.
At present, if a message on the outgoing queue has no reply-to set, then
the messenger will automatically set the reply-to to be an address
corresponding to itself.
You can also use the (in my view slightly clumsy) syntax ~/xyz, which
will be 'munged' into a form that prepends the messengers name to form a
full address. This form I presume would be used if you want to
distinguish different categories of replies.
I'm not exactly enamoured with this, but it does seem at least to
address the key concern expressed here, namely what to put in the
reply-to for the simple case. (The answer being, nothing!)
> (I added the correlation_id argument because it's almost always going to
> be needed). Further, the server could use
>
> reply_msg.in_reply_to(request_msg)
>
> which would take care of the addresses and the correlation_id. It also
> provides a place to report an error if a request cannot be replied to
> (absent or invalid reply_to address) rather than force each server
> implementer to code this check.
This is really just a short hand for:
response.address = request.reply_to
response.correlation_id = request.correlation_id
plus some code or exception indicating whether the address has been set.
I have no objection to such a utility, though I'm not entirely convinced
that a method on message is the best place. It also seems of fairly
marginal benefit rather than removing a 'significant barrier to adoption'.
Have you seen the JMS solution to simplifying the request-response
pattern? There it is done by specifying a distinct class that uses a
supplied session and destination and provides a request() method that
takes a request message as input and blocks until it can return the
response message. (See
http://docs.oracle.com/javaee/1.4/api/javax/jms/QueueRequestor.html).
Details of creating temporary response queues etc are hidden behind
this. I like that general approach as it layers the functionality
cleanly on top of the more flexible base, and would also allow different
configurations or conventions to be used.
--Gordon.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: what *exactly* is Proton? (was Re: Proton Messenger and the Request/Response
pattern)
Posted by Gordon Sim <gs...@redhat.com>.
On 01/20/2013 05:50 PM, Fraser Adams wrote:
> It's starting to become somewhat clearer how proton fits in, though some
> of your comments still leave me with questions about how *ready* proton
> is yet - it looks though the C++ stuff is more advanced than the Java
> stuff - is that accurate?
Yes, but the gap is closing fast.
> Is there a mechanism to get a JMS API around
> the Java proton messenger library or would Java AMQP clients still have
> to talk with the underlying proton API?
There is already an AMQP 1.0 implementation of JMS in Qpid. It predates
the proton work but the plan is to replace it with something that uses
proton underneath at some point (more to ease future maintenance).
> I sort of see what you mean by your comment "Ultimately the vision
> behind AMQP is all about choice" perhaps that's fine but I think there
> needs to be some good guidance, patterns if you will for different
> use-cases. As an example qpid::client and qpid::messaging also offered a
> choice and you and I both know that that's now a bit of a slow motion
> car crash. Without strong guidance I think we'll not so much have
> "choice" as chaos.
I think with qpid::client it was very much a case of one being
deprecated in favour of the other. In hindsight we could have been a lot
clearer about signalling that.
With messenger v. qpid::messaging it is more a question of two different
styles. There may be use cases that firmly imply one or the other, but I
think that in large part it may come down to preference for one style or
another.
> I think that there's scope for confusion in python land too - you
> mention the deprecated library, so I *think* that this means that the
> favoured one is a SWIG wrapper round qpid::messaging - is that correct.
Thats not actually what I meant. In python there is the equivalent of
the old qpid::client API expcept (it's not called that and quite
different in actual methods/classes).
However you do raise a good point. At present there is no AMQP 1.0
implementation of the python qpid.messaging library. There *is* the SWIG
wrapped equivalent from the c++ side (and this can be made to support
AMQP 1.0) which is now pretty close to the native python API in behaviour.
One interesting question is how to proceed there. Is there for example
sufficient interest and demand for (or interest in implementing) a pure
python, AMQP 1.0 implementation of that library or is the SWIG version a
good enough option? I don't know the answer, but collectively this list
is the place to answer it I think.
> I suspect that there are plenty of people confused by which to use and I
> think there's definitely a need for good documentation and examples.
Agreed. People need to be able to quickly get a sense for what a given
API can and can't do, and what it is like to code with.
> Some other concerns/thoughts. You mentioned "There has also been work to
> add 1.0 support to ActiveMQ, and that also uses the (java version) of
> the proton protocol engine." on one hand that's great from an
> interoperability perspective but again has the potential to cause huge
> amounts of confusion!! There needs to be some investment in "branding"
> IMHO for example how would one make a choice between ActiveMQ versus say
> the Qpid Java broker?
Again, a very good question.
> These sorts of choices are very real indeed. In a
> corporate sense one often has to do make choices over messaging systems
> and it's hard enough as it is!!!!! I've got the T-shirt there when
> looking at Qpid versus Tibco and justifying choices to non-techies is
> nobody's idea of fun :-( there needs to be some strong differentiators
> for the "hard of thinking".
He he, I do like that phrase! I do totally agree with your point on
clear differentiation. I think the success of AMQP 1.0 and the upcoming
work on ActiveMQ does change the picture there somewhat and we need to
discuss that as a community.
> You also mention "qpid::messaging has an optional dependency on the c
> version of proton". Does this imply that proton is written in C (as
> opposed to C++)? Just curious on that one.
Yes, it is pure C.
> And don't get me started on QMF. There's two protocols for starters :-)
> then there's a Java implementation by yours truly that is still stuck in
> a Jira, a C++ API that bears no relationship to the published (but still
> draft) QMF2 API, two python implementations (one for each protocol). I
> don't know if things have changed lately but the python tools used to
> use the QMF1 API for a long time. Around this time last year there was
> some talk of a QMF2 sub-project given that it has application beyond
> Qpid given that it's a fairly good request/response API with service
> discovery, but discussion on that (certainly in the Qpid Users mailing
> list) has fizzled out.
>
> I really like QMF and as you can probably see I've done a fair bit with
> it, but my motivation is waning a bit given that I'm not at all clear on
> the status of it all.
First off, thanks and congratulations on what looks like a very nice new
component (I haven't had a chance to unbundle and play with it yet but
will do that asap)! It looks very much like something that should be
included somewhere in the main svn (under extras seems like a good home
initially).
Second, I share your love-hate relationship with QMF! I do love the
simplicity of the overall approach (simply sending & receiving
messages); I'm a lot more reticent about particular APIs though.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: what *exactly* is Proton? (was Re: Proton Messenger and the Request/Response
pattern)
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Thanks for this response Gordon and also (belated) thanks to William.
I'd have said thanks sooner but I've been manically trying to get my
QMF2/GUI stuff tested and released (you might come across my posts on
some of the management remote methods).
It's starting to become somewhat clearer how proton fits in, though some
of your comments still leave me with questions about how *ready* proton
is yet - it looks though the C++ stuff is more advanced than the Java
stuff - is that accurate? Is there a mechanism to get a JMS API around
the Java proton messenger library or would Java AMQP clients still have
to talk with the underlying proton API?
I sort of see what you mean by your comment "Ultimately the vision
behind AMQP is all about choice" perhaps that's fine but I think there
needs to be some good guidance, patterns if you will for different
use-cases. As an example qpid::client and qpid::messaging also offered a
choice and you and I both know that that's now a bit of a slow motion
car crash. Without strong guidance I think we'll not so much have
"choice" as chaos.
I think that there's scope for confusion in python land too - you
mention the deprecated library, so I *think* that this means that the
favoured one is a SWIG wrapper round qpid::messaging - is that correct.
I suspect that there are plenty of people confused by which to use and I
think there's definitely a need for good documentation and examples.
Some other concerns/thoughts. You mentioned "There has also been work to
add 1.0 support to ActiveMQ, and that also uses the (java version) of
the proton protocol engine." on one hand that's great from an
interoperability perspective but again has the potential to cause huge
amounts of confusion!! There needs to be some investment in "branding"
IMHO for example how would one make a choice between ActiveMQ versus say
the Qpid Java broker? These sorts of choices are very real indeed. In a
corporate sense one often has to do make choices over messaging systems
and it's hard enough as it is!!!!! I've got the T-shirt there when
looking at Qpid versus Tibco and justifying choices to non-techies is
nobody's idea of fun :-( there needs to be some strong differentiators
for the "hard of thinking".
You also mention "qpid::messaging has an optional dependency on the c
version of proton". Does this imply that proton is written in C (as
opposed to C++)? Just curious on that one.
And don't get me started on QMF. There's two protocols for starters :-)
then there's a Java implementation by yours truly that is still stuck in
a Jira, a C++ API that bears no relationship to the published (but still
draft) QMF2 API, two python implementations (one for each protocol). I
don't know if things have changed lately but the python tools used to
use the QMF1 API for a long time. Around this time last year there was
some talk of a QMF2 sub-project given that it has application beyond
Qpid given that it's a fairly good request/response API with service
discovery, but discussion on that (certainly in the Qpid Users mailing
list) has fizzled out.
I really like QMF and as you can probably see I've done a fair bit with
it, but my motivation is waning a bit given that I'm not at all clear on
the status of it all.
So QMF rant over :-) but hopefully it ties in with what I've also been
saying about proton.
Thanks again for the update!!
Cheers,
Frase
On 18/01/13 16:51, Gordon Sim wrote:
> On 01/04/2013 10:07 AM, Fraser Adams wrote:
>> I really haven't seen an awful lot of publicity around proton and how it
>> relates to qpid so I think it'd be really good to get the documentation
>> around the relationships properly sorted to avoid a whole world of
>> confusion.
>
> You are quite right. Communication has been very poor and we need to
> change that.
>
> On 01/03/2013 09:14 AM, Fraser Adams wrote:
>> BTW what *exactly* is Proton, I've seen a few references to it but
>> nothing is especially clear to me, is Proton the name for the AMQP v1.0
>> version of Qpid? How does it differ from say Qpid v0.20? Will Proton end
>> up being Qpid v1.0?
>
> Let me try and lay out my own understanding. Forgive me if it seems a
> little long-winded, it isn't really that different from what William
> described...
>
> Qpid is an openly governed community that seeks to aid adoption of
> AMQP through collaborative development of various pieces of open
> sourced software. Historically that has included two different
> brokers, various client libraries and management tools. Although the
> different parts should combine well, not every user will want or need
> every component. Given the purpose of AMQP is to enable
> interoperability and choice, it is important that individual
> components from Qpid can be combined with other software and systems
> in a flexible manner.
>
> Proton is a new component under the Qpid umbrella - a set of
> components is perhaps more accurate as there is both a java and c
> version and some swig wrappers. It is an AMQP 1.0 toolkit.
>
> Part of this is a 'protocol engine'; an implementation of the rules
> specified by AMQP 1.0 in a manner that is independent of any threading
> model or IO mechanism. This makes it easier to integrate in diverse
> contexts including existing brokers and messaging clients and not
> limited to just those within Qpid.
>
> One of the uses has been to add AMQP 1.0 support to existing Qpid
> components, such as the work I have been doing to add 1.0 support to
> the qpid::messaging API in c++ and to the qpidd (i.e. c++) broker.
> There has also been work to add 1.0 support to ActiveMQ, and that also
> uses the (java version) of the proton protocol engine.
>
> Being independent of any IO framework is important for one set of use
> cases, but it does mean that the protocol engine isn't really usable
> 'by itself'. It was recognised that adoption of 1.0 would be aided by
> a simple, self contained library allowing sending and receiving of
> messages.
>
> To that end the messenger API was added to the proton toolkit. This
> ties network IO into the protocol engine, exposing a simple
> abstraction of an input and output queue for messages, each message
> containing an address to which it is to be delivered. It manages
> connections, session and links under the covers. At present it is a
> blocking API though support for non-blocking use is likely to evolve
> as well.
>
> AMQP 1.0 is fully symmetric (i.e a peer protocol) as opposed to its
> asymmetric (i.e. client-server) predecessors. This means it can be
> used with or without intermediaries such as brokers. The messenger API
> allows an application to listen for incoming connections as well as
> initiating its own outgoing connections. It can therefore be used for
> simple, direct communication as well as communication through
> intermediaries. (You can also for example use a messenger based
> application to accept connections from a qpid::messaging or even a JMS
> client).
>
> Proton is on its own release cycle at present, reflecting the early
> stage in its lifecycle. So far all other Qpid components have been on
> the same release cycle. So the 0.20 release includes qpidd,
> qpid-server (i.e. the qpid java broker), a JMS client, the c++
> qpid::messaging library (and the deprecated qpid::client library), the
> python qpid.messaging library (and older deprecated libraries), QMF
> tools, a JMX console etc etc.
>
> The 0.3 release of proton on the other hand contains c and java
> libraries offering the protocol engine and messenger APIs (the java
> version of the messenger API has some outstanding issues still) as
> well as wrappers for python, perl and ruby. The 0.20 release of qpidd
> and qpid::messaging has an optional dependency on the c version of
> proton, which if available enables 1.0 support in those components.
> The subsequent release of those two components will contain enhanced
> support.
>
> So, qpid::messaging isn't going away. A key goal for that API was to
> allow smooth transition to AMQP 1.0 and that is still something I am
> working towards (0.20 represents the first step). Support for JMS
> likewise will continue. However the messenger API is in my opinion a
> novel approach that is may also be interesting for many cases.
> Ultimately I think the different APIs offer different styles. While
> there may be certain things that can't be done in one or other, often
> it may come down to personal preference.
>
> Ultimately the vision behind AMQP is all about choice; enabling the
> emergence of a rich palette of interoperable pieces from which diverse
> solutions can be flexibly composed. AMQP 1.0 in my opinion expands the
> opportunities here and rightly Qpid is innovating and evolving to
> exploit those.
>
> These are exciting times! We do however need more open communication
> (and better documentation ;-).
>
> --Gordon.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
what *exactly* is Proton? (was Re: Proton Messenger and the Request/Response
pattern)
Posted by Gordon Sim <gs...@redhat.com>.
On 01/04/2013 10:07 AM, Fraser Adams wrote:
> I really haven't seen an awful lot of publicity around proton and how it
> relates to qpid so I think it'd be really good to get the documentation
> around the relationships properly sorted to avoid a whole world of
> confusion.
You are quite right. Communication has been very poor and we need to
change that.
On 01/03/2013 09:14 AM, Fraser Adams wrote:
> BTW what *exactly* is Proton, I've seen a few references to it but
> nothing is especially clear to me, is Proton the name for the AMQP v1.0
> version of Qpid? How does it differ from say Qpid v0.20? Will Proton end
> up being Qpid v1.0?
Let me try and lay out my own understanding. Forgive me if it seems a
little long-winded, it isn't really that different from what William
described...
Qpid is an openly governed community that seeks to aid adoption of AMQP
through collaborative development of various pieces of open sourced
software. Historically that has included two different brokers, various
client libraries and management tools. Although the different parts
should combine well, not every user will want or need every component.
Given the purpose of AMQP is to enable interoperability and choice, it
is important that individual components from Qpid can be combined with
other software and systems in a flexible manner.
Proton is a new component under the Qpid umbrella - a set of components
is perhaps more accurate as there is both a java and c version and some
swig wrappers. It is an AMQP 1.0 toolkit.
Part of this is a 'protocol engine'; an implementation of the rules
specified by AMQP 1.0 in a manner that is independent of any threading
model or IO mechanism. This makes it easier to integrate in diverse
contexts including existing brokers and messaging clients and not
limited to just those within Qpid.
One of the uses has been to add AMQP 1.0 support to existing Qpid
components, such as the work I have been doing to add 1.0 support to the
qpid::messaging API in c++ and to the qpidd (i.e. c++) broker. There has
also been work to add 1.0 support to ActiveMQ, and that also uses the
(java version) of the proton protocol engine.
Being independent of any IO framework is important for one set of use
cases, but it does mean that the protocol engine isn't really usable 'by
itself'. It was recognised that adoption of 1.0 would be aided by a
simple, self contained library allowing sending and receiving of messages.
To that end the messenger API was added to the proton toolkit. This ties
network IO into the protocol engine, exposing a simple abstraction of an
input and output queue for messages, each message containing an address
to which it is to be delivered. It manages connections, session and
links under the covers. At present it is a blocking API though support
for non-blocking use is likely to evolve as well.
AMQP 1.0 is fully symmetric (i.e a peer protocol) as opposed to its
asymmetric (i.e. client-server) predecessors. This means it can be used
with or without intermediaries such as brokers. The messenger API allows
an application to listen for incoming connections as well as initiating
its own outgoing connections. It can therefore be used for simple,
direct communication as well as communication through intermediaries.
(You can also for example use a messenger based application to accept
connections from a qpid::messaging or even a JMS client).
Proton is on its own release cycle at present, reflecting the early
stage in its lifecycle. So far all other Qpid components have been on
the same release cycle. So the 0.20 release includes qpidd, qpid-server
(i.e. the qpid java broker), a JMS client, the c++ qpid::messaging
library (and the deprecated qpid::client library), the python
qpid.messaging library (and older deprecated libraries), QMF tools, a
JMX console etc etc.
The 0.3 release of proton on the other hand contains c and java
libraries offering the protocol engine and messenger APIs (the java
version of the messenger API has some outstanding issues still) as well
as wrappers for python, perl and ruby. The 0.20 release of qpidd and
qpid::messaging has an optional dependency on the c version of proton,
which if available enables 1.0 support in those components. The
subsequent release of those two components will contain enhanced support.
So, qpid::messaging isn't going away. A key goal for that API was to
allow smooth transition to AMQP 1.0 and that is still something I am
working towards (0.20 represents the first step). Support for JMS
likewise will continue. However the messenger API is in my opinion a
novel approach that is may also be interesting for many cases.
Ultimately I think the different APIs offer different styles. While
there may be certain things that can't be done in one or other, often it
may come down to personal preference.
Ultimately the vision behind AMQP is all about choice; enabling the
emergence of a rich palette of interoperable pieces from which diverse
solutions can be flexibly composed. AMQP 1.0 in my opinion expands the
opportunities here and rightly Qpid is innovating and evolving to
exploit those.
These are exciting times! We do however need more open communication
(and better documentation ;-).
--Gordon.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton Messenger and the Request/Response pattern
Posted by William Henry <wh...@redhat.com>.
Hi Frase and others.
It is regrettable that there is so much confusion. I understand where you're coming from. Indeed I think I have had similar experiences with some of the changes that you have had and are experiencing.
Before addressing some specific API issues I think it's first worth explaining Qpid Proton. It looks like you have a handle on it now but I'll try to explain again as I understand it.
When looking at implementing AMQP 1.0 it was decided that rather than try to implement it inside the much evolved Qpid, especially qpidd, it would be best to implement AMQP 1.0 as a library that was plugable into other artifacts like qpidd (and for example ActiveMQ). Let's create a pure AMQP 1.0 implementation that can be consumed by others. It was decided to build two libraries, one in C and one in Java. The Java can then be consumed by JMS implementation such as Qpid or ActiveMQ or others. The C implementation can be the bases for other non-Java implmentations, e.g. Qpid's C++. With SWIG we could generate other language bindings on the base API (called the Engine).
As it turns out there was a second API build on top of the core Engine API. This Messenger API provides a wrapper on top of the Engine to provide a very simple (yet very powerful) API for develops that care little about complicated use cases but instead want to just publish and subscribe for messages as simple as possible. This is a really cool for things like Python and Ruby etc. But I've actually used the C based Messenger API to build bridge for the OpenMAMA project. So far it seems to be working fine - but I can always go back to Engine if I find I need some more complex semantics and message flows.
So Ted's email was, as the subject said, in reference to the Messenger API. It is not to do with the JMS API.
Many, or perhaps I should say several, of us believe that AMQP is not limited to traditional MOM like deployments. We believe, in the near term, that AMQP will be used mostly for MOM. However in the future we see that AMQP could, if we don't screw it up, become a popular protocol for building internet applications that require more asynch messaging patterns. AMQP can help app developers avoid some of the unnatural gymnastics that they currently have to perform when using HTTP.
For that reason we think the world won't all be about JMS - there will still be a lot of it and, good news, it will interoperate with everything else. Messenger allows app develops access the power of messaging and in particular AMQP without having to learn much. Having said that if they want to dive in more they will find many common patterns and APIs to support those patterns.
So Ted was looking at the current Messenger API that is part of Qpid Proton and saying, hey for people using this API wouldn't it be nice if ... Remember, people using the Messenger API are not JMS developers.
Now the community does need to do a much better job of making these points clearer. The websites are, as you point out, not well linked or explained.
So here is some more explanation at a high level:
qpid::client the older API that exposed AMQP 0-10 artifacts like queues and exchanges
qpid::messaging the newer API that is more JMS like (even for C++) that recognized that AMQP 0-10 artifacts were not going to be part of the AMQP 1.0 spec.
proton-c and proton-j part of qpid/proton qpid.apache.org/proton. These are libraries that implement the AMQP 1.0 specification. They are intended to be consumed by clients and brokers and other implementation that require AMQP 1.0 support. The project is slightly more than the libraries in that it provides some tools for developers to help them consume AMQP 1.0 including Python, Ruby, Perl language bindings.
Proton Messenger API - a high level, simple but powerful wrapper on the core proton API to provide easy access to AMQP messaging.
JMS implementations like Qpid proper and ActiveMQ (for example) can consume Proton. The C++ implementation in Qpid, including the broker, is planning on (has already been done I think) integrating Proton for AMQP 1.0 support.
I know this doesn't answer all the questions but is this helpful? And others, did I get any of this wrong? I'm sure I've left bits out but is what I said accurate enough?
William
----- Original Message -----
> Hi Mary,
> Thanks for the response, unfortunately I must be a bit thick though,
> cause I'm still not totally clear.
>
> I did take a look at the proton web site a little while back, looking
> at
> the link you attached it looks pretty much the same as the last time
> I
> looked a month or so back. But as I say the relationship between qpid
> and proton is far from clear (to me at least....)
>
> I've just loaded up the main qpid website http://qpid.apache.org/ and
> there's no obvious linkage between that and proton so that's not too
> helpful. Also not helping with my confusion is that there has been
> quite
> a bit of traffic on the main qpid mailing list relating to work that
> Gordon Sim has been doing on qpid 0.2 there seem to be quite a few
> new
> features in this including replacing the flow-to-disk paging with
> something better, but Gordon has also been talking about adding AMQP
> v
> 1.0 functionality to qpid 0.2.
>
> So is the AMQP stuff that Gordon is talking about just about
> combining
> in the proton library? Will the web pages for the qpid 0.2 release
> make
> all this clear.
>
> I have to say that the proton work hasn't been especially well
> publicised, I only stumbled across it by accident a month or so back
> by
> noticing a reference to it in the mailing list so did some digging
> and
> eventually found the web page - and I look at the qpid mailing list
> pretty much daily.
>
> From your description below and from the web page it looks like
> proton
> is primarily a library so you'd use it in combination with something
> like qpid, but is the idea that it could also be used "brokerless" so
> I'm thinking something analogous to say zeromq where say one might
> want
> an extremely lightweight messaging IPC mechanism between processes on
> the same host for example without the overhead of a full message
> broker
> process????
>
> I noticed the proton API documentation - it looks very python centric
> at
> the moment, your maul below also suggests C++ but what's the state of
> Java support at the moment - looks like there's a proton-j download,
> how
> does that relate to JMS and the qpid Java client libraries??
>
> The API stuff also makes me a bit nervous - relating on to the
> original
> thread spawning this question. The is already *way too much*
> confusion
> at the moment relating to qpid::messaging versus qpid::client, more
> or
> less once a week someone posts to the main qpid mailing list about
> that
> so I think it's really important that the relationship between qpid
> and
> proton is spelled out clearly in a very prominent place!! It sounds
> like
> for the most part that the proton APIs are lower level APIs so that
> for
> example a client would use qpid::messaging (or JMS) and that would
> delegate in the client runtime to the proton APIs, is that correct?
> So
> is the only time someone would actually use the proton API directly
> if
> one was running something embedded such as the lightweight messaging
> IPC
> analogy I mentioned above.
>
>
> Sorry if I'm being a bit thick about this, but as I say I really
> haven't
> seen an awful lot of publicity around proton and how it relates to
> qpid
> so I think it'd be really good to get the documentation around the
> relationships properly sorted to avoid a whole world of confusion.
>
> Regards,
> Frase
>
>
> On 03/01/13 20:00, Mary Hinton wrote:
> > I can't answer your other questions, but did anyone answer your
> > question on "What exactly is proton?
> > Have you gone to the proton web site:
> > http://qpid.apache.org/proton/
> >
> > Qpid was built around the older versions of the AMQP protocol.
> > Now that Proton is released, QPID can load the proton library and
> > be AMQP 1.0 compliant.
> > In the C++ code there are new AMQP projects, one for the server and
> > one for the client.
> >
> > After you compile and build all the required libraries (including
> > proton) and executables, you can use the qpidd executable with the
> > following argument
> > qpidd --load-module amqpd
> >
> > For the client you need to set up the environment variable
> > QPID_LOAD_MODULE and set it to the new amqp client project name,
> > e.g. amqpcd
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton Messenger and the Request/Response pattern
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Hi Mary,
Thanks for the response, unfortunately I must be a bit thick though,
cause I'm still not totally clear.
I did take a look at the proton web site a little while back, looking at
the link you attached it looks pretty much the same as the last time I
looked a month or so back. But as I say the relationship between qpid
and proton is far from clear (to me at least....)
I've just loaded up the main qpid website http://qpid.apache.org/ and
there's no obvious linkage between that and proton so that's not too
helpful. Also not helping with my confusion is that there has been quite
a bit of traffic on the main qpid mailing list relating to work that
Gordon Sim has been doing on qpid 0.2 there seem to be quite a few new
features in this including replacing the flow-to-disk paging with
something better, but Gordon has also been talking about adding AMQP v
1.0 functionality to qpid 0.2.
So is the AMQP stuff that Gordon is talking about just about combining
in the proton library? Will the web pages for the qpid 0.2 release make
all this clear.
I have to say that the proton work hasn't been especially well
publicised, I only stumbled across it by accident a month or so back by
noticing a reference to it in the mailing list so did some digging and
eventually found the web page - and I look at the qpid mailing list
pretty much daily.
From your description below and from the web page it looks like proton
is primarily a library so you'd use it in combination with something
like qpid, but is the idea that it could also be used "brokerless" so
I'm thinking something analogous to say zeromq where say one might want
an extremely lightweight messaging IPC mechanism between processes on
the same host for example without the overhead of a full message broker
process????
I noticed the proton API documentation - it looks very python centric at
the moment, your maul below also suggests C++ but what's the state of
Java support at the moment - looks like there's a proton-j download, how
does that relate to JMS and the qpid Java client libraries??
The API stuff also makes me a bit nervous - relating on to the original
thread spawning this question. The is already *way too much* confusion
at the moment relating to qpid::messaging versus qpid::client, more or
less once a week someone posts to the main qpid mailing list about that
so I think it's really important that the relationship between qpid and
proton is spelled out clearly in a very prominent place!! It sounds like
for the most part that the proton APIs are lower level APIs so that for
example a client would use qpid::messaging (or JMS) and that would
delegate in the client runtime to the proton APIs, is that correct? So
is the only time someone would actually use the proton API directly if
one was running something embedded such as the lightweight messaging IPC
analogy I mentioned above.
Sorry if I'm being a bit thick about this, but as I say I really haven't
seen an awful lot of publicity around proton and how it relates to qpid
so I think it'd be really good to get the documentation around the
relationships properly sorted to avoid a whole world of confusion.
Regards,
Frase
On 03/01/13 20:00, Mary Hinton wrote:
> I can't answer your other questions, but did anyone answer your question on "What exactly is proton?
> Have you gone to the proton web site:
> http://qpid.apache.org/proton/
>
> Qpid was built around the older versions of the AMQP protocol.
> Now that Proton is released, QPID can load the proton library and be AMQP 1.0 compliant.
> In the C++ code there are new AMQP projects, one for the server and one for the client.
>
> After you compile and build all the required libraries (including proton) and executables, you can use the qpidd executable with the following argument
> qpidd --load-module amqpd
>
> For the client you need to set up the environment variable QPID_LOAD_MODULE and set it to the new amqp client project name,
> e.g. amqpcd
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton Messenger and the Request/Response pattern
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Hi Guys,
just my 2p worth, but I have to say I'm not at all convinced by any of this.
I personally don't see what's wrong with the standard model for the
request/response pattern of requiring a reply-to address and an optional
correlation ID.
A key point here is that this is and has been for some time the de facto
request/response model across most messaging systems and from my
perspective messing around with it is likely to add confusion and
actually add a barrier to adopting AMQP. I'm not exactly clear why the
current model is seen by anyone as a barrier to adopting AMQP - is this
actually the case? I'm dubious because the model described here is
exactly the same as the approach used in the JMS API.
Actually the JMS bit is significant too, clearly Qpid is more than just
JMS and is clearly a lot more platform agnostic, but I'd certainly wish
to avoid API divergence as far as possible without good reason. For sure
there are useful things that can be done in qpid::messaging that aren't
part of the JMS API but mostly these are to do with Qpid/AMQP
optimisations for message pipelining/asynchrony/prefetch etc.
BTW Ted said " It also sets the correlation_id field if it needs to
dispatch multiple responses.", I'm sure that you know this, but the
correlation_id is also used for the asynchronous request/response
pattern where a response might occur some time after the initial request
- that model is used in QMF2 for query subscriptions among other things.
There's a suggestion in this thread that "However in the more common
case where the response is intended to come back to the client, it's not
clear at all what to put in the reply_to field. ". I'm not sure that's
the case at all in the following code snippet from my QMF2 API
Implementation
"
Destination destination = (replyHandle == null) ?
_replyAddress : _asyncReplyAddress;
MapMessage request = _syncSession.createMapMessage();
request.setJMSReplyTo(destination);
request.setJMSCorrelationID(replyHandle);
request.setStringProperty("x-amqp-0-10.app-id", "qmf2");
request.setStringProperty("method", "request");
request.setStringProperty("qmf.opcode", "_query_request");
request.setStringProperty("qpid.subject", agentName);
"
It was pretty clear to me that the replyTo destination was my main
replyAddress for synchronous responses and the asyncReplyAddress (which
was on a different Session) when a replyHandle was set and so related to
the asynchronous request/response pattern.
Actually that also makes me think that the logic of the statement
"However in the more common case where the response is intended to come
back to the client" is flawed in the sense that this appears to be
asserting that a given client will only have a single replyTo address -
sure that'll be the case in many instances, but as above if one wants a
mixture of synchronous and asynchronous request/response a client will
need at least two addresses bound on different sessions.
There's also an assertion in this thread "requester creates and
subscribes to a temporary queue with a _unique_ name" again that's true
in *most* cases, but it's important to consider that it isn't always the
case that a temporary queue is used. For a scenario where we want a
request/response where the request may take a very long time the client
may have a named durable queue so that if the actual client was "down
for maintenance" when the service provider happened to send it's
asynchronous response + cid when the client comes back up it'll actually
get the response. If the service provider is doing some expensive
processing you really won't want to miss the response when it finally
occurs :-)
Sorry I don't mean to sound like I'm on my high-horse, but I'm not at
all keen on messing around with a fairly well understood and pretty
standard messaging pattern.
If the proposal is simply for an _additional_ API that may be used as a
"shorthand" then perhaps that's OK, but I still thing we should be wary
of adding additional variance between Java and C++ APIs without good
reason. I still wince at the confusion caused by qpid::messaging and
qpid::client (It has taken me ages to get some of my colleagues off
qpid::client) so if you are planning on adding some sugar I'd definitely
prefer to see it in a new namespace so it's pretty easy to differentiate
between the *core* messaging APIs and additional abstractions built on
those APIs. If that is done then it should also be possible to add an
additional Java package that uses new Qpid specific APIs built on top of
vanilla JMS.
FWIW I'd actually prefer to see the effort put into getting the C++ QMF
API up to spec. I don't know if things have changed since the last time
I looked but the QMF2 stuff for C++, whilst it conforms to the QMF2
protocol, doesn't bear any resemblance at all to the QMF2 API spec. If
you want to really provide an abstraction for request/response you won't
get much more of an abstraction than something like:
......
QmfConsoleData queue = _queueCache.get(queueName);
......
QmfData arguments = new QmfData();
arguments.setValue("request", (long)(_purge*msgDepth));
queue.invokeMethod("purge", arguments);
As you know all of this stuff is built on the request/response patterns
and there are pretty regular postings on the mailing list from people
wishing to manipulate/get tats from queues etc. from within normal
messaging apps. Clearly it's possible using the QMF2 map message
protocol if you know what you're doing but there's too little
consistency cross language despite there being a well specified protocol
and API, so personally I'd find that more productive than tinkering with
fairly standard patterns.
All just IMHO of course :-)
BTW what *exactly* is Proton, I've seen a few references to it but
nothing is especially clear to me, is Proton the name for the AMQP v1.0
version of Qpid? How does it differ from say Qpid v0.20? Will Proton end
up being Qpid v1.0?
Best regards,
Frase
On 02/01/13 21:58, Ted Ross wrote:
>
> On 01/02/2013 02:39 PM, William Henry wrote:
>>
>> ----- Original Message -----
>>> I'd like to start a discussion on how, from an API perspective,
>>> applications can use the request/response pattern. If we get this
>>> right, we will remove a significant barrier to adoption of AMQP.
>>>
>>> Middleware messaging systems typically do a poor job of supporting
>>> this
>>> pattern. The Qpid APIs are quite lacking in this regard (requester
>>> creates and subscribes to a temporary queue with a _unique_ name and
>>> places this name in the reply-to field).
>>>
>>> Proton Messenger supports request/reply (see
>>> examples/messenger/$LANG/{client,server}) as follows:
>>>
>>> The requester (client) has to put _something_ into the request
>>> message's
>>> reply_to field. It also sets the correlation_id field if it needs to
>>> dispatch multiple responses. The responder (server) must copy the
>>> request message's reply_to field to the response message's address
>>> field
>>> and also copy the correlation_id.
>>>
>>> This API is good for the case where the client wants the response to
>>> go
>>> to a third party. In this case the reply_to is well understood to be
>>> the address of the third party receiver. However in the more common
>>> case where the response is intended to come back to the client, it's
>>> not
>>> clear at all what to put in the reply_to field.
>>>
>>> I propose that we allow the client to simply say
>>>
>>> request_msg.reply_expected(cid)
>> Could you even block here with:
>>
>> reply_msg = request_msg.reply_expected(cid)
>>
>> You can have a default parameter that indicates blocking. And you could
>>
>> request_msg.reply_expected(cid, FALSE)
>>
>> and then do the usual check for incoming messages etc.
>
> I really like this suggestion. The blocking variant is dirt simple
> and you don't even need to supply a correlation_id. It can be hidden
> beneath the API.
>
> I do think there may be a cleaner syntax/naming for this though.
>
> An alternate syntax for the client side might be to provide an
> alternative to "put" like "put_request" that annotates the message
> appropriately for request/response and returns the correlation_id.
> "send_request" could be the blocking variant.
>
>>
>>> (I added the correlation_id argument because it's almost always going
>>> to
>>> be needed). Further, the server could use
>>>
>>> reply_msg.in_reply_to(request_msg)
>>>
>>> which would take care of the addresses and the correlation_id. It
>>> also
>>> provides a place to report an error if a request cannot be replied to
>>> (absent or invalid reply_to address) rather than force each server
>>> implementer to code this check.
>> Yeah, that's nice too. I see your point. I'm not sure if all
>> messaging purists will agree. But it makes sense to me that we need
>> something to handle this common use case more effectively.
>>
>> William
>>
>>> Thoughts?
>>>
>>> -Ted
>>>
>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
Re: Proton Messenger and the Request/Response pattern
Posted by Ted Ross <tr...@redhat.com>.
On 01/02/2013 02:39 PM, William Henry wrote:
>
> ----- Original Message -----
>> I'd like to start a discussion on how, from an API perspective,
>> applications can use the request/response pattern. If we get this
>> right, we will remove a significant barrier to adoption of AMQP.
>>
>> Middleware messaging systems typically do a poor job of supporting
>> this
>> pattern. The Qpid APIs are quite lacking in this regard (requester
>> creates and subscribes to a temporary queue with a _unique_ name and
>> places this name in the reply-to field).
>>
>> Proton Messenger supports request/reply (see
>> examples/messenger/$LANG/{client,server}) as follows:
>>
>> The requester (client) has to put _something_ into the request
>> message's
>> reply_to field. It also sets the correlation_id field if it needs to
>> dispatch multiple responses. The responder (server) must copy the
>> request message's reply_to field to the response message's address
>> field
>> and also copy the correlation_id.
>>
>> This API is good for the case where the client wants the response to
>> go
>> to a third party. In this case the reply_to is well understood to be
>> the address of the third party receiver. However in the more common
>> case where the response is intended to come back to the client, it's
>> not
>> clear at all what to put in the reply_to field.
>>
>> I propose that we allow the client to simply say
>>
>> request_msg.reply_expected(cid)
> Could you even block here with:
>
> reply_msg = request_msg.reply_expected(cid)
>
> You can have a default parameter that indicates blocking. And you could
>
> request_msg.reply_expected(cid, FALSE)
>
> and then do the usual check for incoming messages etc.
I really like this suggestion. The blocking variant is dirt simple and
you don't even need to supply a correlation_id. It can be hidden
beneath the API.
I do think there may be a cleaner syntax/naming for this though.
An alternate syntax for the client side might be to provide an
alternative to "put" like "put_request" that annotates the message
appropriately for request/response and returns the correlation_id.
"send_request" could be the blocking variant.
>
>> (I added the correlation_id argument because it's almost always going
>> to
>> be needed). Further, the server could use
>>
>> reply_msg.in_reply_to(request_msg)
>>
>> which would take care of the addresses and the correlation_id. It
>> also
>> provides a place to report an error if a request cannot be replied to
>> (absent or invalid reply_to address) rather than force each server
>> implementer to code this check.
> Yeah, that's nice too. I see your point. I'm not sure if all messaging purists will agree. But it makes sense to me that we need something to handle this common use case more effectively.
>
> William
>
>> Thoughts?
>>
>> -Ted
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton Messenger and the Request/Response pattern
Posted by Ted Ross <tr...@redhat.com>.
On 01/02/2013 02:39 PM, William Henry wrote:
>
> ----- Original Message -----
>> I'd like to start a discussion on how, from an API perspective,
>> applications can use the request/response pattern. If we get this
>> right, we will remove a significant barrier to adoption of AMQP.
>>
>> Middleware messaging systems typically do a poor job of supporting
>> this
>> pattern. The Qpid APIs are quite lacking in this regard (requester
>> creates and subscribes to a temporary queue with a _unique_ name and
>> places this name in the reply-to field).
>>
>> Proton Messenger supports request/reply (see
>> examples/messenger/$LANG/{client,server}) as follows:
>>
>> The requester (client) has to put _something_ into the request
>> message's
>> reply_to field. It also sets the correlation_id field if it needs to
>> dispatch multiple responses. The responder (server) must copy the
>> request message's reply_to field to the response message's address
>> field
>> and also copy the correlation_id.
>>
>> This API is good for the case where the client wants the response to
>> go
>> to a third party. In this case the reply_to is well understood to be
>> the address of the third party receiver. However in the more common
>> case where the response is intended to come back to the client, it's
>> not
>> clear at all what to put in the reply_to field.
>>
>> I propose that we allow the client to simply say
>>
>> request_msg.reply_expected(cid)
> Could you even block here with:
>
> reply_msg = request_msg.reply_expected(cid)
>
> You can have a default parameter that indicates blocking. And you could
>
> request_msg.reply_expected(cid, FALSE)
>
> and then do the usual check for incoming messages etc.
I really like this suggestion. The blocking variant is dirt simple and
you don't even need to supply a correlation_id. It can be hidden
beneath the API.
I do think there may be a cleaner syntax/naming for this though.
An alternate syntax for the client side might be to provide an
alternative to "put" like "put_request" that annotates the message
appropriately for request/response and returns the correlation_id.
"send_request" could be the blocking variant.
>
>> (I added the correlation_id argument because it's almost always going
>> to
>> be needed). Further, the server could use
>>
>> reply_msg.in_reply_to(request_msg)
>>
>> which would take care of the addresses and the correlation_id. It
>> also
>> provides a place to report an error if a request cannot be replied to
>> (absent or invalid reply_to address) rather than force each server
>> implementer to code this check.
> Yeah, that's nice too. I see your point. I'm not sure if all messaging purists will agree. But it makes sense to me that we need something to handle this common use case more effectively.
>
> William
>
>> Thoughts?
>>
>> -Ted
>>
>>
Re: Proton Messenger and the Request/Response pattern
Posted by William Henry <wh...@redhat.com>.
----- Original Message -----
> I'd like to start a discussion on how, from an API perspective,
> applications can use the request/response pattern. If we get this
> right, we will remove a significant barrier to adoption of AMQP.
>
> Middleware messaging systems typically do a poor job of supporting
> this
> pattern. The Qpid APIs are quite lacking in this regard (requester
> creates and subscribes to a temporary queue with a _unique_ name and
> places this name in the reply-to field).
>
> Proton Messenger supports request/reply (see
> examples/messenger/$LANG/{client,server}) as follows:
>
> The requester (client) has to put _something_ into the request
> message's
> reply_to field. It also sets the correlation_id field if it needs to
> dispatch multiple responses. The responder (server) must copy the
> request message's reply_to field to the response message's address
> field
> and also copy the correlation_id.
>
> This API is good for the case where the client wants the response to
> go
> to a third party. In this case the reply_to is well understood to be
> the address of the third party receiver. However in the more common
> case where the response is intended to come back to the client, it's
> not
> clear at all what to put in the reply_to field.
>
> I propose that we allow the client to simply say
>
> request_msg.reply_expected(cid)
Could you even block here with:
reply_msg = request_msg.reply_expected(cid)
You can have a default parameter that indicates blocking. And you could
request_msg.reply_expected(cid, FALSE)
and then do the usual check for incoming messages etc.
>
> (I added the correlation_id argument because it's almost always going
> to
> be needed). Further, the server could use
>
> reply_msg.in_reply_to(request_msg)
>
> which would take care of the addresses and the correlation_id. It
> also
> provides a place to report an error if a request cannot be replied to
> (absent or invalid reply_to address) rather than force each server
> implementer to code this check.
Yeah, that's nice too. I see your point. I'm not sure if all messaging purists will agree. But it makes sense to me that we need something to handle this common use case more effectively.
William
>
> Thoughts?
>
> -Ted
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Proton Messenger and the Request/Response pattern
Posted by William Henry <wh...@redhat.com>.
----- Original Message -----
> I'd like to start a discussion on how, from an API perspective,
> applications can use the request/response pattern. If we get this
> right, we will remove a significant barrier to adoption of AMQP.
>
> Middleware messaging systems typically do a poor job of supporting
> this
> pattern. The Qpid APIs are quite lacking in this regard (requester
> creates and subscribes to a temporary queue with a _unique_ name and
> places this name in the reply-to field).
>
> Proton Messenger supports request/reply (see
> examples/messenger/$LANG/{client,server}) as follows:
>
> The requester (client) has to put _something_ into the request
> message's
> reply_to field. It also sets the correlation_id field if it needs to
> dispatch multiple responses. The responder (server) must copy the
> request message's reply_to field to the response message's address
> field
> and also copy the correlation_id.
>
> This API is good for the case where the client wants the response to
> go
> to a third party. In this case the reply_to is well understood to be
> the address of the third party receiver. However in the more common
> case where the response is intended to come back to the client, it's
> not
> clear at all what to put in the reply_to field.
>
> I propose that we allow the client to simply say
>
> request_msg.reply_expected(cid)
Could you even block here with:
reply_msg = request_msg.reply_expected(cid)
You can have a default parameter that indicates blocking. And you could
request_msg.reply_expected(cid, FALSE)
and then do the usual check for incoming messages etc.
>
> (I added the correlation_id argument because it's almost always going
> to
> be needed). Further, the server could use
>
> reply_msg.in_reply_to(request_msg)
>
> which would take care of the addresses and the correlation_id. It
> also
> provides a place to report an error if a request cannot be replied to
> (absent or invalid reply_to address) rather than force each server
> implementer to code this check.
Yeah, that's nice too. I see your point. I'm not sure if all messaging purists will agree. But it makes sense to me that we need something to handle this common use case more effectively.
William
>
> Thoughts?
>
> -Ted
>
>