You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Gordon Sim <gs...@redhat.com> on 2014/05/13 15:48:15 UTC
Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect
to a server without sending or subscribing)
On 04/28/2014 10:21 PM, Gordon Sim wrote:
> On 04/23/2014 05:17 PM, Fraser Adams wrote:
>> BTW I wouldn't want to come across as favouring proton Messenger or
>> qpid::messaging over the other, as I said previously they are peer APIs
>> with different advantages and disadvantages,
>
> I'd certainly agree they both have different disadvantages :-) The
> picture faced by users looking for AMQP 1.0 clients is still confusing
> and suboptimal.
To elaborate a little more, here is my summary of the current AMQP 1.0
client choices. I think this is a topic that would benefit from some
(more) discussion and debate.
If you are using java the most obvious option is JMS, the one well
established API in messaging! It will be even better when the new
implementation with JMS 2 support comes along, bring simplification as
well as new features such as proper asynchronous publishing. Any
confusion here is really transitory.
If JMS doesn't fit for whatever reason, chances are the next option
considered will be proton, messenger or engine, available in different
languages: proton-j, proton-c and swigged versions of proton-c. An
increasing number of different things use proton to provide AMQP 1.0
support in one way or another.
I do think the inclusion of the two APIs in the same library continues
to be a source of confusion. Whatever the original or ultimate vision
is, they are at present quite different things. I also feel that by
being coupled together, both are held back a little from pursuing their
own distinctive goals.
The engine has been used relatively successfully in qpid::messaging,
qpidd and dispatch router already. The java variant has been used by
ActiveMQ and HornetQ and by the ongoing JMS work. It's reasonably
complete in terms of providing access to most aspects of the protocol,
though perhaps there are more additions/simplifications to the API (such
as the recent addition of events). It's pretty flexible but requires
more work than the user of a typical messaging library might expect.
Messenger is a message-oriented API where the library takes care of
connection and link management. The message-oriented nature hides the
differences between accepting incoming connections and making outgoing
connections, in theory allowing applications to support either direct
connections or intermediated/ brokered connections without any code
changes. It also - in theory - makes handling connection failure simpler
and takes it out of the hands of the application entirely. At present
this isn't true of course. There is no reconnect functionality built in
yet, nor does the API allow the application to reliably detect and
handle disconnection itself. Personally though, while I'm intrigued by
the notion of a message-oriented API, I'm not yet convinced by this
incarnation and the 'simple but powerful' billing.
Then there is qpid::messaging, the API that was designed specifically
with the transition to AMQP 1.0 in mind. Though there are still some
gaps - lack of transactions for example - its getting reasonably solid
now. The main drawback with the API is around non-blocking use. That
could be addressed if there was sufficient interest (I did some
prototyping a long time ago[1]). Integration into an existing event loop
isn't possible at present either.
The address syntax has also been a bit of a failure. For 1.0 I think it
is much simpler, without the need for the various x-bindings etc, but
could still be made simpler and clearer. There is also the
Qpid.Messaging .NET wrapper for qpid::messaging and some swigged
qpid::messaging clients. The pure python version doesn't support 1.0 and
there has been some further divergence of the c++ implementation from
this. The lack of appealing (to me) python support is a bee in my bonnet
at present.
There are also some new developments and/or experiments in various stages:
Chuck has built initial AMQP 1.0 support for NMS, the .NET API in the
ActiveMQ project. Andrew Stitcher has been working on something similar
for CMS (the c++ equivalent). It will be interesting to see if there is
any demand for these.
Fraser has been working on a javascript cross-compilation of proton
messenger. Also on the subject of javascript:
https://github.com/pofallon/node-qpid and the recently announced mqlight
node client: https://www.ibmdw.net/messaging/mq-light/node-js-api/. Both
of these use proton in some way. I haven't yet had a chance to dig into
any of these in earnest
Darryl has been exploring an eventful ruby API built on proton:
https://github.com/mcpierce/eventful-qpid-proton, again I've not yet had
a chance to look at this.
On the python side and interesting development is pyngus
(https://github.com/kgiusti/pyngus,
https://pypi.python.org/pypi/pyngus), developed by Ken. This is a
callback based python wrapper around the proton engine. I used this when
doing some work with Ken around AMQP 1.0 support in OpenStack's
messaging library and found it nice to work with. I think there is more
that could be done here as well. As I mentioned above, I don't really
like any of the alternatives for python at this point, and to me that is
a big gap.
This is already longer than intended and I've only summarised what's
there! Any other choices I've missed? Anyone else have opinions on the
existing options and/or directions for new development? If you were to
advise someone using language X on which API to pick, what would say? If
you are using one or more of these APIs, what is your opinion of them?
[1] https://reviews.apache.org/r/1687/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Gordon Sim <gs...@redhat.com>.
On 05/21/2014 10:50 PM, Andrew Stitcher wrote:
> On Wed, 2014-05-21 at 22:02 +0100, Gordon Sim wrote:
>> ...
>> So here is an initial, non-exhaustive list of current shortcomings from
>> me. Please all feel free to add to and correct this.
>>
>> qpid::messaging
>>
>> * boost dependency
> - This is a library dependency only*. And is no different in kind from
> its libuuid, cyrus-sasl or NSS dependency
I highlighted boost because its generally the one people have expressed
unhappiness with (mostly back in the old qpid::client era) and
represents the biggest difference in regards to dependencies as compared
to Messenger.
The cyrus-sasl and NSS dependencies are optional (though you get limited
sasl and no ssl without them). These choices will be the same fro proton
as well. The libuuid dependency is I think the least problematic and
again that is also needed by proton I believe.
> - In fact it is conceivable to
> remove boost if we want to put the effort into it.
Yes. Indeed, all the limitations I listed could be remedied if the
requisite effort was expended. The aim was not to list intrinsic defects
of the APIs, but simply to present the current snapshot of things that
may hinder users.
>> * limited to windows and linux
>
> It will build/run on FreeBSD and OSX too (and probably other
> BSDs/Unixes). Due to the heavyweight dependencies and internal threading
> it would be a challenge to put on an embedded system.
Ah, good to know! Thanks for the correction.
>> * requires thread per session
>> * can't be integrated into existing event loop
>> * can't handle incoming connections
>> * no support for transactions
>> * sasl support in windows very limited
>
> * Meaning you need to have the boost header/library present to build
> qpid::messaging but only the library present to use it. Boost is most
> problematic on Windows where there is no easy packaging for it.
>
> Andrew
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Andrew Stitcher <as...@redhat.com>.
On Wed, 2014-05-21 at 22:02 +0100, Gordon Sim wrote:
> ...
> So here is an initial, non-exhaustive list of current shortcomings from
> me. Please all feel free to add to and correct this.
>
> qpid::messaging
>
> * boost dependency
- This is a library dependency only*. And is no different in kind from
its libuuid, cyrus-sasl or NSS dependency - In fact it is conceivable to
remove boost if we want to put the effort into it.
> * limited to windows and linux
It will build/run on FreeBSD and OSX too (and probably other
BSDs/Unixes). Due to the heavyweight dependencies and internal threading
it would be a challenge to put on an embedded system.
> * requires thread per session
> * can't be integrated into existing event loop
> * can't handle incoming connections
> * no support for transactions
> * sasl support in windows very limited
* Meaning you need to have the boost header/library present to build
qpid::messaging but only the library present to use it. Boost is most
problematic on Windows where there is no easy packaging for it.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Gordon Sim <gs...@redhat.com>.
On 05/21/2014 07:32 PM, Fraser Adams wrote:
> I guess that I'm fairly agnostic about there being two APIs, provided
> there doesn't wind up being a competition, and that we stop at two :-)
[...]
> I guess I'm slightly disappointed that the tone of this thread seems to
> be that it's something of a competition, I personally don't think this
> it should be and that each has a valid place. I think I'd prefer the
> energy to be spent making both as good as they can be :-)
I'm sorry you are disappointed by the tone. I don't actually view this
as a competition between Messenger and qpid::messaging.
I've clearly not done a great job of expressing myself. The intent in
starting this thread was to highlight the difficulty I believe is faced
by any prospective adopter of AMQP 1.0 in selecting the API on which to
start building their applications (I could perhaps exempt those who head
straight for JMS) in the hope that an open and frank debate helps us
collectively identify ways to improve things.
I am not arguing that there is only room for one API, and certainly not
that qpid::messaging should be the only API. As a matter of fact, my
personal belief is that those two APIs are not sufficient, neither of
them being suitable for certain applications.
Choosing between the existing APIs is I think a small part of the
problem, but that is because it is not clear what limitations a given
choice imposes rather than because the choice exists.
Perhaps we've been reluctant to describe the choice in those terms
because it seems negative or because it is emphasising gaps at a given
point in time rather than intrinsic aspects of the respective designs.
However I think it is actually very important to users, and so its
perhaps useful for us to try and list the limitations, if for no other
reason than to see if there is a roadmap for resolution.
So here is an initial, non-exhaustive list of current shortcomings from
me. Please all feel free to add to and correct this.
qpid::messaging
* boost dependency
* limited to windows and linux
* requires thread per session
* can't be integrated into existing event loop
* can't handle incoming connections
* no support for transactions
* sasl support in windows very limited
Messenger
* no reconnect; no reliable way to handle it by application either
* no automatic message replay
* can't control lifecycle of links or connections (e.g. can't cancel
subscriptions or close a connection that will never be used again
without stopping the whole messenger)
* can't control source or target (can't use selectors, can't specify
dynamic node-properties, capabilities etc)
* can't set or see connection properties
* doesn't handle errors well, e.g. authentication authorisation failure,
node not found etc
* responds to incoming link attach requests by erroneously cloning
source/target, regardless of whether it understands or supports the
expectations represented
* no support for transactions
* sasl support limited to anonymous and plain
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
I've tried to stay fairly neutral on this issue, but FWIW here's my 2P
I guess that I'm fairly agnostic about there being two APIs, provided
there doesn't wind up being a competition, and that we stop at two :-)
Cards on the table I've come from a "traditional" connection oriented
messaging background, so I've got more familiarity with
JMS/qpid::messaging and that's where I'd naturally tend to gravitate,
but equally I've started to warm to Messenger lately and quite like
that I don't need to care about connections/sessions, it seems quite a
natural approach when there are lots of endpoints.
I think a lot of the difference does seem down to "style" and TBH that's
fine, it's often more natural for a particular application to gravitate
towards one style or another, so it's good to have the choice (as a
rubbish analogy think synchronous and asynchronous IO they both let you
do roughly the same thing, but sometimes it's more natural to do one
thing over another).
I do find Messenger a bit frustrating too though, probably because it's
a less familiar API and I don't really understand the nuances and I
really think it could do with a lot more published examples - especially
around trying to eke out the maximum throughput.
In many places I have a feeling that qpid::messaging is more complete,
FWIW I think that Gordon has done a great job on that and on the AMQP
1.0 support in qpidd. A place where qpid::messaging is *way* stronger
than Messenger (unless it has changed since I looked a month or so back)
is in the ability to set up complex subscriptions (so fancy link options
enabling selectors, named subscription queues etc.) when I looked
Messenger could only cope with basic subscriptions to named nodes (or #).
The place where I'm liking Messenger at the moment though is its
"dependency lite" nature, I guess that the vision is to be light,
portable and embeddable and I'm surprised that nobody has brought this
aspect up in this debate. Ultimately when all is said and done that's
probably the single most significant "compelling feature" of Messenger
vice qpid::messaging Messenger was ground-up intended for this use-case.
That's not to say that qpid::messaging couldn't fit in there (or be
modified to do so) but that wasn't (I don't believe) one of the intended
design goals.
I guess I'm slightly disappointed that the tone of this thread seems to
be that it's something of a competition, I personally don't think this
it should be and that each has a valid place. I think I'd prefer the
energy to be spent making both as good as they can be :-)
Regards,
Frase
On 21/05/14 15:48, Gordon Sim wrote:
> On 05/21/2014 02:10 PM, Ken Giusti wrote:
>> I think of qpid::messaging as being a "traditional" client api.
> [...]
>> Messenger, as an alternative, provides (or at least promises to
>> provide) solutions to a lot of the issues a "traditional" API has
>> left to the application implementation. Things like connection
>> failover, message retries,
>
> Automatic failover and message retry *is* supported in qpid::messaging
> (it isn't yet in Messenger).
>
>> credit scheduling,
>
> What is that exactly? Messenger::recv(N) essentially distributes N
> credits across however many incoming links there are, right? Whereas
> qpid::messaging allows capacity to be set per subscriber and maintains
> the window of credits accordingly.
>
> So is the key difference here that in one API the credit is controlled
> per-subscription whereas in the other it is controlled in aggregate.
>
> Where the number of receivers is larger than the number of messages
> the application is prepared to accept, dealing with the credit in
> aggregate and having it automatically (re)distributed as needed may
> indeed be useful. Of course the same feature could easily be built as
> a utility on top of something like qpid::messaging.
>
>> routing,
>
> So by this we mean the fact that Messenger looks at the address 'to'
> field of the message, applies some optional rules to that, and then
> find or creates the link to send it over.
>
> This could of course also be built on top of qpid::messaging (or
> indeed JMS).
>
>> and even client-side store are provided by Messenger.
>
> When you say 'are provided' you mean 'might be provided in the future'?
>
>> Such features would probably feel cumbersome
>
> I don't think it is the 'features' that are cumbersome, it's the
> restrictions.
>
>> to someone looking for a JMS-like API (and IMHO may be better off
>> with qpid::messaging), but for those folks who may not be bound to a
>> legacy application, Messenger offers some useful features.
>
> I've heard this sentiment in different ways quite a lot. I.e.
> qpid::messaging and JMS are 'legacy' approaches, are for people who
> aren't free to choose etc, whereas Messenger represents the future,
> the ideal if nothing holds you back etc.
>
> I don't go along with that view personally; I see nothing that really
> justifies it. It also seems to me to be quite counter to the notion
> that the APIs 'complement' each other, at least in my understanding of
> what that means[1].
>
> I'm certainly not arguing that qpid::messaging is the ideal API
> either, or that there is only room for one API. I'm keen to see if we
> can improve the general situation and feel that some debate around the
> different visions that exist within the community would be helpful in
> enabling better collaboration on that goal.
>
> --Gordon.
>
> [1] complement, verb, /ˈkɒm.plɪ.ment/:
>
> "to make something else seem better or more attractive
> when combining with it"
>
> (from http://dictionary.cambridge.org)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Gordon Sim <gs...@redhat.com>.
On 05/21/2014 02:10 PM, Ken Giusti wrote:
> I think of qpid::messaging as being a "traditional" client api.
[...]
> Messenger, as an alternative, provides (or at least promises to
> provide) solutions to a lot of the issues a "traditional" API has
> left to the application implementation. Things like connection
> failover, message retries,
Automatic failover and message retry *is* supported in qpid::messaging
(it isn't yet in Messenger).
> credit scheduling,
What is that exactly? Messenger::recv(N) essentially distributes N
credits across however many incoming links there are, right? Whereas
qpid::messaging allows capacity to be set per subscriber and maintains
the window of credits accordingly.
So is the key difference here that in one API the credit is controlled
per-subscription whereas in the other it is controlled in aggregate.
Where the number of receivers is larger than the number of messages the
application is prepared to accept, dealing with the credit in aggregate
and having it automatically (re)distributed as needed may indeed be
useful. Of course the same feature could easily be built as a utility on
top of something like qpid::messaging.
> routing,
So by this we mean the fact that Messenger looks at the address 'to'
field of the message, applies some optional rules to that, and then find
or creates the link to send it over.
This could of course also be built on top of qpid::messaging (or indeed
JMS).
> and even client-side store are provided by Messenger.
When you say 'are provided' you mean 'might be provided in the future'?
> Such features would probably feel cumbersome
I don't think it is the 'features' that are cumbersome, it's the
restrictions.
> to someone looking for a JMS-like API (and IMHO may be better off
> with qpid::messaging), but for those folks who may not be bound to a
> legacy application, Messenger offers some useful features.
I've heard this sentiment in different ways quite a lot. I.e.
qpid::messaging and JMS are 'legacy' approaches, are for people who
aren't free to choose etc, whereas Messenger represents the future, the
ideal if nothing holds you back etc.
I don't go along with that view personally; I see nothing that really
justifies it. It also seems to me to be quite counter to the notion that
the APIs 'complement' each other, at least in my understanding of what
that means[1].
I'm certainly not arguing that qpid::messaging is the ideal API either,
or that there is only room for one API. I'm keen to see if we can
improve the general situation and feel that some debate around the
different visions that exist within the community would be helpful in
enabling better collaboration on that goal.
--Gordon.
[1] complement, verb, /ˈkɒm.plɪ.ment/:
"to make something else seem better or more attractive
when combining with it"
(from http://dictionary.cambridge.org)
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Gordon Sim <gs...@redhat.com>.
On 05/21/2014 02:10 PM, Ken Giusti wrote:
> I think of qpid::messaging as being a "traditional" client api.
[...]
> Messenger, as an alternative, provides (or at least promises to
> provide) solutions to a lot of the issues a "traditional" API has
> left to the application implementation. Things like connection
> failover, message retries,
Automatic failover and message retry *is* supported in qpid::messaging
(it isn't yet in Messenger).
> credit scheduling,
What is that exactly? Messenger::recv(N) essentially distributes N
credits across however many incoming links there are, right? Whereas
qpid::messaging allows capacity to be set per subscriber and maintains
the window of credits accordingly.
So is the key difference here that in one API the credit is controlled
per-subscription whereas in the other it is controlled in aggregate.
Where the number of receivers is larger than the number of messages the
application is prepared to accept, dealing with the credit in aggregate
and having it automatically (re)distributed as needed may indeed be
useful. Of course the same feature could easily be built as a utility on
top of something like qpid::messaging.
> routing,
So by this we mean the fact that Messenger looks at the address 'to'
field of the message, applies some optional rules to that, and then find
or creates the link to send it over.
This could of course also be built on top of qpid::messaging (or indeed
JMS).
> and even client-side store are provided by Messenger.
When you say 'are provided' you mean 'might be provided in the future'?
> Such features would probably feel cumbersome
I don't think it is the 'features' that are cumbersome, it's the
restrictions.
> to someone looking for a JMS-like API (and IMHO may be better off
> with qpid::messaging), but for those folks who may not be bound to a
> legacy application, Messenger offers some useful features.
I've heard this sentiment in different ways quite a lot. I.e.
qpid::messaging and JMS are 'legacy' approaches, are for people who
aren't free to choose etc, whereas Messenger represents the future, the
ideal if nothing holds you back etc.
I don't go along with that view personally; I see nothing that really
justifies it. It also seems to me to be quite counter to the notion that
the APIs 'complement' each other, at least in my understanding of what
that means[1].
I'm certainly not arguing that qpid::messaging is the ideal API either,
or that there is only room for one API. I'm keen to see if we can
improve the general situation and feel that some debate around the
different visions that exist within the community would be helpful in
enabling better collaboration on that goal.
--Gordon.
[1] complement, verb, /ˈkɒm.plɪ.ment/:
"to make something else seem better or more attractive
when combining with it"
(from http://dictionary.cambridge.org)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Ken Giusti <kg...@redhat.com>.
Hi Gordon,
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: proton@qpid.apache.org
> Sent: Monday, May 19, 2014 11:25:41 AM
> Subject: Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or
> subscribing)
>
> On 05/15/2014 01:44 PM, Ken Giusti wrote:
> > I think we should develop Messenger as an alternative client API to
> > qpid::messaging, focusing on use cases that are not necessarily well
> > covered by the existing qpid::messaging API. I think they
> > complement each other nicely.
>
> In what way do you think they complement each other?
>
I think you've touched on it below - they do differ primarily in style. But I think it goes beyond that. I think of qpid::messaging as being a "traditional" client api. It fits best in those scenarios where the application directly manages the connections (setup and fail-over), message sending/receiving, and credit. I suspect there's a lot of existing messaging systems that expect that kind of API, and will find qpid::messaging a better fit than Messenger.
Messenger, as an alternative, provides (or at least promises to provide) solutions to a lot of the issues a "traditional" API has left to the application implementation. Things like connection failover, message retries, credit scheduling, routing, and even client-side store are provided by Messenger. Such features would probably feel cumbersome to someone looking for a JMS-like API (and IMHO may be better off with qpid::messaging), but for those folks who may not be bound to a legacy application, Messenger offers some useful features.
> [...]
> > I think we'd be much better off if we can separate the problem spaces
> > these two client APIs attempt to address, and clearly communicate
> > these differences so that users can find the right API for their
> > particular use cases
>
> That sounds neat and tidy in theory. I suspect it is not so simple in
> practice.
>
> > (example: connection oriented vs message oriented).
>
> I view that as more a question of 'style' than problem space. (I suspect
> it also raises almost as many questions as it answers).
>
> The existence of alternatives is not itself inherently problematic. What
> matters is how confident a prospective adopter feels when evaluating
> options for AMQP and how easily he or she would succeed if AMQP were
> embraced. It's not a question of eliminating choices, its a question of
> improving the experience.
>
> [...]
> > I think we should take an active role in promoting this new
> > experimental, community-led APIs that you mentioned. To be clear,
> > I'm not advocating that we (QPID) _support_ them, but I think we
> > should add links to them directly from our QPID web site, along side
> > the links to Messenger and qpid::messaging.
>
> I'm not sure what taking 'an active role in promoting' would mean, but I
> confess it makes me nervous. For one thing the projects I linked to vary
> widely in license, governance and maturity.
>
> On reflection and re-reading, my post was rather rushed and confused and
> the list of links was perhaps a mistake.
>
> The central point I am trying to make, is that though there are a
> variety of different *individual* initiatives, selecting an AMQP 1.0
> client one can have confidence in is still not easy and it seems to me
> there is no real *collective* initiative to improve this.
>
Sadly, I have to agree. How do we (qpid) go about solving this?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Ken Giusti <kg...@redhat.com>.
Hi Gordon,
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: proton@qpid.apache.org
> Sent: Monday, May 19, 2014 11:25:41 AM
> Subject: Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or
> subscribing)
>
> On 05/15/2014 01:44 PM, Ken Giusti wrote:
> > I think we should develop Messenger as an alternative client API to
> > qpid::messaging, focusing on use cases that are not necessarily well
> > covered by the existing qpid::messaging API. I think they
> > complement each other nicely.
>
> In what way do you think they complement each other?
>
I think you've touched on it below - they do differ primarily in style. But I think it goes beyond that. I think of qpid::messaging as being a "traditional" client api. It fits best in those scenarios where the application directly manages the connections (setup and fail-over), message sending/receiving, and credit. I suspect there's a lot of existing messaging systems that expect that kind of API, and will find qpid::messaging a better fit than Messenger.
Messenger, as an alternative, provides (or at least promises to provide) solutions to a lot of the issues a "traditional" API has left to the application implementation. Things like connection failover, message retries, credit scheduling, routing, and even client-side store are provided by Messenger. Such features would probably feel cumbersome to someone looking for a JMS-like API (and IMHO may be better off with qpid::messaging), but for those folks who may not be bound to a legacy application, Messenger offers some useful features.
> [...]
> > I think we'd be much better off if we can separate the problem spaces
> > these two client APIs attempt to address, and clearly communicate
> > these differences so that users can find the right API for their
> > particular use cases
>
> That sounds neat and tidy in theory. I suspect it is not so simple in
> practice.
>
> > (example: connection oriented vs message oriented).
>
> I view that as more a question of 'style' than problem space. (I suspect
> it also raises almost as many questions as it answers).
>
> The existence of alternatives is not itself inherently problematic. What
> matters is how confident a prospective adopter feels when evaluating
> options for AMQP and how easily he or she would succeed if AMQP were
> embraced. It's not a question of eliminating choices, its a question of
> improving the experience.
>
> [...]
> > I think we should take an active role in promoting this new
> > experimental, community-led APIs that you mentioned. To be clear,
> > I'm not advocating that we (QPID) _support_ them, but I think we
> > should add links to them directly from our QPID web site, along side
> > the links to Messenger and qpid::messaging.
>
> I'm not sure what taking 'an active role in promoting' would mean, but I
> confess it makes me nervous. For one thing the projects I linked to vary
> widely in license, governance and maturity.
>
> On reflection and re-reading, my post was rather rushed and confused and
> the list of links was perhaps a mistake.
>
> The central point I am trying to make, is that though there are a
> variety of different *individual* initiatives, selecting an AMQP 1.0
> client one can have confidence in is still not easy and it seems to me
> there is no real *collective* initiative to improve this.
>
Sadly, I have to agree. How do we (qpid) go about solving this?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Gordon Sim <gs...@redhat.com>.
On 05/15/2014 01:44 PM, Ken Giusti wrote:
> I think we should develop Messenger as an alternative client API to
> qpid::messaging, focusing on use cases that are not necessarily well
> covered by the existing qpid::messaging API. I think they
> complement each other nicely.
In what way do you think they complement each other?
[...]
> I think we'd be much better off if we can separate the problem spaces
> these two client APIs attempt to address, and clearly communicate
> these differences so that users can find the right API for their
> particular use cases
That sounds neat and tidy in theory. I suspect it is not so simple in
practice.
> (example: connection oriented vs message oriented).
I view that as more a question of 'style' than problem space. (I suspect
it also raises almost as many questions as it answers).
The existence of alternatives is not itself inherently problematic. What
matters is how confident a prospective adopter feels when evaluating
options for AMQP and how easily he or she would succeed if AMQP were
embraced. It's not a question of eliminating choices, its a question of
improving the experience.
[...]
> I think we should take an active role in promoting this new
> experimental, community-led APIs that you mentioned. To be clear,
> I'm not advocating that we (QPID) _support_ them, but I think we
> should add links to them directly from our QPID web site, along side
> the links to Messenger and qpid::messaging.
I'm not sure what taking 'an active role in promoting' would mean, but I
confess it makes me nervous. For one thing the projects I linked to vary
widely in license, governance and maturity.
On reflection and re-reading, my post was rather rushed and confused and
the list of links was perhaps a mistake.
The central point I am trying to make, is that though there are a
variety of different *individual* initiatives, selecting an AMQP 1.0
client one can have confidence in is still not easy and it seems to me
there is no real *collective* initiative to improve this.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Gordon Sim <gs...@redhat.com>.
On 05/15/2014 01:44 PM, Ken Giusti wrote:
> I think we should develop Messenger as an alternative client API to
> qpid::messaging, focusing on use cases that are not necessarily well
> covered by the existing qpid::messaging API. I think they
> complement each other nicely.
In what way do you think they complement each other?
[...]
> I think we'd be much better off if we can separate the problem spaces
> these two client APIs attempt to address, and clearly communicate
> these differences so that users can find the right API for their
> particular use cases
That sounds neat and tidy in theory. I suspect it is not so simple in
practice.
> (example: connection oriented vs message oriented).
I view that as more a question of 'style' than problem space. (I suspect
it also raises almost as many questions as it answers).
The existence of alternatives is not itself inherently problematic. What
matters is how confident a prospective adopter feels when evaluating
options for AMQP and how easily he or she would succeed if AMQP were
embraced. It's not a question of eliminating choices, its a question of
improving the experience.
[...]
> I think we should take an active role in promoting this new
> experimental, community-led APIs that you mentioned. To be clear,
> I'm not advocating that we (QPID) _support_ them, but I think we
> should add links to them directly from our QPID web site, along side
> the links to Messenger and qpid::messaging.
I'm not sure what taking 'an active role in promoting' would mean, but I
confess it makes me nervous. For one thing the projects I linked to vary
widely in license, governance and maturity.
On reflection and re-reading, my post was rather rushed and confused and
the list of links was perhaps a mistake.
The central point I am trying to make, is that though there are a
variety of different *individual* initiatives, selecting an AMQP 1.0
client one can have confidence in is still not easy and it seems to me
there is no real *collective* initiative to improve this.
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Ken Giusti <kg...@redhat.com>.
Hi Gordon,
My thoughts:
For Java client API I think JMS should be our primary focus.
I'm quite fond of qpid::messaging. I would like to see us (QPID) continue to support this api and evolve it. For multi-language support, I think we should leverage swig as is done by Messenger today. However, I'd recommend that we provide only a direct mapping of the API into the target language rather than create a more complex swig wrapper that tries to make this API more "native" to the language. I wouldn't prevent this if someone would like to step up and own such an effort, but I think the additional testing and documentation of such a layer would require a larger investment of developer resources than a simple direct swig mapping.
I'm also a big fan of the Messenger client API. However, I think we should develop Messenger as an alternative client API to qpid::messaging, focusing on use cases that are not necessarily well covered by the existing qpid::messaging API. I think they complement each other nicely. My biggest fear is that Messenger tries to become The One Client API for Every Problem, and in the process actually becomes The Bloated Mess that Everyone Hates. I think we'd be much better off if we can separate the problem spaces these two client APIs attempt to address, and clearly communicate these differences so that users can find the right API for their particular use cases (example: connection oriented vs message oriented).
We should not promote Engine (in either flavor) as a _client_ api. It requires way too much familiarity with AMQP 1.0 and thus it has a learning curve that is not appropriate for a client API. We _should_ be promoting Engine as the AMQP 1.0 toolkit for building things like client api's (and brokers, switches, services, etc), but definitely not a client API. I'd even go further and recommend splitting the Engine API into its own proper library separate from Messenger.
Finally, I think we should take an active role in promoting this new experimental, community-led APIs that you mentioned. To be clear, I'm not advocating that we (QPID) _support_ them, but I think we should add links to them directly from our QPID web site, along side the links to Messenger and qpid::messaging. Of course, the web page should make it clear that these are non-QPID projects, and users should contact the developers directly for fixes, questions, etc. The important thing about these efforts is that they have the potential to become The Next Big Thing for some subset of the AMQP user base - think of them as 'incubator projects'. If one or more of them really take off, we could pursue having them become proper QPID sub projects. In other words, there's a potential for a real benefit from these that requires a minimum effort from us (links on our web page).
thanks,
-K
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: proton@qpid.apache.org
> Sent: Tuesday, May 13, 2014 9:48:15 AM
> Subject: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or
> subscribing)
>
> On 04/28/2014 10:21 PM, Gordon Sim wrote:
> > On 04/23/2014 05:17 PM, Fraser Adams wrote:
> >> BTW I wouldn't want to come across as favouring proton Messenger or
> >> qpid::messaging over the other, as I said previously they are peer APIs
> >> with different advantages and disadvantages,
> >
> > I'd certainly agree they both have different disadvantages :-) The
> > picture faced by users looking for AMQP 1.0 clients is still confusing
> > and suboptimal.
>
> To elaborate a little more, here is my summary of the current AMQP 1.0
> client choices. I think this is a topic that would benefit from some
> (more) discussion and debate.
>
> If you are using java the most obvious option is JMS, the one well
> established API in messaging! It will be even better when the new
> implementation with JMS 2 support comes along, bring simplification as
> well as new features such as proper asynchronous publishing. Any
> confusion here is really transitory.
>
> If JMS doesn't fit for whatever reason, chances are the next option
> considered will be proton, messenger or engine, available in different
> languages: proton-j, proton-c and swigged versions of proton-c. An
> increasing number of different things use proton to provide AMQP 1.0
> support in one way or another.
>
> I do think the inclusion of the two APIs in the same library continues
> to be a source of confusion. Whatever the original or ultimate vision
> is, they are at present quite different things. I also feel that by
> being coupled together, both are held back a little from pursuing their
> own distinctive goals.
>
> The engine has been used relatively successfully in qpid::messaging,
> qpidd and dispatch router already. The java variant has been used by
> ActiveMQ and HornetQ and by the ongoing JMS work. It's reasonably
> complete in terms of providing access to most aspects of the protocol,
> though perhaps there are more additions/simplifications to the API (such
> as the recent addition of events). It's pretty flexible but requires
> more work than the user of a typical messaging library might expect.
>
> Messenger is a message-oriented API where the library takes care of
> connection and link management. The message-oriented nature hides the
> differences between accepting incoming connections and making outgoing
> connections, in theory allowing applications to support either direct
> connections or intermediated/ brokered connections without any code
> changes. It also - in theory - makes handling connection failure simpler
> and takes it out of the hands of the application entirely. At present
> this isn't true of course. There is no reconnect functionality built in
> yet, nor does the API allow the application to reliably detect and
> handle disconnection itself. Personally though, while I'm intrigued by
> the notion of a message-oriented API, I'm not yet convinced by this
> incarnation and the 'simple but powerful' billing.
>
> Then there is qpid::messaging, the API that was designed specifically
> with the transition to AMQP 1.0 in mind. Though there are still some
> gaps - lack of transactions for example - its getting reasonably solid
> now. The main drawback with the API is around non-blocking use. That
> could be addressed if there was sufficient interest (I did some
> prototyping a long time ago[1]). Integration into an existing event loop
> isn't possible at present either.
>
> The address syntax has also been a bit of a failure. For 1.0 I think it
> is much simpler, without the need for the various x-bindings etc, but
> could still be made simpler and clearer. There is also the
> Qpid.Messaging .NET wrapper for qpid::messaging and some swigged
> qpid::messaging clients. The pure python version doesn't support 1.0 and
> there has been some further divergence of the c++ implementation from
> this. The lack of appealing (to me) python support is a bee in my bonnet
> at present.
>
> There are also some new developments and/or experiments in various stages:
>
> Chuck has built initial AMQP 1.0 support for NMS, the .NET API in the
> ActiveMQ project. Andrew Stitcher has been working on something similar
> for CMS (the c++ equivalent). It will be interesting to see if there is
> any demand for these.
>
> Fraser has been working on a javascript cross-compilation of proton
> messenger. Also on the subject of javascript:
> https://github.com/pofallon/node-qpid and the recently announced mqlight
> node client: https://www.ibmdw.net/messaging/mq-light/node-js-api/. Both
> of these use proton in some way. I haven't yet had a chance to dig into
> any of these in earnest
>
> Darryl has been exploring an eventful ruby API built on proton:
> https://github.com/mcpierce/eventful-qpid-proton, again I've not yet had
> a chance to look at this.
>
> On the python side and interesting development is pyngus
> (https://github.com/kgiusti/pyngus,
> https://pypi.python.org/pypi/pyngus), developed by Ken. This is a
> callback based python wrapper around the proton engine. I used this when
> doing some work with Ken around AMQP 1.0 support in OpenStack's
> messaging library and found it nice to work with. I think there is more
> that could be done here as well. As I mentioned above, I don't really
> like any of the alternatives for python at this point, and to me that is
> a big gap.
>
> This is already longer than intended and I've only summarised what's
> there! Any other choices I've missed? Anyone else have opinions on the
> existing options and/or directions for new development? If you were to
> advise someone using language X on which API to pick, what would say? If
> you are using one or more of these APIs, what is your opinion of them?
>
> [1] https://reviews.apache.org/r/1687/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to
connect to a server without sending or subscribing)
Posted by Ken Giusti <kg...@redhat.com>.
Hi Gordon,
My thoughts:
For Java client API I think JMS should be our primary focus.
I'm quite fond of qpid::messaging. I would like to see us (QPID) continue to support this api and evolve it. For multi-language support, I think we should leverage swig as is done by Messenger today. However, I'd recommend that we provide only a direct mapping of the API into the target language rather than create a more complex swig wrapper that tries to make this API more "native" to the language. I wouldn't prevent this if someone would like to step up and own such an effort, but I think the additional testing and documentation of such a layer would require a larger investment of developer resources than a simple direct swig mapping.
I'm also a big fan of the Messenger client API. However, I think we should develop Messenger as an alternative client API to qpid::messaging, focusing on use cases that are not necessarily well covered by the existing qpid::messaging API. I think they complement each other nicely. My biggest fear is that Messenger tries to become The One Client API for Every Problem, and in the process actually becomes The Bloated Mess that Everyone Hates. I think we'd be much better off if we can separate the problem spaces these two client APIs attempt to address, and clearly communicate these differences so that users can find the right API for their particular use cases (example: connection oriented vs message oriented).
We should not promote Engine (in either flavor) as a _client_ api. It requires way too much familiarity with AMQP 1.0 and thus it has a learning curve that is not appropriate for a client API. We _should_ be promoting Engine as the AMQP 1.0 toolkit for building things like client api's (and brokers, switches, services, etc), but definitely not a client API. I'd even go further and recommend splitting the Engine API into its own proper library separate from Messenger.
Finally, I think we should take an active role in promoting this new experimental, community-led APIs that you mentioned. To be clear, I'm not advocating that we (QPID) _support_ them, but I think we should add links to them directly from our QPID web site, along side the links to Messenger and qpid::messaging. Of course, the web page should make it clear that these are non-QPID projects, and users should contact the developers directly for fixes, questions, etc. The important thing about these efforts is that they have the potential to become The Next Big Thing for some subset of the AMQP user base - think of them as 'incubator projects'. If one or more of them really take off, we could pursue having them become proper QPID sub projects. In other words, there's a potential for a real benefit from these that requires a minimum effort from us (links on our web page).
thanks,
-K
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: proton@qpid.apache.org
> Sent: Tuesday, May 13, 2014 9:48:15 AM
> Subject: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or
> subscribing)
>
> On 04/28/2014 10:21 PM, Gordon Sim wrote:
> > On 04/23/2014 05:17 PM, Fraser Adams wrote:
> >> BTW I wouldn't want to come across as favouring proton Messenger or
> >> qpid::messaging over the other, as I said previously they are peer APIs
> >> with different advantages and disadvantages,
> >
> > I'd certainly agree they both have different disadvantages :-) The
> > picture faced by users looking for AMQP 1.0 clients is still confusing
> > and suboptimal.
>
> To elaborate a little more, here is my summary of the current AMQP 1.0
> client choices. I think this is a topic that would benefit from some
> (more) discussion and debate.
>
> If you are using java the most obvious option is JMS, the one well
> established API in messaging! It will be even better when the new
> implementation with JMS 2 support comes along, bring simplification as
> well as new features such as proper asynchronous publishing. Any
> confusion here is really transitory.
>
> If JMS doesn't fit for whatever reason, chances are the next option
> considered will be proton, messenger or engine, available in different
> languages: proton-j, proton-c and swigged versions of proton-c. An
> increasing number of different things use proton to provide AMQP 1.0
> support in one way or another.
>
> I do think the inclusion of the two APIs in the same library continues
> to be a source of confusion. Whatever the original or ultimate vision
> is, they are at present quite different things. I also feel that by
> being coupled together, both are held back a little from pursuing their
> own distinctive goals.
>
> The engine has been used relatively successfully in qpid::messaging,
> qpidd and dispatch router already. The java variant has been used by
> ActiveMQ and HornetQ and by the ongoing JMS work. It's reasonably
> complete in terms of providing access to most aspects of the protocol,
> though perhaps there are more additions/simplifications to the API (such
> as the recent addition of events). It's pretty flexible but requires
> more work than the user of a typical messaging library might expect.
>
> Messenger is a message-oriented API where the library takes care of
> connection and link management. The message-oriented nature hides the
> differences between accepting incoming connections and making outgoing
> connections, in theory allowing applications to support either direct
> connections or intermediated/ brokered connections without any code
> changes. It also - in theory - makes handling connection failure simpler
> and takes it out of the hands of the application entirely. At present
> this isn't true of course. There is no reconnect functionality built in
> yet, nor does the API allow the application to reliably detect and
> handle disconnection itself. Personally though, while I'm intrigued by
> the notion of a message-oriented API, I'm not yet convinced by this
> incarnation and the 'simple but powerful' billing.
>
> Then there is qpid::messaging, the API that was designed specifically
> with the transition to AMQP 1.0 in mind. Though there are still some
> gaps - lack of transactions for example - its getting reasonably solid
> now. The main drawback with the API is around non-blocking use. That
> could be addressed if there was sufficient interest (I did some
> prototyping a long time ago[1]). Integration into an existing event loop
> isn't possible at present either.
>
> The address syntax has also been a bit of a failure. For 1.0 I think it
> is much simpler, without the need for the various x-bindings etc, but
> could still be made simpler and clearer. There is also the
> Qpid.Messaging .NET wrapper for qpid::messaging and some swigged
> qpid::messaging clients. The pure python version doesn't support 1.0 and
> there has been some further divergence of the c++ implementation from
> this. The lack of appealing (to me) python support is a bee in my bonnet
> at present.
>
> There are also some new developments and/or experiments in various stages:
>
> Chuck has built initial AMQP 1.0 support for NMS, the .NET API in the
> ActiveMQ project. Andrew Stitcher has been working on something similar
> for CMS (the c++ equivalent). It will be interesting to see if there is
> any demand for these.
>
> Fraser has been working on a javascript cross-compilation of proton
> messenger. Also on the subject of javascript:
> https://github.com/pofallon/node-qpid and the recently announced mqlight
> node client: https://www.ibmdw.net/messaging/mq-light/node-js-api/. Both
> of these use proton in some way. I haven't yet had a chance to dig into
> any of these in earnest
>
> Darryl has been exploring an eventful ruby API built on proton:
> https://github.com/mcpierce/eventful-qpid-proton, again I've not yet had
> a chance to look at this.
>
> On the python side and interesting development is pyngus
> (https://github.com/kgiusti/pyngus,
> https://pypi.python.org/pypi/pyngus), developed by Ken. This is a
> callback based python wrapper around the proton engine. I used this when
> doing some work with Ken around AMQP 1.0 support in OpenStack's
> messaging library and found it nice to work with. I think there is more
> that could be done here as well. As I mentioned above, I don't really
> like any of the alternatives for python at this point, and to me that is
> a big gap.
>
> This is already longer than intended and I've only summarised what's
> there! Any other choices I've missed? Anyone else have opinions on the
> existing options and/or directions for new development? If you were to
> advise someone using language X on which API to pick, what would say? If
> you are using one or more of these APIs, what is your opinion of them?
>
> [1] https://reviews.apache.org/r/1687/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K