what *exactly* is Proton? (was Re: Proton Messenger and the Request/Response pattern)

[Gordon Sim <gs...@redhat.com>]

On 01/04/2013 10:07 AM, Fraser Adams wrote:
> I really haven't seen an awful lot of publicity around proton and how it
> relates to qpid so I think it'd be really good to get the documentation
> around the relationships properly sorted to avoid a whole world of
> confusion.

You are quite right. Communication has been very poor and we need to 
change that.

On 01/03/2013 09:14 AM, Fraser Adams wrote:
> BTW what *exactly* is Proton, I've seen a few references to it but
> nothing is especially clear to me, is Proton the name for the AMQP v1.0
> version of Qpid? How does it differ from say Qpid v0.20? Will Proton end
> up being Qpid v1.0?

Let me try and lay out my own understanding. Forgive me if it seems a 
little long-winded, it isn't really that different from what William 
described...

Qpid is an openly governed community that seeks to aid adoption of AMQP 
through collaborative development of various pieces of open sourced 
software. Historically that has included two different brokers, various 
client libraries and management tools. Although the different parts 
should combine well, not every user will want or need every component. 
Given the purpose of AMQP is to enable interoperability and choice, it 
is important that individual components from Qpid can be combined with 
other software and systems in a flexible manner.

Proton is a new component under the Qpid umbrella - a set of components 
is perhaps more accurate as there is both a java and c version and some 
swig wrappers. It is an AMQP 1.0 toolkit.

Part of this is a 'protocol engine'; an implementation of the rules 
specified by AMQP 1.0 in a manner that is independent of any threading 
model or IO mechanism. This makes it easier to integrate in diverse 
contexts including existing brokers and messaging clients and not 
limited to just those within Qpid.

One of the uses has been to add AMQP 1.0 support to existing Qpid 
components, such as the work I have been doing to add 1.0 support to the 
qpid::messaging API in c++ and to the qpidd (i.e. c++) broker. There has 
also been work to add 1.0 support to ActiveMQ, and that also uses the 
(java version) of the proton protocol engine.

Being independent of any IO framework is important for one set of use 
cases, but it does mean that the protocol engine isn't really usable 'by 
itself'. It was recognised that adoption of 1.0 would be aided by a 
simple, self contained library allowing sending and receiving of messages.

To that end the messenger API was added to the proton toolkit. This ties 
network IO into the protocol engine, exposing a simple abstraction of an 
input and output queue for messages, each message containing an address 
to which it is to be delivered. It manages connections, session and 
links under the covers. At present it is a blocking API though support 
for non-blocking use is likely to evolve as well.

AMQP 1.0 is fully symmetric (i.e a peer protocol) as opposed to its 
asymmetric (i.e. client-server) predecessors. This means it can be used 
with or without intermediaries such as brokers. The messenger API allows 
an application to listen for incoming connections as well as initiating 
its own outgoing connections. It can therefore be used for simple, 
direct communication as well as communication through intermediaries. 
(You can also for example use a messenger based application to accept 
connections from a qpid::messaging or even a JMS client).

Proton is on its own release cycle at present, reflecting the early 
stage in its lifecycle. So far all other Qpid components have been on 
the same release cycle. So the 0.20 release includes qpidd, qpid-server 
(i.e. the qpid java broker), a JMS client, the c++ qpid::messaging 
library (and the deprecated qpid::client library), the python 
qpid.messaging library (and older deprecated libraries), QMF tools, a 
JMX console etc etc.

The 0.3 release of proton on the other hand contains c and java 
libraries offering the protocol engine and messenger APIs (the java 
version of the messenger API has some outstanding issues still) as well 
as wrappers for python, perl and ruby. The 0.20 release of qpidd and 
qpid::messaging has an optional dependency on the c version of proton, 
which if available enables 1.0 support in those components. The 
subsequent release of those two components will contain enhanced support.

So, qpid::messaging isn't going away. A key goal for that API was to 
allow smooth transition to AMQP 1.0 and that is still something I am 
working towards (0.20 represents the first step). Support for JMS 
likewise will continue. However the messenger API is in my opinion a 
novel approach that is may also be interesting for many cases. 
Ultimately I think the different APIs offer different styles. While 
there may be certain things that can't be done in one or other, often it 
may come down to personal preference.

Ultimately the vision behind AMQP is all about choice; enabling the 
emergence of a rich palette of interoperable pieces from which diverse 
solutions can be flexibly composed. AMQP 1.0 in my opinion expands the 
opportunities here and rightly Qpid is innovating and evolving to 
exploit those.

These are exciting times! We do however need more open communication 
(and better documentation ;-).

--Gordon.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: what *exactly* is Proton? (was Re: Proton Messenger and the Request/Response pattern)

[Gordon Sim <gs...@redhat.com>]

On 01/20/2013 05:50 PM, Fraser Adams wrote:
> It's starting to become somewhat clearer how proton fits in, though some
> of your comments still leave me with questions about how *ready* proton
> is yet - it looks though the C++ stuff is more advanced than the Java
> stuff - is that accurate?

Yes, but the gap is closing fast.

> Is there a mechanism to get a JMS API around
> the Java proton messenger library or would Java AMQP clients still have
> to talk with the underlying proton API?

There is already an AMQP 1.0 implementation of JMS in Qpid. It predates 
the proton work but the plan is to replace it with something that uses 
proton underneath at some point (more to ease future maintenance).

> I sort of see what you mean by your comment "Ultimately the vision
> behind AMQP is all about choice" perhaps that's fine but I think there
> needs to be some good guidance, patterns if you will for different
> use-cases. As an example qpid::client and qpid::messaging also offered a
> choice and you and I both know that that's now a bit of a slow motion
> car crash. Without strong guidance I think we'll not so much have
> "choice" as chaos.

I think with qpid::client it was very much a case of one being 
deprecated in favour of the other. In hindsight we could have been a lot 
clearer about signalling that.

With messenger v. qpid::messaging it is more a question of two different 
styles. There may be use cases that firmly imply one or the other, but I 
think that in large part it may come down to preference for one style or 
another.

> I think that there's scope for confusion in python land too - you
> mention the deprecated library, so I *think* that this means that the
> favoured one is a SWIG wrapper round qpid::messaging - is that correct.

Thats not actually what I meant. In python there is the equivalent of 
the old qpid::client API expcept (it's not called that and quite 
different in actual methods/classes).

However you do raise a good point. At present there is no AMQP 1.0 
implementation of the python qpid.messaging library. There *is* the SWIG 
wrapped equivalent from the c++ side (and this can be made to support 
AMQP 1.0) which is now pretty close to the native python API in behaviour.

One interesting question is how to proceed there. Is there for example 
sufficient interest and demand for (or interest in implementing) a pure 
python, AMQP 1.0 implementation of that library or is the SWIG version a 
good enough option? I don't know the answer, but collectively this list 
is the place to answer it I think.

> I suspect that there are plenty of people confused by which to use and I
> think there's definitely a need for good documentation and examples.

Agreed. People need to be able to quickly get a sense for what a given 
API can and can't do, and what it is like to code with.

> Some other concerns/thoughts. You mentioned "There has also been work to
> add 1.0 support to ActiveMQ, and that also uses the (java version) of
> the proton protocol engine." on one hand that's great from an
> interoperability perspective but again has the potential to cause huge
> amounts of confusion!! There needs to be some investment in "branding"
> IMHO for example how would one make a choice between ActiveMQ versus say
> the Qpid Java broker?

Again, a very good question.

> These sorts of choices are very real indeed. In a
> corporate sense one often has to do make choices over messaging systems
> and it's hard enough as it is!!!!! I've got the T-shirt there when
> looking at Qpid versus Tibco and justifying choices to non-techies is
> nobody's idea of fun :-( there needs to be some strong differentiators
> for the "hard of thinking".

He he, I do like that phrase! I do totally agree with your point on 
clear differentiation. I think the success of AMQP 1.0 and the upcoming 
work on ActiveMQ does change the picture there somewhat and we need to 
discuss that as a community.

> You also mention "qpid::messaging has an optional dependency on the c
> version of proton". Does this imply that proton is written in C (as
> opposed to C++)? Just curious on that one.

Yes, it is pure C.

> And don't get me started on QMF. There's two protocols for starters :-)
> then there's a Java implementation by yours truly that is still stuck in
> a Jira, a C++ API that bears no relationship to the published (but still
> draft) QMF2 API, two python implementations (one for each protocol). I
> don't know if things have changed lately but the python tools used to
> use the QMF1 API for a long time. Around this time last year there was
> some talk of a QMF2 sub-project given that it has application beyond
> Qpid given that it's a fairly good request/response API with service
> discovery, but discussion on that (certainly in the Qpid Users mailing
> list) has fizzled out.
>
> I really like QMF and as you can probably see I've done a fair bit with
> it, but my motivation is waning a bit given that I'm not at all clear on
> the status of it all.

First off, thanks and congratulations on what looks like a very nice new 
component (I haven't had a chance to unbundle and play with it yet but 
will do that asap)! It looks very much like something that should be 
included somewhere in the main svn (under extras seems like a good home 
initially).

Second, I share your love-hate relationship with QMF! I do love the 
simplicity of the overall approach (simply sending & receiving 
messages); I'm a lot more reticent about particular APIs though.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: what *exactly* is Proton? (was Re: Proton Messenger and the Request/Response pattern)

[Fraser Adams <fr...@blueyonder.co.uk>]

Thanks for this response Gordon and also (belated) thanks to William. 
I'd have said thanks sooner but I've been manically trying to get my 
QMF2/GUI stuff tested and released (you might come across my posts on 
some of the management remote methods).

It's starting to become somewhat clearer how proton fits in, though some 
of your comments still leave me with questions about how *ready* proton 
is yet - it looks though the C++ stuff is more advanced than the Java 
stuff - is that accurate? Is there a mechanism to get a JMS API around 
the Java proton messenger library or would Java AMQP clients still have 
to talk with the underlying proton API?

I sort of see what you mean by your comment "Ultimately the vision 
behind AMQP is all about choice" perhaps that's fine but I think there 
needs to be some good guidance, patterns if you will for different 
use-cases. As an example qpid::client and qpid::messaging also offered a 
choice and you and I both know that that's now a bit of a slow motion 
car crash. Without strong guidance I think we'll not so much have 
"choice" as chaos.

I think that there's scope for confusion in python land too - you 
mention the deprecated library, so I *think* that this means that the 
favoured one is a SWIG wrapper round qpid::messaging - is that correct. 
I suspect that there are plenty of people confused by which to use and I 
think there's definitely a need for good documentation and examples.

Some other concerns/thoughts. You mentioned "There has also been work to 
add 1.0 support to ActiveMQ, and that also uses the (java version) of 
the proton protocol engine." on one hand that's great from an 
interoperability perspective but again has the potential to cause huge 
amounts of confusion!! There needs to be some investment in "branding" 
IMHO for example how would one make a choice between ActiveMQ versus say 
the Qpid Java broker? These sorts of choices are very real indeed. In a 
corporate sense one often has to do make choices over messaging systems 
and it's hard enough as it is!!!!! I've got the T-shirt there when 
looking at Qpid versus Tibco and justifying choices to non-techies is 
nobody's idea of fun :-( there needs to be some strong differentiators 
for the "hard of thinking".

You also mention "qpid::messaging has an optional dependency on the c 
version of proton". Does this imply that proton is written in C (as 
opposed to C++)? Just curious on that one.


And don't get me started on QMF. There's two protocols for starters :-) 
then there's a Java implementation by yours truly that is still stuck in 
a Jira, a C++ API that bears no relationship to the published (but still 
draft) QMF2 API, two python implementations (one for each protocol). I 
don't know if things have changed lately but the python tools used to 
use the QMF1 API for a long time. Around this time last year there was 
some talk of a QMF2 sub-project given that it has application beyond 
Qpid given that it's a fairly good request/response API with service 
discovery, but discussion on that (certainly in the Qpid Users mailing 
list) has fizzled out.

I really like QMF and as you can probably see I've done a fair bit with 
it, but my motivation is waning a bit given that I'm not at all clear on 
the status of it all.

So QMF rant over :-) but hopefully it ties in with what I've also been 
saying about proton.
Thanks again for the update!!
Cheers,
Frase




On 18/01/13 16:51, Gordon Sim wrote:
> On 01/04/2013 10:07 AM, Fraser Adams wrote:
>> I really haven't seen an awful lot of publicity around proton and how it
>> relates to qpid so I think it'd be really good to get the documentation
>> around the relationships properly sorted to avoid a whole world of
>> confusion.
>
> You are quite right. Communication has been very poor and we need to 
> change that.
>
> On 01/03/2013 09:14 AM, Fraser Adams wrote:
>> BTW what *exactly* is Proton, I've seen a few references to it but
>> nothing is especially clear to me, is Proton the name for the AMQP v1.0
>> version of Qpid? How does it differ from say Qpid v0.20? Will Proton end
>> up being Qpid v1.0?
>
> Let me try and lay out my own understanding. Forgive me if it seems a 
> little long-winded, it isn't really that different from what William 
> described...
>
> Qpid is an openly governed community that seeks to aid adoption of 
> AMQP through collaborative development of various pieces of open 
> sourced software. Historically that has included two different 
> brokers, various client libraries and management tools. Although the 
> different parts should combine well, not every user will want or need 
> every component. Given the purpose of AMQP is to enable 
> interoperability and choice, it is important that individual 
> components from Qpid can be combined with other software and systems 
> in a flexible manner.
>
> Proton is a new component under the Qpid umbrella - a set of 
> components is perhaps more accurate as there is both a java and c 
> version and some swig wrappers. It is an AMQP 1.0 toolkit.
>
> Part of this is a 'protocol engine'; an implementation of the rules 
> specified by AMQP 1.0 in a manner that is independent of any threading 
> model or IO mechanism. This makes it easier to integrate in diverse 
> contexts including existing brokers and messaging clients and not 
> limited to just those within Qpid.
>
> One of the uses has been to add AMQP 1.0 support to existing Qpid 
> components, such as the work I have been doing to add 1.0 support to 
> the qpid::messaging API in c++ and to the qpidd (i.e. c++) broker. 
> There has also been work to add 1.0 support to ActiveMQ, and that also 
> uses the (java version) of the proton protocol engine.
>
> Being independent of any IO framework is important for one set of use 
> cases, but it does mean that the protocol engine isn't really usable 
> 'by itself'. It was recognised that adoption of 1.0 would be aided by 
> a simple, self contained library allowing sending and receiving of 
> messages.
>
> To that end the messenger API was added to the proton toolkit. This 
> ties network IO into the protocol engine, exposing a simple 
> abstraction of an input and output queue for messages, each message 
> containing an address to which it is to be delivered. It manages 
> connections, session and links under the covers. At present it is a 
> blocking API though support for non-blocking use is likely to evolve 
> as well.
>
> AMQP 1.0 is fully symmetric (i.e a peer protocol) as opposed to its 
> asymmetric (i.e. client-server) predecessors. This means it can be 
> used with or without intermediaries such as brokers. The messenger API 
> allows an application to listen for incoming connections as well as 
> initiating its own outgoing connections. It can therefore be used for 
> simple, direct communication as well as communication through 
> intermediaries. (You can also for example use a messenger based 
> application to accept connections from a qpid::messaging or even a JMS 
> client).
>
> Proton is on its own release cycle at present, reflecting the early 
> stage in its lifecycle. So far all other Qpid components have been on 
> the same release cycle. So the 0.20 release includes qpidd, 
> qpid-server (i.e. the qpid java broker), a JMS client, the c++ 
> qpid::messaging library (and the deprecated qpid::client library), the 
> python qpid.messaging library (and older deprecated libraries), QMF 
> tools, a JMX console etc etc.
>
> The 0.3 release of proton on the other hand contains c and java 
> libraries offering the protocol engine and messenger APIs (the java 
> version of the messenger API has some outstanding issues still) as 
> well as wrappers for python, perl and ruby. The 0.20 release of qpidd 
> and qpid::messaging has an optional dependency on the c version of 
> proton, which if available enables 1.0 support in those components. 
> The subsequent release of those two components will contain enhanced 
> support.
>
> So, qpid::messaging isn't going away. A key goal for that API was to 
> allow smooth transition to AMQP 1.0 and that is still something I am 
> working towards (0.20 represents the first step). Support for JMS 
> likewise will continue. However the messenger API is in my opinion a 
> novel approach that is may also be interesting for many cases. 
> Ultimately I think the different APIs offer different styles. While 
> there may be certain things that can't be done in one or other, often 
> it may come down to personal preference.
>
> Ultimately the vision behind AMQP is all about choice; enabling the 
> emergence of a rich palette of interoperable pieces from which diverse 
> solutions can be flexibly composed. AMQP 1.0 in my opinion expands the 
> opportunities here and rightly Qpid is innovating and evolving to 
> exploit those.
>
> These are exciting times! We do however need more open communication 
> (and better documentation ;-).
>
> --Gordon.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org