new team working with AMQP and Apache Qpid Proton

[Rob Nicholson <ro...@uk.ibm.com>]

Hello proton mailing list. 

I think that some folks here have noticed that within IBM we have an 
incubator project called MQ Light [1] which is using the AMQP 1.0 wire 
protocol and is making use of Apache Qpid Proton both standalone[1] and in 
our cloud PaaS incubator [2]. 
Up until this point we have been largely in listen mode on the mailing 
list but now we will be engaging with the community so we thought it would 
be polite to introduce ourselves, what we are doing and how we are 
thinking of engaging with you.

Currently we use the proton C messenger API in our client and the Java 
messaging API in our "server" which uses code derived from IBM's Websphere 
MQ products. 

We have raised some Jiras, we plan to raise more for some specific 
problems we have had, supplying patches which show how we worked around or 
addressed each problem we had. 

We also have some of more strategic queries the first of which are:

 - We are finding that we need to extend Messenger in order to make it 
capable of doing what we want it to. Is this valid or did you want to keep 
messenger really simple.   Should we just supply patches for these 
extensions also? 

-  We want to create language bindings for MQ Light which are _really_ 
easy to use by programmers in that language. Specifically we find these 
bindings need to understand our messaging model. AMQP does not have a 
preconceived idea of a messaging model. We would like to put all of the 
client code into open source but do these language bindings belong in the 
Proton project? If not, I suspect we'll create a separate project which 
has a dependency to proton. 

-  As we have consumed the Java engine API we have run up against some 
threading issues. This might be because we do not understand the threading 
model in the Engine.  At some point we'd like to have a design discussion 
with the community on the threading model in the Engine implementation. 


[1] https://www.ibmdw.net/messaging/mq-light/
[2] http://www.bluemix.net
[3] https://www.npmjs.org/package/mqlight


Regards,

Rob Nicholson
CEng MEng FBCS CITP


Phone: +44-1962-817517 | Mobile: +44-7843-368769
E-mail: rob_nicholson@uk.ibm.com
Find me on:   and within IBM on:   


Hursley Park
Hursley, SO212JN
United Kingdom


I
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: new team working with AMQP and Apache Qpid Proton

[Rafael Schloming <rh...@alum.mit.edu>]

Hello and welcome,

First off let me just say this is great news and please don't feel shy
about engaging. The whole point of Proton is to make it easy to speak AMQP
and if there is any way we can make it easier then we welcome the feedback.

I'll do my best to answer your questions below...

On Thu, Apr 17, 2014 at 8:36 AM, Rob Nicholson <ro...@uk.ibm.com>wrote:

> Hello proton mailing list.
>
> I think that some folks here have noticed that within IBM we have an
> incubator project called MQ Light [1] which is using the AMQP 1.0 wire
> protocol and is making use of Apache Qpid Proton both standalone[1] and in
> our cloud PaaS incubator [2].
> Up until this point we have been largely in listen mode on the mailing
> list but now we will be engaging with the community so we thought it would
> be polite to introduce ourselves, what we are doing and how we are thinking
> of engaging with you.
>
> Currently we use the proton C messenger API in our client and the Java
> messaging API in our "server" which uses code derived from IBM's Websphere
> MQ products.
>
> We have raised some Jiras, we plan to raise more for some specific
> problems we have had, supplying patches which show how we worked around or
> addressed each problem we had.
>
> We also have some of more strategic queries the first of which are:
>
>    -
>
>  - We are finding that we need to extend Messenger in order to make it
> capable of doing what we want it to. Is this valid or did you want to keep
> messenger really simple.   Should we just supply patches for these
> extensions also?
>

This is hard question to answer in general. Simplicity is definitely a
goal, however we want it to solve real use cases as well, and sometimes
those two things work against each other. Understanding your use cases is
probably key here, so patches would be a great start even if they just
serve to illustrate the use case.


>
> -  We want to create language bindings for MQ Light which are _really_
> easy to use by programmers in that language. Specifically we find these
> bindings need to understand our messaging model. AMQP does not have a
> preconceived idea of a messaging model. We would like to put all of the
> client code into open source but do these language bindings belong in the
>  Proton project? If not, I suspect we'll create a separate project which
> has a dependency to proton.
>

At the lowest layer (i.e. the protocol engine) proton cannot make any
assumptions that would limit interop with any conforming AMQP
implementation. In other words if you are using the engine to speak AMQP
you need to be able to express and interact with the full range of protocol
capabilities. But, if layered properly, there is room for simplified models
built on top of the engine, e.g. for the sake of simplifying the user
experience Messenger makes some choices about how it uses the protocol that
limit the full range of interactions you can express purely through the
Messenger API. I'd say this kind of thing is fine so long as it is generic,
i.e. not limited to interacting with just one implementation that behaves
in an overly specific way.

There is also a contrib area in the Proton tree that we have the option to
use for the kind of thing you're describing, and this isn't necessarily an
either or thing. We could start out with something in contrib and factor
parts of it into core and then move the rest of it to an external project
if that ever became warranted. Again, it's hard to say exactly what makes
sense without specifics, but hopefully you get a sense of the options.


>
> -  As we have consumed the Java engine API we have run up against some
> threading issues. This might be because we do not understand the threading
> model in the Engine.  At some point we'd like to have a design discussion
> with the community on the threading model in the Engine implementation.


I'm eager to hear what your issues are. I'm aware of some common problems
people have had and there are some ideas in the works to address those, so
it would be good to take into account your experiences sooner rather than
later.

--Rafael

Re: new team working with AMQP and Apache Qpid Proton

[Ted Ross <tr...@redhat.com>]

Rob and team,

Thanks for the introduction and welcome to the project.

The mission of Qpid Proton is to encourage adoption of AMQP and to make
it as easy as possible to integrate AMQP into anything and everything.
As such, I think your goals are in alignment with the project's goals.

I'll address your specific questions in-line below:

On 04/17/2014 08:36 AM, Rob Nicholson wrote:
> Hello proton mailing list. 
> 
> I think that some folks here have noticed that within IBM we have an 
> incubator project called MQ Light [1] which is using the AMQP 1.0 wire 
> protocol and is making use of Apache Qpid Proton both standalone[1] and in 
> our cloud PaaS incubator [2]. 
> Up until this point we have been largely in listen mode on the mailing 
> list but now we will be engaging with the community so we thought it would 
> be polite to introduce ourselves, what we are doing and how we are 
> thinking of engaging with you.
> 
> Currently we use the proton C messenger API in our client and the Java 
> messaging API in our "server" which uses code derived from IBM's Websphere 
> MQ products. 
> 
> We have raised some Jiras, we plan to raise more for some specific 
> problems we have had, supplying patches which show how we worked around or 
> addressed each problem we had. 
> 
> We also have some of more strategic queries the first of which are:
> 
>  - We are finding that we need to extend Messenger in order to make it 
> capable of doing what we want it to. Is this valid or did you want to keep 
> messenger really simple.   Should we just supply patches for these 
> extensions also? 

Others have also identified gaps in Messenger, specifically in allowing
it to be integrated into an existing program's poll loop and to run
asynchronously.

We always welcome patches, but this is a case where a discussion of the
gaps you've identified is appropriate so we can arrive at a solution
that solves the real problems without muddying the API.

> 
> -  We want to create language bindings for MQ Light which are _really_ 
> easy to use by programmers in that language. Specifically we find these 
> bindings need to understand our messaging model. AMQP does not have a 
> preconceived idea of a messaging model. We would like to put all of the 
> client code into open source but do these language bindings belong in the 
> Proton project? If not, I suspect we'll create a separate project which 
> has a dependency to proton. 

It is in AMQP's best interest for there to be freely available bindings
that are easy to use.  It is also consistent with Qpid's purpose to have
them hosted here.

I'd like to understand more about the implications of your messaging
model and whether clients built for that model can be used with other
messaging services.  Such a conversation would help determine whether
these clients belong in Proton or as a separate sub-project.

> 
> -  As we have consumed the Java engine API we have run up against some 
> threading issues. This might be because we do not understand the threading 
> model in the Engine.  At some point we'd like to have a design discussion 
> with the community on the threading model in the Engine implementation. 

There is at least one team within Red Hat that has also identified
threading and performance issues with the Proton-J engine.  We should
work together to identify and fix the problems.

> 
> 
> [1] https://www.ibmdw.net/messaging/mq-light/
> [2] http://www.bluemix.net
> [3] https://www.npmjs.org/package/mqlight
> 
> 
> Regards,
> 
> Rob Nicholson
> CEng MEng FBCS CITP
> 
> 

-Ted