You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk> on 2013/10/09 20:22:02 UTC
Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid Dispatch
Router component
Hey all,
The thread below on the dev list has prompted me to ask something that
I've tentatively mentioned before, but am still a bit embarrassed to
raise 'cause it probably makes me seem a bit stupid :-( here goes
anyway.....
So I've kind of held off going down the AMQP 1.0 path partly due to lack
of time, but also partly due to lack of understanding of how it "all
hangs together", the new website helps a bit - but TBH I'm still left
scratching my head somewhat.
I'll try to explain:
Now I know that Proton is intended to be a component usable beyond just
the Qpid "product set", but there's a "protocol engine" and a "messenger
API" and I'm not even that clear on the relationship between the two of
those - for example could one use the protocol engine completely
independently (is there an engine API?) or is the messenger API intended
to be the lowest "unit of currency", what would be the benefit using the
raw engine?
Then beyond that there's the relationship with say qpidd and
qpid::messaging. Now I'm aware that when the Proton libraries are
detected qpidd and qpid::messaging get built with Proton support, I'm
"guessing" that in that case the relationship analogous to that of
qpid::client where qpid::client was the low level AMQP speaking API and
qpid::messaging provides a higher level abstraction, so I *think* that's
the relationship with Proton there - but I'm not sure? Is the proton API
close to the AMQP 1.0 specification in say the way that qpid::client was?
But then there's more nuance, so I'm aware that with AMQP 1.0 there's a
more peer-to-peer relationship and indeed the Proton tests seem to have
msgr-recv and msgr-send talking directly to each other without a broker.
So that leads me to ask the question what's the relationship with the
broker - in other words what services are provided in messenger, what
are enhanced in qpid::messaging and what are layered on top of that via
the broker (and how does the addressing and routing work?).
Some examples of where I'm befuddled include how does subscription work
at a peer to peer level? For example I think that exchange nodes are
only something I've heard discussed in the context of qpidd and
similarly I think the same is true of message selectors, so does Proton
only provide low level network connectivity and data serialisation (and
possibly single client queue) and all the other stuff needed for
connecting a network of clients are part of the broker services.
I suppose what I'm really asking is what "services" are provided at each
"layer" of the Qpid "stack" - clearly you can do useful stuff with just
Proton - but what stuff and what are the limits? What would you then get
from qpid:messaging and what then does the broker throw into the mix.
Are there any diagrams that illustrate this sort of relationship?
The dispatch router adds yet more nuance into the mix. From my (limited)
understanding it seems to offer at least some of the same services as
the broker - but I'm not quite sure what. In my case I've got a very
large federated topology and I have lots of left hand systems feeding in
to fewer systems towards the right. Given that it's only on the right
hand side broker that I have lots of consumers doing complex
subscriptions and the rest of the brokers are employing fairly simple
queue routes I'm thinking that the dispatch router could ultimately be
something to "tidy up" the left hand side of my system - but I'm not
quite sure.
Apologies if these seem silly questions, I'm sure that the answers are
obvious to those who've been involved at the architectural stages, but
ultimately from my perspective the overall holistic architecture isn't
totally clear.
Even at a basic level I've not actually noticed anything in the
programming book
http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html
that seems to mention even how to connect via AMQP 1.0 vice 0.10. I
think that it has been mentioned on the mailing list by Gordon so I'm
sure I could dig the info out, but is it missing from the docs (or am I
just not looking hard enough). On a similar note for Proton the
msgr-send and msgr-recv examples are fine as far as it goes, but I'm
thinking that to figure out how to do anything more complex my best bet
is likely to be to "reverse engineer" the qpid::messaging bindings - I
can't see anything obvious for how to send a map for example. I'm
guessing that Proton is just as erm "nuanced" as qpid::client, so really
powerful and flexible, but you have to know what you're doing to get the
best (say performance) out of it, the API documentation looks pretty
decent to be fair but I'm not sure that's enough to help me drive it
really effectively.
On top of that there seems to be a growing number of JMS clients,
there's the original AMQP 0.10, there's an AMQP 1.0 one in the main Qpid
tree and there's a separate Proton based AMQP 1.0 one that's a separate
component (in a similar vein to Proton). I can see that the increased
modularisation is a good thing and I assume that at some point the
original AMQP 1.0 JMS client will be deprecated in favour of the Proton
based one, but at the moment it's all a bit confusing without anything
that describes the relationship between then. I'm gleaning what little
knowledge I have out of a range of threads on the mailing list and I've
probably missed something.
I'm sorry if this comes across in any way as critical in email form,
it's really not intended to, I'm just keen to finally make a proper
start on my AMQP 1.0 journey and to be honest I feel a little out of my
depth at the moment :-(
Blame Ted for prompting me to write this ;->
Cheers,
Frase
On 09/10/13 17:20, Rob Godfrey wrote:
> Hi Ted,
>
> I think before we make this a full sub project, it would be good to have
> clarity on exactly the proposed scope of Dispatch, how it is expected to
> interact with other components within Qpid, or within wider AMQP networks.
> I think in retrospect we didn't do this clearly enough with Proton (for
> example).
>
> Moreover I would personally like to understand which AMQP standards it will
> be looking to implement, and which not. For instance I notice this line in
> the docs for Dispatch:
>
> *Address**Description* /_local/agentThe management agent on the attached
> router/container. This address would be used by an endpoint that is a
> management client/console/tool wishing to access management data from the
> attached container.
> Which doesn't seem to conform with the proposed management specification
> for AMQP, nor does the document make any mention of how dispatch is to be
> managed.
>
>
> Cheers,
> Rob
>
>
> On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
>
>> The AMQP Router project (Qpid Dispatch, announced previously on the user
>> list) is gaining in community interest and is nearing the point where a
>> first release is appropriate. In preparation for a release, I proposethat
>> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
>> projects. This involves:
>>
>> 1. Moving the code from qpid/extras to
>> http://svn.apache.org/repos/**asf/qpid/dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
>> ,
>> 2. Requesting, by vote, the creation of a JIRA project to track its
>> issues and releases.
>>
>> Unless there are objections, I will move forward with the above two tasks.
>>
>> Regards,
>>
>> -Ted
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid
Dispatch Router component
Posted by William Henry <wh...@redhat.com>.
Ted,
Can someone add your text to the community web pages and link parts to appropriate pages?
i.e. can we use Frase's and your posts to augment the community pages rather than just leaving here in list archives?
William
----- Original Message -----
> Frase,
>
> This is an excellent post, and I believe quite relevant. I'll try to
> address your questions at an abstract level rather than point-by-point.
> Your confusion is not unique, but quite justified.
>
> AMQP 1.0 is simply a wire-level protocol specification for symmetric
> point-to-point data communication. It has very advanced semantic
> capabilities that are inspired by middleware messaging. AMQP is, in my
> opinion, a game changer for distributed computing. It opens up
> possibilities that go way beyond client/broker messaging. In fact, it
> holds the potential to revolutionize the much broader world of data
> networking and distributed systems because it offers capabilities that
> TCP/IP and HTTP can't touch.
>
> Apache Qpid is the premier open source implementation of AMQP. In the
> AMQP 0-{8,9,10} era, Qpid was pretty much just another
> middleware-oriented-messaging (MOM) system based on the emerging
> standard. With the ratification of AMQP 1.0, which is significantly
> different (and superior) to the predecessor specifications, Qpid is
> undergoing a bit of a transition.
>
> We have, of course, updated our brokers and clients to use the new
> specification and thus continue to serve the MOM use cases.
>
> The Proton project was created for the purpose of promoting AMQP and
> making it easy to integrate into many different environments (I believe
> we've lost sight of that purpose somewhat).
>
> The Dispatch project was created for the purpose of exploring how AMQP
> can be applied in non-broker networks. Dispatch is a form of advanced
> interconnect based on AMQP with the goal of building large-scale and
> high-performance networks that take advantage of the unique capabilities
> of the protocol. It is intended to extend the reach and usefulness of
> brokers.
>
> There are a lot of new ideas floating around, some of them overlapping.
> I think Qpid is a perfect place to explore and develop new technologies
> based on AMQP. This will cause some confusion and force us to work at
> articulating what we are doing and thinking, and it will be people like
> you who will prompt the important discussions.
>
> Regards,
>
> -Ted
>
>
> On 10/09/2013 02:22 PM, Fraser Adams wrote:
> > Hey all,
> > The thread below on the dev list has prompted me to ask something that
> > I've tentatively mentioned before, but am still a bit embarrassed to
> > raise 'cause it probably makes me seem a bit stupid :-( here goes
> > anyway.....
> >
> >
> > So I've kind of held off going down the AMQP 1.0 path partly due to
> > lack of time, but also partly due to lack of understanding of how it
> > "all hangs together", the new website helps a bit - but TBH I'm still
> > left scratching my head somewhat.
> >
> > I'll try to explain:
> >
> > Now I know that Proton is intended to be a component usable beyond
> > just the Qpid "product set", but there's a "protocol engine" and a
> > "messenger API" and I'm not even that clear on the relationship
> > between the two of those - for example could one use the protocol
> > engine completely independently (is there an engine API?) or is the
> > messenger API intended to be the lowest "unit of currency", what would
> > be the benefit using the raw engine?
> >
> > Then beyond that there's the relationship with say qpidd and
> > qpid::messaging. Now I'm aware that when the Proton libraries are
> > detected qpidd and qpid::messaging get built with Proton support, I'm
> > "guessing" that in that case the relationship analogous to that of
> > qpid::client where qpid::client was the low level AMQP speaking API
> > and qpid::messaging provides a higher level abstraction, so I *think*
> > that's the relationship with Proton there - but I'm not sure? Is the
> > proton API close to the AMQP 1.0 specification in say the way that
> > qpid::client was?
> >
> >
> > But then there's more nuance, so I'm aware that with AMQP 1.0 there's
> > a more peer-to-peer relationship and indeed the Proton tests seem to
> > have msgr-recv and msgr-send talking directly to each other without a
> > broker. So that leads me to ask the question what's the relationship
> > with the broker - in other words what services are provided in
> > messenger, what are enhanced in qpid::messaging and what are layered
> > on top of that via the broker (and how does the addressing and routing
> > work?).
> >
> > Some examples of where I'm befuddled include how does subscription
> > work at a peer to peer level? For example I think that exchange nodes
> > are only something I've heard discussed in the context of qpidd and
> > similarly I think the same is true of message selectors, so does
> > Proton only provide low level network connectivity and data
> > serialisation (and possibly single client queue) and all the other
> > stuff needed for connecting a network of clients are part of the
> > broker services.
> >
> > I suppose what I'm really asking is what "services" are provided at
> > each "layer" of the Qpid "stack" - clearly you can do useful stuff
> > with just Proton - but what stuff and what are the limits? What would
> > you then get from qpid:messaging and what then does the broker throw
> > into the mix. Are there any diagrams that illustrate this sort of
> > relationship?
> >
> > The dispatch router adds yet more nuance into the mix. From my
> > (limited) understanding it seems to offer at least some of the same
> > services as the broker - but I'm not quite sure what. In my case I've
> > got a very large federated topology and I have lots of left hand
> > systems feeding in to fewer systems towards the right. Given that it's
> > only on the right hand side broker that I have lots of consumers doing
> > complex subscriptions and the rest of the brokers are employing fairly
> > simple queue routes I'm thinking that the dispatch router could
> > ultimately be something to "tidy up" the left hand side of my system -
> > but I'm not quite sure.
> >
> > Apologies if these seem silly questions, I'm sure that the answers are
> > obvious to those who've been involved at the architectural stages, but
> > ultimately from my perspective the overall holistic architecture isn't
> > totally clear.
> >
> > Even at a basic level I've not actually noticed anything in the
> > programming book
> > http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html
> > that seems to mention even how to connect via AMQP 1.0 vice 0.10. I
> > think that it has been mentioned on the mailing list by Gordon so I'm
> > sure I could dig the info out, but is it missing from the docs (or am
> > I just not looking hard enough). On a similar note for Proton the
> > msgr-send and msgr-recv examples are fine as far as it goes, but I'm
> > thinking that to figure out how to do anything more complex my best
> > bet is likely to be to "reverse engineer" the qpid::messaging bindings
> > - I can't see anything obvious for how to send a map for example. I'm
> > guessing that Proton is just as erm "nuanced" as qpid::client, so
> > really powerful and flexible, but you have to know what you're doing
> > to get the best (say performance) out of it, the API documentation
> > looks pretty decent to be fair but I'm not sure that's enough to help
> > me drive it really effectively.
> >
> > On top of that there seems to be a growing number of JMS clients,
> > there's the original AMQP 0.10, there's an AMQP 1.0 one in the main
> > Qpid tree and there's a separate Proton based AMQP 1.0 one that's a
> > separate component (in a similar vein to Proton). I can see that the
> > increased modularisation is a good thing and I assume that at some
> > point the original AMQP 1.0 JMS client will be deprecated in favour of
> > the Proton based one, but at the moment it's all a bit confusing
> > without anything that describes the relationship between then. I'm
> > gleaning what little knowledge I have out of a range of threads on the
> > mailing list and I've probably missed something.
> >
> > I'm sorry if this comes across in any way as critical in email form,
> > it's really not intended to, I'm just keen to finally make a proper
> > start on my AMQP 1.0 journey and to be honest I feel a little out of
> > my depth at the moment :-(
> >
> > Blame Ted for prompting me to write this ;->
> >
> > Cheers,
> > Frase
> >
> >
> > On 09/10/13 17:20, Rob Godfrey wrote:
> >> Hi Ted,
> >>
> >> I think before we make this a full sub project, it would be good to have
> >> clarity on exactly the proposed scope of Dispatch, how it is expected to
> >> interact with other components within Qpid, or within wider AMQP
> >> networks.
> >> I think in retrospect we didn't do this clearly enough with Proton (for
> >> example).
> >>
> >> Moreover I would personally like to understand which AMQP standards
> >> it will
> >> be looking to implement, and which not. For instance I notice this
> >> line in
> >> the docs for Dispatch:
> >>
> >> *Address**Description* /_local/agentThe management agent on the attached
> >> router/container. This address would be used by an endpoint that is a
> >> management client/console/tool wishing to access management data from
> >> the
> >> attached container.
> >> Which doesn't seem to conform with the proposed management specification
> >> for AMQP, nor does the document make any mention of how dispatch is
> >> to be
> >> managed.
> >>
> >>
> >> Cheers,
> >> Rob
> >>
> >>
> >> On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
> >>
> >>> The AMQP Router project (Qpid Dispatch, announced previously on the
> >>> user
> >>> list) is gaining in community interest and is nearing the point where a
> >>> first release is appropriate. In preparation for a release, I
> >>> proposethat
> >>> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
> >>> projects. This involves:
> >>>
> >>> 1. Moving the code from qpid/extras to
> >>> http://svn.apache.org/repos/**asf/qpid/dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
> >>> ,
> >>> 2. Requesting, by vote, the creation of a JIRA project to track its
> >>> issues and releases.
> >>>
> >>> Unless there are objections, I will move forward with the above two
> >>> tasks.
> >>>
> >>> Regards,
> >>>
> >>> -Ted
> >>>
> >>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid
Dispatch Router component
Posted by Gordon Sim <gs...@redhat.com>.
On 10/09/2013 08:04 PM, Ted Ross wrote:
> There are a lot of new ideas floating around, some of them overlapping.
> I think Qpid is a perfect place to explore and develop new technologies
> based on AMQP. This will cause some confusion and force us to work at
> articulating what we are doing and thinking, and it will be people like
> you who will prompt the important discussions.
Very well said, I heartily agree!
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid
Dispatch Router component
Posted by Ted Ross <tr...@redhat.com>.
Frase,
This is an excellent post, and I believe quite relevant. I'll try to
address your questions at an abstract level rather than point-by-point.
Your confusion is not unique, but quite justified.
AMQP 1.0 is simply a wire-level protocol specification for symmetric
point-to-point data communication. It has very advanced semantic
capabilities that are inspired by middleware messaging. AMQP is, in my
opinion, a game changer for distributed computing. It opens up
possibilities that go way beyond client/broker messaging. In fact, it
holds the potential to revolutionize the much broader world of data
networking and distributed systems because it offers capabilities that
TCP/IP and HTTP can't touch.
Apache Qpid is the premier open source implementation of AMQP. In the
AMQP 0-{8,9,10} era, Qpid was pretty much just another
middleware-oriented-messaging (MOM) system based on the emerging
standard. With the ratification of AMQP 1.0, which is significantly
different (and superior) to the predecessor specifications, Qpid is
undergoing a bit of a transition.
We have, of course, updated our brokers and clients to use the new
specification and thus continue to serve the MOM use cases.
The Proton project was created for the purpose of promoting AMQP and
making it easy to integrate into many different environments (I believe
we've lost sight of that purpose somewhat).
The Dispatch project was created for the purpose of exploring how AMQP
can be applied in non-broker networks. Dispatch is a form of advanced
interconnect based on AMQP with the goal of building large-scale and
high-performance networks that take advantage of the unique capabilities
of the protocol. It is intended to extend the reach and usefulness of
brokers.
There are a lot of new ideas floating around, some of them overlapping.
I think Qpid is a perfect place to explore and develop new technologies
based on AMQP. This will cause some confusion and force us to work at
articulating what we are doing and thinking, and it will be people like
you who will prompt the important discussions.
Regards,
-Ted
On 10/09/2013 02:22 PM, Fraser Adams wrote:
> Hey all,
> The thread below on the dev list has prompted me to ask something that
> I've tentatively mentioned before, but am still a bit embarrassed to
> raise 'cause it probably makes me seem a bit stupid :-( here goes
> anyway.....
>
>
> So I've kind of held off going down the AMQP 1.0 path partly due to
> lack of time, but also partly due to lack of understanding of how it
> "all hangs together", the new website helps a bit - but TBH I'm still
> left scratching my head somewhat.
>
> I'll try to explain:
>
> Now I know that Proton is intended to be a component usable beyond
> just the Qpid "product set", but there's a "protocol engine" and a
> "messenger API" and I'm not even that clear on the relationship
> between the two of those - for example could one use the protocol
> engine completely independently (is there an engine API?) or is the
> messenger API intended to be the lowest "unit of currency", what would
> be the benefit using the raw engine?
>
> Then beyond that there's the relationship with say qpidd and
> qpid::messaging. Now I'm aware that when the Proton libraries are
> detected qpidd and qpid::messaging get built with Proton support, I'm
> "guessing" that in that case the relationship analogous to that of
> qpid::client where qpid::client was the low level AMQP speaking API
> and qpid::messaging provides a higher level abstraction, so I *think*
> that's the relationship with Proton there - but I'm not sure? Is the
> proton API close to the AMQP 1.0 specification in say the way that
> qpid::client was?
>
>
> But then there's more nuance, so I'm aware that with AMQP 1.0 there's
> a more peer-to-peer relationship and indeed the Proton tests seem to
> have msgr-recv and msgr-send talking directly to each other without a
> broker. So that leads me to ask the question what's the relationship
> with the broker - in other words what services are provided in
> messenger, what are enhanced in qpid::messaging and what are layered
> on top of that via the broker (and how does the addressing and routing
> work?).
>
> Some examples of where I'm befuddled include how does subscription
> work at a peer to peer level? For example I think that exchange nodes
> are only something I've heard discussed in the context of qpidd and
> similarly I think the same is true of message selectors, so does
> Proton only provide low level network connectivity and data
> serialisation (and possibly single client queue) and all the other
> stuff needed for connecting a network of clients are part of the
> broker services.
>
> I suppose what I'm really asking is what "services" are provided at
> each "layer" of the Qpid "stack" - clearly you can do useful stuff
> with just Proton - but what stuff and what are the limits? What would
> you then get from qpid:messaging and what then does the broker throw
> into the mix. Are there any diagrams that illustrate this sort of
> relationship?
>
> The dispatch router adds yet more nuance into the mix. From my
> (limited) understanding it seems to offer at least some of the same
> services as the broker - but I'm not quite sure what. In my case I've
> got a very large federated topology and I have lots of left hand
> systems feeding in to fewer systems towards the right. Given that it's
> only on the right hand side broker that I have lots of consumers doing
> complex subscriptions and the rest of the brokers are employing fairly
> simple queue routes I'm thinking that the dispatch router could
> ultimately be something to "tidy up" the left hand side of my system -
> but I'm not quite sure.
>
> Apologies if these seem silly questions, I'm sure that the answers are
> obvious to those who've been involved at the architectural stages, but
> ultimately from my perspective the overall holistic architecture isn't
> totally clear.
>
> Even at a basic level I've not actually noticed anything in the
> programming book
> http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html
> that seems to mention even how to connect via AMQP 1.0 vice 0.10. I
> think that it has been mentioned on the mailing list by Gordon so I'm
> sure I could dig the info out, but is it missing from the docs (or am
> I just not looking hard enough). On a similar note for Proton the
> msgr-send and msgr-recv examples are fine as far as it goes, but I'm
> thinking that to figure out how to do anything more complex my best
> bet is likely to be to "reverse engineer" the qpid::messaging bindings
> - I can't see anything obvious for how to send a map for example. I'm
> guessing that Proton is just as erm "nuanced" as qpid::client, so
> really powerful and flexible, but you have to know what you're doing
> to get the best (say performance) out of it, the API documentation
> looks pretty decent to be fair but I'm not sure that's enough to help
> me drive it really effectively.
>
> On top of that there seems to be a growing number of JMS clients,
> there's the original AMQP 0.10, there's an AMQP 1.0 one in the main
> Qpid tree and there's a separate Proton based AMQP 1.0 one that's a
> separate component (in a similar vein to Proton). I can see that the
> increased modularisation is a good thing and I assume that at some
> point the original AMQP 1.0 JMS client will be deprecated in favour of
> the Proton based one, but at the moment it's all a bit confusing
> without anything that describes the relationship between then. I'm
> gleaning what little knowledge I have out of a range of threads on the
> mailing list and I've probably missed something.
>
> I'm sorry if this comes across in any way as critical in email form,
> it's really not intended to, I'm just keen to finally make a proper
> start on my AMQP 1.0 journey and to be honest I feel a little out of
> my depth at the moment :-(
>
> Blame Ted for prompting me to write this ;->
>
> Cheers,
> Frase
>
>
> On 09/10/13 17:20, Rob Godfrey wrote:
>> Hi Ted,
>>
>> I think before we make this a full sub project, it would be good to have
>> clarity on exactly the proposed scope of Dispatch, how it is expected to
>> interact with other components within Qpid, or within wider AMQP
>> networks.
>> I think in retrospect we didn't do this clearly enough with Proton (for
>> example).
>>
>> Moreover I would personally like to understand which AMQP standards
>> it will
>> be looking to implement, and which not. For instance I notice this
>> line in
>> the docs for Dispatch:
>>
>> *Address**Description* /_local/agentThe management agent on the attached
>> router/container. This address would be used by an endpoint that is a
>> management client/console/tool wishing to access management data from
>> the
>> attached container.
>> Which doesn't seem to conform with the proposed management specification
>> for AMQP, nor does the document make any mention of how dispatch is
>> to be
>> managed.
>>
>>
>> Cheers,
>> Rob
>>
>>
>> On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
>>
>>> The AMQP Router project (Qpid Dispatch, announced previously on the
>>> user
>>> list) is gaining in community interest and is nearing the point where a
>>> first release is appropriate. In preparation for a release, I
>>> proposethat
>>> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
>>> projects. This involves:
>>>
>>> 1. Moving the code from qpid/extras to
>>> http://svn.apache.org/repos/**asf/qpid/dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
>>> ,
>>> 2. Requesting, by vote, the creation of a JIRA project to track its
>>> issues and releases.
>>>
>>> Unless there are objections, I will move forward with the above two
>>> tasks.
>>>
>>> Regards,
>>>
>>> -Ted
>>>
>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid
Dispatch Router component
Posted by Rob Godfrey <ro...@gmail.com>.
This is an excellent post which I think highlights the need for us to
properly define the scopes of our current components, their roadmaps, and
the vision we have for Qpid and AMQP 1.0.
I know some people have already replied in thread, but would it seem like a
good idea if we used these questions to help us formulate some sort of
documentation for our site where we can provide answers both to others
looking to get started with the project, but also perhaps to ourselves?
-- Rob
On 9 October 2013 20:22, Fraser Adams <fr...@blueyonder.co.uk> wrote:
> Hey all,
> The thread below on the dev list has prompted me to ask something that
> I've tentatively mentioned before, but am still a bit embarrassed to raise
> 'cause it probably makes me seem a bit stupid :-( here goes anyway.....
>
>
> So I've kind of held off going down the AMQP 1.0 path partly due to lack
> of time, but also partly due to lack of understanding of how it "all hangs
> together", the new website helps a bit - but TBH I'm still left scratching
> my head somewhat.
>
> I'll try to explain:
>
> Now I know that Proton is intended to be a component usable beyond just
> the Qpid "product set", but there's a "protocol engine" and a "messenger
> API" and I'm not even that clear on the relationship between the two of
> those - for example could one use the protocol engine completely
> independently (is there an engine API?) or is the messenger API intended to
> be the lowest "unit of currency", what would be the benefit using the raw
> engine?
>
> Then beyond that there's the relationship with say qpidd and
> qpid::messaging. Now I'm aware that when the Proton libraries are detected
> qpidd and qpid::messaging get built with Proton support, I'm "guessing"
> that in that case the relationship analogous to that of qpid::client where
> qpid::client was the low level AMQP speaking API and qpid::messaging
> provides a higher level abstraction, so I *think* that's the relationship
> with Proton there - but I'm not sure? Is the proton API close to the AMQP
> 1.0 specification in say the way that qpid::client was?
>
>
> But then there's more nuance, so I'm aware that with AMQP 1.0 there's a
> more peer-to-peer relationship and indeed the Proton tests seem to have
> msgr-recv and msgr-send talking directly to each other without a broker. So
> that leads me to ask the question what's the relationship with the broker -
> in other words what services are provided in messenger, what are enhanced
> in qpid::messaging and what are layered on top of that via the broker (and
> how does the addressing and routing work?).
>
> Some examples of where I'm befuddled include how does subscription work at
> a peer to peer level? For example I think that exchange nodes are only
> something I've heard discussed in the context of qpidd and similarly I
> think the same is true of message selectors, so does Proton only provide
> low level network connectivity and data serialisation (and possibly single
> client queue) and all the other stuff needed for connecting a network of
> clients are part of the broker services.
>
> I suppose what I'm really asking is what "services" are provided at each
> "layer" of the Qpid "stack" - clearly you can do useful stuff with just
> Proton - but what stuff and what are the limits? What would you then get
> from qpid:messaging and what then does the broker throw into the mix. Are
> there any diagrams that illustrate this sort of relationship?
>
> The dispatch router adds yet more nuance into the mix. From my (limited)
> understanding it seems to offer at least some of the same services as the
> broker - but I'm not quite sure what. In my case I've got a very large
> federated topology and I have lots of left hand systems feeding in to fewer
> systems towards the right. Given that it's only on the right hand side
> broker that I have lots of consumers doing complex subscriptions and the
> rest of the brokers are employing fairly simple queue routes I'm thinking
> that the dispatch router could ultimately be something to "tidy up" the
> left hand side of my system - but I'm not quite sure.
>
> Apologies if these seem silly questions, I'm sure that the answers are
> obvious to those who've been involved at the architectural stages, but
> ultimately from my perspective the overall holistic architecture isn't
> totally clear.
>
> Even at a basic level I've not actually noticed anything in the
> programming book http://qpid.apache.org/**releases/qpid-0.24/**
> programming/book/index.html<http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html>that seems to mention even how to connect via AMQP 1.0 vice 0.10. I think
> that it has been mentioned on the mailing list by Gordon so I'm sure I
> could dig the info out, but is it missing from the docs (or am I just not
> looking hard enough). On a similar note for Proton the msgr-send and
> msgr-recv examples are fine as far as it goes, but I'm thinking that to
> figure out how to do anything more complex my best bet is likely to be to
> "reverse engineer" the qpid::messaging bindings - I can't see anything
> obvious for how to send a map for example. I'm guessing that Proton is just
> as erm "nuanced" as qpid::client, so really powerful and flexible, but you
> have to know what you're doing to get the best (say performance) out of it,
> the API documentation looks pretty decent to be fair but I'm not sure
> that's enough to help me drive it really effectively.
>
> On top of that there seems to be a growing number of JMS clients, there's
> the original AMQP 0.10, there's an AMQP 1.0 one in the main Qpid tree and
> there's a separate Proton based AMQP 1.0 one that's a separate component
> (in a similar vein to Proton). I can see that the increased modularisation
> is a good thing and I assume that at some point the original AMQP 1.0 JMS
> client will be deprecated in favour of the Proton based one, but at the
> moment it's all a bit confusing without anything that describes the
> relationship between then. I'm gleaning what little knowledge I have out of
> a range of threads on the mailing list and I've probably missed something.
>
> I'm sorry if this comes across in any way as critical in email form, it's
> really not intended to, I'm just keen to finally make a proper start on my
> AMQP 1.0 journey and to be honest I feel a little out of my depth at the
> moment :-(
>
> Blame Ted for prompting me to write this ;->
>
> Cheers,
> Frase
>
>
> On 09/10/13 17:20, Rob Godfrey wrote:
>
>> Hi Ted,
>>
>> I think before we make this a full sub project, it would be good to have
>> clarity on exactly the proposed scope of Dispatch, how it is expected to
>> interact with other components within Qpid, or within wider AMQP networks.
>> I think in retrospect we didn't do this clearly enough with Proton (for
>> example).
>>
>> Moreover I would personally like to understand which AMQP standards it
>> will
>> be looking to implement, and which not. For instance I notice this line
>> in
>> the docs for Dispatch:
>>
>> *Address**Description* /_local/agentThe management agent on the attached
>> router/container. This address would be used by an endpoint that is a
>> management client/console/tool wishing to access management data from the
>> attached container.
>> Which doesn't seem to conform with the proposed management specification
>> for AMQP, nor does the document make any mention of how dispatch is to be
>> managed.
>>
>>
>> Cheers,
>> Rob
>>
>>
>> On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
>>
>> The AMQP Router project (Qpid Dispatch, announced previously on the user
>>> list) is gaining in community interest and is nearing the point where a
>>> first release is appropriate. In preparation for a release, I proposethat
>>> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
>>> projects. This involves:
>>>
>>> 1. Moving the code from qpid/extras to
>>> http://svn.apache.org/repos/****asf/qpid/dispatch<http://svn.apache.org/repos/**asf/qpid/dispatch>
>>> <http://svn.**apache.org/repos/asf/qpid/**dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
>>> >
>>> ,
>>> 2. Requesting, by vote, the creation of a JIRA project to track its
>>> issues and releases.
>>>
>>> Unless there are objections, I will move forward with the above two
>>> tasks.
>>>
>>> Regards,
>>>
>>> -Ted
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid
Dispatch Router component
Posted by Ted Ross <tr...@redhat.com>.
Now, to answer your actual questions... See inline below.
On 10/09/2013 02:22 PM, Fraser Adams wrote:
> Hey all,
> The thread below on the dev list has prompted me to ask something that
> I've tentatively mentioned before, but am still a bit embarrassed to
> raise 'cause it probably makes me seem a bit stupid :-( here goes
> anyway.....
>
>
> So I've kind of held off going down the AMQP 1.0 path partly due to
> lack of time, but also partly due to lack of understanding of how it
> "all hangs together", the new website helps a bit - but TBH I'm still
> left scratching my head somewhat.
>
> I'll try to explain:
>
> Now I know that Proton is intended to be a component usable beyond
> just the Qpid "product set", but there's a "protocol engine" and a
> "messenger API" and I'm not even that clear on the relationship
> between the two of those - for example could one use the protocol
> engine completely independently (is there an engine API?) or is the
> messenger API intended to be the lowest "unit of currency", what would
> be the benefit using the raw engine?
Proton is a multi-layered thing. There is an "engine" API, but it's
neither documented nor stabilized. The Messenger API is built on top of
the engine API and is better documented. There's also a separate driver
component in Proton for integration into a particular run-time environment.
The benefit of using the raw engine API is that it gives detailed access
to the specific parts of the AMQP protocol. If you need to work
directly with connections, sessions, links and deliveries, the engine
API is what you need.
Dispatch is built on the engine API (the C language one) and bypasses
all the services offered by Messenger. There's a layer within Dispatch
that provides multi-threaded access to engine (which is by itself,
non-thread-safe).
>
> Then beyond that there's the relationship with say qpidd and
> qpid::messaging. Now I'm aware that when the Proton libraries are
> detected qpidd and qpid::messaging get built with Proton support, I'm
> "guessing" that in that case the relationship analogous to that of
> qpid::client where qpid::client was the low level AMQP speaking API
> and qpid::messaging provides a higher level abstraction, so I *think*
> that's the relationship with Proton there - but I'm not sure? Is the
> proton API close to the AMQP 1.0 specification in say the way that
> qpid::client was?
This is sort of true. The qpid::client API was the original one and it
was tied pretty tightly to the protocol. The qpid::messaging APIs (both
C++ and Python) were created prior to AMQP 1.0 for the purpose of being
forward compatible to the new protocol. It is simply an implementation
detail that qpid::messaging is layered on qpid::client.
Messenger and qpid::messaging are peers. They are alternative APIs for
messaging. Messenger is more abstract in that it deals with messages
and hides connections, sessions, links, etc. Qpid::messaging is closer
to JMS in its design.
>
>
> But then there's more nuance, so I'm aware that with AMQP 1.0 there's
> a more peer-to-peer relationship and indeed the Proton tests seem to
> have msgr-recv and msgr-send talking directly to each other without a
> broker. So that leads me to ask the question what's the relationship
> with the broker - in other words what services are provided in
> messenger, what are enhanced in qpid::messaging and what are layered
> on top of that via the broker (and how does the addressing and routing
> work?).
This is a pretty big question... AMQP 1.0 is a point-to-point,
symmetric protocol. Because of this, there is no requirement for a
broker. It allows for the full semantics of message exchange to occur
between endpoints regardless of whether one (or both) of them is a broker.
Regarding addressing and routing. This is a topic of research. Dispatch
is one research vehicle for these topics. It takes a particular
approach to addressing and routing that may be different from other
approaches. We should explore them all.
>
> Some examples of where I'm befuddled include how does subscription
> work at a peer to peer level? For example I think that exchange nodes
> are only something I've heard discussed in the context of qpidd and
> similarly I think the same is true of message selectors, so does
> Proton only provide low level network connectivity and data
> serialisation (and possibly single client queue) and all the other
> stuff needed for connecting a network of clients are part of the
> broker services.
Pub-sub in a brokerless environment is another topic of research.
Dispatch is attempting to solve this problem. I expect others will as well.
>
> I suppose what I'm really asking is what "services" are provided at
> each "layer" of the Qpid "stack" - clearly you can do useful stuff
> with just Proton - but what stuff and what are the limits? What would
> you then get from qpid:messaging and what then does the broker throw
> into the mix. Are there any diagrams that illustrate this sort of
> relationship?
I think the Qpid "stack" is still being envisioned. I see a network of
routers interconnecting brokers and clients. Communication patterns are
varied and use brokers only when Queueing (capital Q as opposed to
merely buffering) and persistence are required. The brokers and routers
are both optional and can be deployed if needed.
>
> The dispatch router adds yet more nuance into the mix. From my
> (limited) understanding it seems to offer at least some of the same
> services as the broker - but I'm not quite sure what. In my case I've
> got a very large federated topology and I have lots of left hand
> systems feeding in to fewer systems towards the right. Given that it's
> only on the right hand side broker that I have lots of consumers doing
> complex subscriptions and the rest of the brokers are employing fairly
> simple queue routes I'm thinking that the dispatch router could
> ultimately be something to "tidy up" the left hand side of my system -
> but I'm not quite sure.
I think your description is correct. Dispatch is for routing. The
broker is for "brokering". Dispatch will never accept a message itself,
only pass it through to an endpoint or broker that can take ownership of
the message.
>
> Apologies if these seem silly questions, I'm sure that the answers are
> obvious to those who've been involved at the architectural stages, but
> ultimately from my perspective the overall holistic architecture isn't
> totally clear.
>
> Even at a basic level I've not actually noticed anything in the
> programming book
> http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html
> that seems to mention even how to connect via AMQP 1.0 vice 0.10. I
> think that it has been mentioned on the mailing list by Gordon so I'm
> sure I could dig the info out, but is it missing from the docs (or am
> I just not looking hard enough). On a similar note for Proton the
> msgr-send and msgr-recv examples are fine as far as it goes, but I'm
> thinking that to figure out how to do anything more complex my best
> bet is likely to be to "reverse engineer" the qpid::messaging bindings
> - I can't see anything obvious for how to send a map for example. I'm
> guessing that Proton is just as erm "nuanced" as qpid::client, so
> really powerful and flexible, but you have to know what you're doing
> to get the best (say performance) out of it, the API documentation
> looks pretty decent to be fair but I'm not sure that's enough to help
> me drive it really effectively.
If you set the connection-option of "protocol" to "amqp1.0", you will
enable the new protocol.
>
> On top of that there seems to be a growing number of JMS clients,
> there's the original AMQP 0.10, there's an AMQP 1.0 one in the main
> Qpid tree and there's a separate Proton based AMQP 1.0 one that's a
> separate component (in a similar vein to Proton). I can see that the
> increased modularisation is a good thing and I assume that at some
> point the original AMQP 1.0 JMS client will be deprecated in favour of
> the Proton based one, but at the moment it's all a bit confusing
> without anything that describes the relationship between then. I'm
> gleaning what little knowledge I have out of a range of threads on the
> mailing list and I've probably missed something.
Hopefully there will be fewer JMS clients in the future. The 0-*
versions will be legacy and eventually become obsolete. The "prototype"
client for AMQP 1.0 will be deprecated once the proton-based one is
complete.
>
> I'm sorry if this comes across in any way as critical in email form,
> it's really not intended to, I'm just keen to finally make a proper
> start on my AMQP 1.0 journey and to be honest I feel a little out of
> my depth at the moment :-(
>
> Blame Ted for prompting me to write this ;->
>
> Cheers,
> Frase
>
>
> On 09/10/13 17:20, Rob Godfrey wrote:
>> Hi Ted,
>>
>> I think before we make this a full sub project, it would be good to have
>> clarity on exactly the proposed scope of Dispatch, how it is expected to
>> interact with other components within Qpid, or within wider AMQP
>> networks.
>> I think in retrospect we didn't do this clearly enough with Proton (for
>> example).
>>
>> Moreover I would personally like to understand which AMQP standards
>> it will
>> be looking to implement, and which not. For instance I notice this
>> line in
>> the docs for Dispatch:
>>
>> *Address**Description* /_local/agentThe management agent on the attached
>> router/container. This address would be used by an endpoint that is a
>> management client/console/tool wishing to access management data from
>> the
>> attached container.
>> Which doesn't seem to conform with the proposed management specification
>> for AMQP, nor does the document make any mention of how dispatch is
>> to be
>> managed.
>>
>>
>> Cheers,
>> Rob
>>
>>
>> On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
>>
>>> The AMQP Router project (Qpid Dispatch, announced previously on the
>>> user
>>> list) is gaining in community interest and is nearing the point where a
>>> first release is appropriate. In preparation for a release, I
>>> proposethat
>>> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
>>> projects. This involves:
>>>
>>> 1. Moving the code from qpid/extras to
>>> http://svn.apache.org/repos/**asf/qpid/dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
>>> ,
>>> 2. Requesting, by vote, the creation of a JIRA project to track its
>>> issues and releases.
>>>
>>> Unless there are objections, I will move forward with the above two
>>> tasks.
>>>
>>> Regards,
>>>
>>> -Ted
>>>
>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid
Dispatch Router component
Posted by Robbie Gemmell <ro...@gmail.com>.
Adding some more specific bits to what Ted covered in his mail.....though I
now see he just beat me to doing so himself, oh well :P
On 9 October 2013 19:22, Fraser Adams <fr...@blueyonder.co.uk> wrote:
> Hey all,
> The thread below on the dev list has prompted me to ask something that
> I've tentatively mentioned before, but am still a bit embarrassed to raise
> 'cause it probably makes me seem a bit stupid :-( here goes anyway.....
>
>
> So I've kind of held off going down the AMQP 1.0 path partly due to lack
> of time, but also partly due to lack of understanding of how it "all hangs
> together", the new website helps a bit - but TBH I'm still left scratching
> my head somewhat.
>
> I'll try to explain:
>
> Now I know that Proton is intended to be a component usable beyond just
> the Qpid "product set", but there's a "protocol engine" and a "messenger
> API" and I'm not even that clear on the relationship between the two of
> those - for example could one use the protocol engine completely
> independently (is there an engine API?) or is the messenger API intended to
> be the lowest "unit of currency", what would be the benefit using the raw
> engine?
>
The engine is effectively an AMQP 1.0 implementation that can be used to
add support to a [potentially existing] product without requiring
implementation of AMQP 1.0 from scrach, e.g. this is what ActiveMQ and Qpid
C++ brokers do for their 1.0 support. Messenger is a form of API for
applications doing messaging that uses the underlying engine as its AMQP
1.0 implementation, and I'll now defer to others to better describe it
beyond that :)
>
> Then beyond that there's the relationship with say qpidd and
> qpid::messaging. Now I'm aware that when the Proton libraries are detected
> qpidd and qpid::messaging get built with Proton support, I'm "guessing"
> that in that case the relationship analogous to that of qpid::client where
> qpid::client was the low level AMQP speaking API and qpid::messaging
> provides a higher level abstraction, so I *think* that's the relationship
> with Proton there - but I'm not sure? Is the proton API close to the AMQP
> 1.0 specification in say the way that qpid::client was?
>
Similar, but not entirely the same, as Qpid::client was the actual C++
client interface for some time but as mentioned was close to the
specification itself, and so a higher level messaging API was later created
to abstract away some of that detail and provide a replacement messaging
API that could allow easier transition to newer protocols. Proton is in
part something that could be used to underneath API X to make it support
AMQP 1.0 (e.g. the qpid messaging API), but there also isnt necessarily a
client API above it at and it could jsut be getting used directly (e.g
possibly with the brokers).
>
>
> But then there's more nuance, so I'm aware that with AMQP 1.0 there's a
> more peer-to-peer relationship and indeed the Proton tests seem to have
> msgr-recv and msgr-send talking directly to each other without a broker. So
> that leads me to ask the question what's the relationship with the broker -
> in other words what services are provided in messenger, what are enhanced
> in qpid::messaging and what are layered on top of that via the broker (and
> how does the addressing and routing work?).
>
You can stick a broker in the middle and the above just becomes a
client<->broker<->client model like you are used to. Depending on what you
are actually doing though, you might not need or want a broker, and that
field is likely to develop as time goes. For example, brokers can do things
like provide Queues, but that is essentially just something with a name
that accepts your messages...you migiht want to be sending to something
with a name that isnt a queue on a broker.
>
> Some examples of where I'm befuddled include how does subscription work at
> a peer to peer level? For example I think that exchange nodes are only
> something I've heard discussed in the context of qpidd and similarly I
> think the same is true of message selectors, so does Proton only provide
> low level network connectivity and data serialisation (and possibly single
> client queue) and all the other stuff needed for connecting a network of
> clients are part of the broker services.
>
> I suppose what I'm really asking is what "services" are provided at each
> "layer" of the Qpid "stack" - clearly you can do useful stuff with just
> Proton - but what stuff and what are the limits? What would you then get
> from qpid:messaging and what then does the broker throw into the mix. Are
> there any diagrams that illustrate this sort of relationship?
>
I think all I would say here would be to echo my points above: you can end
up with something that very closely resembles what you are used to, i.e.
clients talking directly to brokers and using queues and topics, or
depending on your needs you might want to end up with something different.
>
> The dispatch router adds yet more nuance into the mix. From my (limited)
> understanding it seems to offer at least some of the same services as the
> broker - but I'm not quite sure what. In my case I've got a very large
> federated topology and I have lots of left hand systems feeding in to fewer
> systems towards the right. Given that it's only on the right hand side
> broker that I have lots of consumers doing complex subscriptions and the
> rest of the brokers are employing fairly simple queue routes I'm thinking
> that the dispatch router could ultimately be something to "tidy up" the
> left hand side of my system - but I'm not quite sure.
>
> Apologies if these seem silly questions, I'm sure that the answers are
> obvious to those who've been involved at the architectural stages, but
> ultimately from my perspective the overall holistic architecture isn't
> totally clear.
>
> Even at a basic level I've not actually noticed anything in the
> programming book http://qpid.apache.org/**releases/qpid-0.24/**
> programming/book/index.html<http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html>that seems to mention even how to connect via AMQP 1.0 vice 0.10. I think
> that it has been mentioned on the mailing list by Gordon so I'm sure I
> could dig the info out, but is it missing from the docs (or am I just not
> looking hard enough). On a similar note for Proton the msgr-send and
> msgr-recv examples are fine as far as it goes, but I'm thinking that to
> figure out how to do anything more complex my best bet is likely to be to
> "reverse engineer" the qpid::messaging bindings - I can't see anything
> obvious for how to send a map for example. I'm guessing that Proton is just
> as erm "nuanced" as qpid::client, so really powerful and flexible, but you
> have to know what you're doing to get the best (say performance) out of it,
> the API documentation looks pretty decent to be fair but I'm not sure
> that's enough to help me drive it really effectively.
>
> On top of that there seems to be a growing number of JMS clients, there's
> the original AMQP 0.10, there's an AMQP 1.0 one in the main Qpid tree and
> there's a separate Proton based AMQP 1.0 one that's a separate component
> (in a similar vein to Proton). I can see that the increased modularisation
> is a good thing and I assume that at some point the original AMQP 1.0 JMS
> client will be deprecated in favour of the Proton based one, but at the
> moment it's all a bit confusing without anything that describes the
> relationship between then. I'm gleaning what little knowledge I have out of
> a range of threads on the mailing list and I've probably missed something.
>
>
The existing AMQP 1.0 JMS 1.1 client was essentially the result of
prototyping work to prove out the 1.0 implementation as it was being
authored, before Proton was conceived. The new client you mention is a
separate implementation getting underway which will support the new JMS 2.0
spec released since then, be built around Proton now that it exists, and
develop in tandem with work to create a documented JMS mapping for AMQP 1.0
within the OASIS AMQP Bindings & Mapping TC. It will indeed eventually
replace the prior 1.0 client.
> I'm sorry if this comes across in any way as critical in email form, it's
> really not intended to, I'm just keen to finally make a proper start on my
> AMQP 1.0 journey and to be honest I feel a little out of my depth at the
> moment :-(
>
> Blame Ted for prompting me to write this ;->
>
> Cheers,
> Frase
>
>
> On 09/10/13 17:20, Rob Godfrey wrote:
>
>> Hi Ted,
>>
>> I think before we make this a full sub project, it would be good to have
>> clarity on exactly the proposed scope of Dispatch, how it is expected to
>> interact with other components within Qpid, or within wider AMQP networks.
>> I think in retrospect we didn't do this clearly enough with Proton (for
>> example).
>>
>> Moreover I would personally like to understand which AMQP standards it
>> will
>> be looking to implement, and which not. For instance I notice this line
>> in
>> the docs for Dispatch:
>>
>> *Address**Description* /_local/agentThe management agent on the attached
>> router/container. This address would be used by an endpoint that is a
>> management client/console/tool wishing to access management data from the
>> attached container.
>> Which doesn't seem to conform with the proposed management specification
>> for AMQP, nor does the document make any mention of how dispatch is to be
>> managed.
>>
>>
>> Cheers,
>> Rob
>>
>>
>> On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
>>
>> The AMQP Router project (Qpid Dispatch, announced previously on the user
>>> list) is gaining in community interest and is nearing the point where a
>>> first release is appropriate. In preparation for a release, I proposethat
>>> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
>>> projects. This involves:
>>>
>>> 1. Moving the code from qpid/extras to
>>> http://svn.apache.org/repos/****asf/qpid/dispatch<http://svn.apache.org/repos/**asf/qpid/dispatch>
>>> <http://svn.**apache.org/repos/asf/qpid/**dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
>>> >
>>> ,
>>> 2. Requesting, by vote, the creation of a JIRA project to track its
>>> issues and releases.
>>>
>>> Unless there are objections, I will move forward with the above two
>>> tasks.
>>>
>>> Regards,
>>>
>>> -Ted
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Qpid AMQP 1.0 - How does it all hang together? - was Re: Qpid
Dispatch Router component
Posted by William Henry <wh...@redhat.com>.
Great post! +1.
The community needs to do more to explain, document, and promote.
William
----- Original Message -----
> Hey all,
> The thread below on the dev list has prompted me to ask something that
> I've tentatively mentioned before, but am still a bit embarrassed to
> raise 'cause it probably makes me seem a bit stupid :-( here goes
> anyway.....
>
>
> So I've kind of held off going down the AMQP 1.0 path partly due to lack
> of time, but also partly due to lack of understanding of how it "all
> hangs together", the new website helps a bit - but TBH I'm still left
> scratching my head somewhat.
>
> I'll try to explain:
>
> Now I know that Proton is intended to be a component usable beyond just
> the Qpid "product set", but there's a "protocol engine" and a "messenger
> API" and I'm not even that clear on the relationship between the two of
> those - for example could one use the protocol engine completely
> independently (is there an engine API?) or is the messenger API intended
> to be the lowest "unit of currency", what would be the benefit using the
> raw engine?
>
> Then beyond that there's the relationship with say qpidd and
> qpid::messaging. Now I'm aware that when the Proton libraries are
> detected qpidd and qpid::messaging get built with Proton support, I'm
> "guessing" that in that case the relationship analogous to that of
> qpid::client where qpid::client was the low level AMQP speaking API and
> qpid::messaging provides a higher level abstraction, so I *think* that's
> the relationship with Proton there - but I'm not sure? Is the proton API
> close to the AMQP 1.0 specification in say the way that qpid::client was?
>
>
> But then there's more nuance, so I'm aware that with AMQP 1.0 there's a
> more peer-to-peer relationship and indeed the Proton tests seem to have
> msgr-recv and msgr-send talking directly to each other without a broker.
> So that leads me to ask the question what's the relationship with the
> broker - in other words what services are provided in messenger, what
> are enhanced in qpid::messaging and what are layered on top of that via
> the broker (and how does the addressing and routing work?).
>
> Some examples of where I'm befuddled include how does subscription work
> at a peer to peer level? For example I think that exchange nodes are
> only something I've heard discussed in the context of qpidd and
> similarly I think the same is true of message selectors, so does Proton
> only provide low level network connectivity and data serialisation (and
> possibly single client queue) and all the other stuff needed for
> connecting a network of clients are part of the broker services.
>
> I suppose what I'm really asking is what "services" are provided at each
> "layer" of the Qpid "stack" - clearly you can do useful stuff with just
> Proton - but what stuff and what are the limits? What would you then get
> from qpid:messaging and what then does the broker throw into the mix.
> Are there any diagrams that illustrate this sort of relationship?
>
> The dispatch router adds yet more nuance into the mix. From my (limited)
> understanding it seems to offer at least some of the same services as
> the broker - but I'm not quite sure what. In my case I've got a very
> large federated topology and I have lots of left hand systems feeding in
> to fewer systems towards the right. Given that it's only on the right
> hand side broker that I have lots of consumers doing complex
> subscriptions and the rest of the brokers are employing fairly simple
> queue routes I'm thinking that the dispatch router could ultimately be
> something to "tidy up" the left hand side of my system - but I'm not
> quite sure.
>
> Apologies if these seem silly questions, I'm sure that the answers are
> obvious to those who've been involved at the architectural stages, but
> ultimately from my perspective the overall holistic architecture isn't
> totally clear.
>
> Even at a basic level I've not actually noticed anything in the
> programming book
> http://qpid.apache.org/releases/qpid-0.24/programming/book/index.html
> that seems to mention even how to connect via AMQP 1.0 vice 0.10. I
> think that it has been mentioned on the mailing list by Gordon so I'm
> sure I could dig the info out, but is it missing from the docs (or am I
> just not looking hard enough). On a similar note for Proton the
> msgr-send and msgr-recv examples are fine as far as it goes, but I'm
> thinking that to figure out how to do anything more complex my best bet
> is likely to be to "reverse engineer" the qpid::messaging bindings - I
> can't see anything obvious for how to send a map for example. I'm
> guessing that Proton is just as erm "nuanced" as qpid::client, so really
> powerful and flexible, but you have to know what you're doing to get the
> best (say performance) out of it, the API documentation looks pretty
> decent to be fair but I'm not sure that's enough to help me drive it
> really effectively.
>
> On top of that there seems to be a growing number of JMS clients,
> there's the original AMQP 0.10, there's an AMQP 1.0 one in the main Qpid
> tree and there's a separate Proton based AMQP 1.0 one that's a separate
> component (in a similar vein to Proton). I can see that the increased
> modularisation is a good thing and I assume that at some point the
> original AMQP 1.0 JMS client will be deprecated in favour of the Proton
> based one, but at the moment it's all a bit confusing without anything
> that describes the relationship between then. I'm gleaning what little
> knowledge I have out of a range of threads on the mailing list and I've
> probably missed something.
>
> I'm sorry if this comes across in any way as critical in email form,
> it's really not intended to, I'm just keen to finally make a proper
> start on my AMQP 1.0 journey and to be honest I feel a little out of my
> depth at the moment :-(
>
> Blame Ted for prompting me to write this ;->
>
> Cheers,
> Frase
>
>
> On 09/10/13 17:20, Rob Godfrey wrote:
> > Hi Ted,
> >
> > I think before we make this a full sub project, it would be good to have
> > clarity on exactly the proposed scope of Dispatch, how it is expected to
> > interact with other components within Qpid, or within wider AMQP networks.
> > I think in retrospect we didn't do this clearly enough with Proton (for
> > example).
> >
> > Moreover I would personally like to understand which AMQP standards it will
> > be looking to implement, and which not. For instance I notice this line in
> > the docs for Dispatch:
> >
> > *Address**Description* /_local/agentThe management agent on the attached
> > router/container. This address would be used by an endpoint that is a
> > management client/console/tool wishing to access management data from the
> > attached container.
> > Which doesn't seem to conform with the proposed management specification
> > for AMQP, nor does the document make any mention of how dispatch is to be
> > managed.
> >
> >
> > Cheers,
> > Rob
> >
> >
> > On 9 October 2013 17:22, Ted Ross <tr...@redhat.com> wrote:
> >
> >> The AMQP Router project (Qpid Dispatch, announced previously on the user
> >> list) is gaining in community interest and is nearing the point where a
> >> first release is appropriate. In preparation for a release, I proposethat
> >> this sub-project follow the lead of both Proton and the AMQP1.0 JMS
> >> projects. This involves:
> >>
> >> 1. Moving the code from qpid/extras to
> >> http://svn.apache.org/repos/**asf/qpid/dispatch<http://svn.apache.org/repos/asf/qpid/dispatch>
> >> ,
> >> 2. Requesting, by vote, the creation of a JIRA project to track its
> >> issues and releases.
> >>
> >> Unless there are objections, I will move forward with the above two tasks.
> >>
> >> Regards,
> >>
> >> -Ted
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org