You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk> on 2014/08/29 15:55:38 UTC
Use of subject for routing - moved thread to user list from earlier
private discussion.
All,
I've moved this thread to the user list. It started out on the back of
an email conversation between Rob Godfrey. Gordon Sim and and myself
when I couldn't get my QMF messages routed on the Java Broker.
The original context was that I was sending messages to
qmf.default.direct and setting the message subject to "broker", this is
pretty much exactly what the python qpid tools such as qpid-config do.
in AMQP 0.10 this works fine because the subject (which in AMQP 0.10 is
a user property called qpid.subject) gets treated as the routing key
when sent to a direct exchange.
When I did my AMQP 1.0 JavaScript qpid-config port I set the AMQP 1.0
subject (which is now an immutable message property and not an
application property) to "broker" and sent to
amqp://guest:guest@localhost:5673/qmf.default.direct
This worked fine with the C++ broker, but with the Java Broker the
message wasn't routed and in subsequent conversations with Rob and
Gordon I discovered that the Java Broker doesn't route on direct
exchanges based on Subject rather it first tries to use to "to" - in
other words if I sent to
amqp://guest:guest@localhost:5673/qmf.default.direct/broker it'd work or
if the to isn't set it uses the application property "routing-key".
So basically this thread is around an inconsistency between the C++ and
Java Brokers where the C++ Broker continues to route on Subject for AMQP
1.0 but the Java Broker does not.
Rob and Gordon can fill in if I've missed anything, but it's probably
best to share this discussion on the user list.
Frase
On 29/08/14 14:03, Rob Godfrey wrote:
> I think in the context of where AMQP 1.0 is now, routing by subject is
> somewhat counter-intuitive, though when we started we did see the
> subject field in a routing-key sort of role.
From my perspective I'd say exactly the opposite, that is routing by
subject seems entirely intuitive and is also consistent with what
happened with AMQP 0.10. Given the use of subject as the defacto routing
key on all of the python tools I don't think I'm alone in feeling that's
intuitive. I'm pretty sure that the subject is used for routing on topic
exchanges so I'm not sure why you think its counter-intuitive on derect
exchanges?
>
> In terms of current behaviour - the legacy filters in the Java Broker
> just set up the bindings between the queue and the exchange and thus
> use the same routing as previously described (i.e. they're not
> currently using subject). Inside the Java Broker an exchange routes
> based on an abstract notion of the "routingAddress" of a message
> instance. In AMQP 0.x that's taken to be the routing key. In 1.0 it
> is as I described in my previous mail.
>
> If we collectively (and this discussion should really be on the users
> list) think that 0.x exchanges should route on subject when routing a
> 1.0 message then I'm happy to change the default behaviour of the Java
> Broker - except for the "no name" exchange which will route on "to"
> because that's what we're defining on the addressing spec. I'll then
> probably add options to the exchange implementation so that a user can
> configure on a per exchange basis to route on something else (except
> for the non default amq.* exchanges).
>
> -- Rob
>
>
> On 29 August 2014 13:45, Gordon Sim <gsim@redhat.com
> <ma...@redhat.com>> wrote:
>
> On 08/29/2014 11:49 AM, Rob Godfrey wrote:
>
> For the moment I guess we'll just have to live with the C++
> and Java
> Brokers having different ideas about how to route 1.0 messages
> at exchanges.
>
>
> The legacy-amqp filters are defined to work on the subject when
> using the exchange as the source. I thought the java broker
> supported those?
>
> If I create a receiving link from amq.direct with a
> apache.org:legacy-amqp-direct-binding:string of foo, then send a
> message to amq.direct whose subject is foo, my expectation would
> be that the receiver will receive that.
>
>
Re: Use of subject for routing - moved thread to user list from
earlier private discussion.
Posted by Rob Godfrey <ro...@gmail.com>.
On 29 August 2014 19:02, Gordon Sim <gs...@redhat.com> wrote:
> On 08/29/2014 04:57 PM, Rob Godfrey wrote:
>
>> On 29 August 2014 17:39, Gordon Sim <gs...@redhat.com> wrote:
>>
>>> I'm not opposed to change. However I don't think the use of subject for
>>> what might be termed 'routing' is obviously wrong or unintended by the
>>> current spec and therefore an illogical choice for anything 'designed
>>> around AMQP 1.0'.
>>>
>>>
>>> I'm not saying it's wrong...
>>
>
> I think you are (and that in itself I don't object to):
>
I make a distinction between wrong as in "incorrect" and bad as in
sub-optimal :-)
>
> On 08/29/2014 03:42 PM, Rob Godfrey wrote:
>>
>>> On 29 August 2014 16:24, Gordon Sim <gs...@redhat.com> wrote:
>>>
>>>> On 08/29/2014 03:12 PM, Rob Godfrey wrote:
>>>>
>>>>> If an application is designed around using AMQP 1.0, I would not
>>>>> normally
>>>>> expect them to be putting routing information in the subject field.
>>>>>
>>>>
>>>> That depends on what you define to be routing information. I think it
>>>> is a
>>>> perfectly sensible approach to use the subject to indicate some sort of
>>>> logical category and then have the broker be configured to 'route'
>>>> messages
>>>> according to that logical category.
>>>>
>>>
>>> I think that's one choice you could make, certainly. I just think it's a
>>> bad choice for the general case.
>>>
>>
> You also appeared to be implying that the definitions of the 'to' and
> 'subject' fields in the specification backed this view up. That's where I
> disagreed with you.
>
> I can accept that using the subject may not be the best approach for other
> reasons, i.e. that it is wrong. What I objected to is the notion that it is
> *obviously* wrong based on a reading of the specification and therefore
> somehow not really AMQP 1.0.
To be honest I was surprised at the definition of subject when I reread it
as I expected it (from my woolly memory of these things) to more obviously
support the notion of it as a primary routing attributes. It is very likely
that you have a better memory than me and thus were less surprised at the
definition :-)
>
>
> And indeed originally I was very much of the
>> opinion that subject was a surrogate for the routing-key in 0.10 (which is
>> why I originally implemented filters in that way).
>>
>
> I think those legacy filters turned out to be a mistake. I quite agree
> that a convention around the address (either of the source/target, or of
> the 'to' field on the message) leads to a simpler (yet also more flexible
> and powerful) solution for the use cases the direct and topic exchanges
> were intended to support.
>
>
Yes - probably my fault :-(
> The description in the spec for the 'to' field is again in hindsight not a
> good one (not entirely sure its the best choice of name either), and it may
> indeed have been better to leave the subject field out entirely.
>
Also quite possibly my fault :-( Agreed on all points though.
>
> [...]
>
> However I think the potentially
>> more contentious question is actually message translation.
>>
>
> I doubt it's that contentious :-)
>
>
> Behaviour of
>> exchanges can be considered a property of a given exchange/node/address
>> whatever you want to call it. Message translation is externally visible
>> and obviously we should be striving to have commonality there. Given an
>> AMQP 1.0 message how does one translate that into an AMQP 0-9-1 or 0-10
>> message (along with their associated delivery headers). In particular for
>> reply to... I am translating an AMQP 0-10 exchange + routing-key combo
>> into
>> and AMQP 1.0 address of "exchange/routing-key" ... How does the C++ broker
>> translate the replyTo address?
>>
>
> It does the same thing.
>
> I've actually described the mapping for the qpid::messaging API and the
> c++ brokers behaviours onto 1.0 on this list in past threads and in a
> readme: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/AMQP_1.0?
> view=markup.
>
> Obviously I think this is a sensible mapping... but the issue here is that
the reply to in a 0.x sense may not work as expected if the 1.0 user
happens to innocently set the subject of their reply message to, say, "I
like tomatoes". That is an 0.x client sends a message with reply-to
exchange: foo , routing-key: bar ... this message is received by an AMQP
1.0 client which sees the reply to as "foo/bar". A reply message is sent
and it is routed at the foo exchange... but because the message happens to
be about the senders love of tomatoes, the message will never reach the
intended destination because subject is used in preference to the (parsed)
to address.
> My hope in doing so was that it would allow others to comment and
> criticise the choices made, suggesting alternatives or enhancements (indeed
> Fraser's feedback was very valuable) and that this would encourage more
> open analysis and discussion around AMQP 1.0 extended semantics which I
> feel is sorely lacking. Sadly this failed.
>
>
Think of it as delayed success... I think a lot of discussion will sadly,
but necessarily, only happen as people stumble upon actual issues
-- Rob
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Use of subject for routing - moved thread to user list from earlier
private discussion.
Posted by Gordon Sim <gs...@redhat.com>.
On 08/29/2014 04:57 PM, Rob Godfrey wrote:
> On 29 August 2014 17:39, Gordon Sim <gs...@redhat.com> wrote:
>> I'm not opposed to change. However I don't think the use of subject for
>> what might be termed 'routing' is obviously wrong or unintended by the
>> current spec and therefore an illogical choice for anything 'designed
>> around AMQP 1.0'.
>>
>>
> I'm not saying it's wrong...
I think you are (and that in itself I don't object to):
> On 08/29/2014 03:42 PM, Rob Godfrey wrote:
>> On 29 August 2014 16:24, Gordon Sim <gs...@redhat.com> wrote:
>>> On 08/29/2014 03:12 PM, Rob Godfrey wrote:
>>>> If an application is designed around using AMQP 1.0, I would not normally
>>>> expect them to be putting routing information in the subject field.
>>>
>>> That depends on what you define to be routing information. I think it is a
>>> perfectly sensible approach to use the subject to indicate some sort of
>>> logical category and then have the broker be configured to 'route' messages
>>> according to that logical category.
>>
>> I think that's one choice you could make, certainly. I just think it's a
>> bad choice for the general case.
You also appeared to be implying that the definitions of the 'to' and
'subject' fields in the specification backed this view up. That's where
I disagreed with you.
I can accept that using the subject may not be the best approach for
other reasons, i.e. that it is wrong. What I objected to is the notion
that it is *obviously* wrong based on a reading of the specification and
therefore somehow not really AMQP 1.0.
> And indeed originally I was very much of the
> opinion that subject was a surrogate for the routing-key in 0.10 (which is
> why I originally implemented filters in that way).
I think those legacy filters turned out to be a mistake. I quite agree
that a convention around the address (either of the source/target, or of
the 'to' field on the message) leads to a simpler (yet also more
flexible and powerful) solution for the use cases the direct and topic
exchanges were intended to support.
The description in the spec for the 'to' field is again in hindsight not
a good one (not entirely sure its the best choice of name either), and
it may indeed have been better to leave the subject field out entirely.
[...]
> However I think the potentially
> more contentious question is actually message translation.
I doubt it's that contentious :-)
> Behaviour of
> exchanges can be considered a property of a given exchange/node/address
> whatever you want to call it. Message translation is externally visible
> and obviously we should be striving to have commonality there. Given an
> AMQP 1.0 message how does one translate that into an AMQP 0-9-1 or 0-10
> message (along with their associated delivery headers). In particular for
> reply to... I am translating an AMQP 0-10 exchange + routing-key combo into
> and AMQP 1.0 address of "exchange/routing-key" ... How does the C++ broker
> translate the replyTo address?
It does the same thing.
I've actually described the mapping for the qpid::messaging API and the
c++ brokers behaviours onto 1.0 on this list in past threads and in a
readme:
http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/AMQP_1.0?view=markup.
My hope in doing so was that it would allow others to comment and
criticise the choices made, suggesting alternatives or enhancements
(indeed Fraser's feedback was very valuable) and that this would
encourage more open analysis and discussion around AMQP 1.0 extended
semantics which I feel is sorely lacking. Sadly this failed.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Use of subject for routing - moved thread to user list from
earlier private discussion.
Posted by Rob Godfrey <ro...@gmail.com>.
On 29 August 2014 17:39, Gordon Sim <gs...@redhat.com> wrote:
> On 08/29/2014 03:42 PM, Rob Godfrey wrote:
>
>> The to field indicates the destination in terms of what the sender wants
>> to
>> send it to. It says nothing about where it actually arrives.
>>
>
> "The to field identifies the node that is the intended
> destination of the message. On any given transfer this
> might not be the node at the receiving end of the link."
>
>
> "Examples of AMQP nodes are producers, consumers, and
> queues."
>
> I can certainly see that it is useful to 'redirect' messages, i.e. to
> deliver them to nodes other than that 'identified' as the 'destination'. I
> can also certainly see that you might want a looser interpretation of the
> to field to be some sort of abstract or logical description of 'message
> content and purpose' rather than the identity of a specific node.
> I'm not opposed to change. However I don't think the use of subject for
> what might be termed 'routing' is obviously wrong or unintended by the
> current spec and therefore an illogical choice for anything 'designed
> around AMQP 1.0'.
>
>
I'm not saying it's wrong... And indeed originally I was very much of the
opinion that subject was a surrogate for the routing-key in 0.10 (which is
why I originally implemented filters in that way).
> As an example let's take a simple news service. Senders send in some news,
> recipients subscribe to receive news. Messages are classified with regard
> to the type of news, e.g. sports, weather, politics whatever.
>
> In such a case I think it would be a perfectly understandable choice given
> the current spec to specify the type of news, the classification, in the
> subject. It is after all clearly indicating in a brief way the content and
> purpose of the news.
>
>
Absolutely, though you could also envisage a scenario where there were
other dimensions to the message (such as the geographical location)... and
a user might only want to subscribe to say women's football sport news in
Malmo, (Sweden, Europe). I think subject's utility as such a field is
really dependent on how easily it is surfaced in various APIs.
> The c++ broker uses the subject when routing through pre 1.0 style
> exchanges because that's what the only document on the subject - namely the
> legacy filters, defined 'to allow a consistent mechanism for addressing
> legacy AMQP Exchanges over AMQP 1.0' - specifies.
>
> There may be good reasons to change this and reinterpret the property
> descriptions. I don't think the wording of the current specification is one
> of them.
>
>
>
As previously stated - the change to the behaviour of the existing
exchanges in the Java Broker was unintentional, and I'm happy to restore
(as soon as I can figure out how without breaking the ability to use things
like the AMQP management node from 0-9-1). However I think the potentially
more contentious question is actually message translation. Behaviour of
exchanges can be considered a property of a given exchange/node/address
whatever you want to call it. Message translation is externally visible
and obviously we should be striving to have commonality there. Given an
AMQP 1.0 message how does one translate that into an AMQP 0-9-1 or 0-10
message (along with their associated delivery headers). In particular for
reply to... I am translating an AMQP 0-10 exchange + routing-key combo into
and AMQP 1.0 address of "exchange/routing-key" ... How does the C++ broker
translate the replyTo address?
-- Rob
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Use of subject for routing - moved thread to user list from earlier
private discussion.
Posted by Gordon Sim <gs...@redhat.com>.
On 08/29/2014 03:42 PM, Rob Godfrey wrote:
> The to field indicates the destination in terms of what the sender wants to
> send it to. It says nothing about where it actually arrives.
"The to field identifies the node that is the intended
destination of the message. On any given transfer this
might not be the node at the receiving end of the link."
"Examples of AMQP nodes are producers, consumers, and
queues."
I can certainly see that it is useful to 'redirect' messages, i.e. to
deliver them to nodes other than that 'identified' as the 'destination'.
I can also certainly see that you might want a looser interpretation of
the to field to be some sort of abstract or logical description of
'message content and purpose' rather than the identity of a specific node.
I'm not opposed to change. However I don't think the use of subject for
what might be termed 'routing' is obviously wrong or unintended by the
current spec and therefore an illogical choice for anything 'designed
around AMQP 1.0'.
As an example let's take a simple news service. Senders send in some
news, recipients subscribe to receive news. Messages are classified with
regard to the type of news, e.g. sports, weather, politics whatever.
In such a case I think it would be a perfectly understandable choice
given the current spec to specify the type of news, the classification,
in the subject. It is after all clearly indicating in a brief way the
content and purpose of the news.
The c++ broker uses the subject when routing through pre 1.0 style
exchanges because that's what the only document on the subject - namely
the legacy filters, defined 'to allow a consistent mechanism for
addressing legacy AMQP Exchanges over AMQP 1.0' - specifies.
There may be good reasons to change this and reinterpret the property
descriptions. I don't think the wording of the current specification is
one of them.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Use of subject for routing - moved thread to user list from
earlier private discussion.
Posted by Rob Godfrey <ro...@gmail.com>.
On 29 August 2014 16:24, Gordon Sim <gs...@redhat.com> wrote:
> On 08/29/2014 03:12 PM, Rob Godfrey wrote:
>
>> In general, however, I think the subject is not the best fit for a mapping
>> of the semantics of the 0.x "routing-key". The AMQP 1.0 spec defines
>> subject as "A common field for summary information about the message
>> content and purpose", rather than giving it and explicit or implicit
>> routing semantics.
>>
>
> In my view, the 'routing key' in the old exchanges model is intended to be
> exactly such a logical categorisation of the messages content or purpose.
>
>
>
But the definition is "summary"... such as "Use of subject for routing -
moved thread to user list from earlier private discussion." is a summary
for this thread. It does not in any way imply structured.
> The "to" field is defined as "identifies the node that
>> is the intended destination of the message" which definitely implies some
>> routing semantics but does not make them explicit.
>>
>
> As you state, the to field identifies the destination. That is not what
> the old exchange types are about in my view. In the typical use cases for
> that old model, the sender doesn't want to describe who gets the queue,
> they merely want to indicate the type or purpose of the messages. It is
> then through the bindings that this type is mapped to the appropriate
> recipient(s).
The to field indicates the destination in terms of what the sender wants to
send it to. It says nothing about where it actually arrives. How an
address is translated to an ultimate recipient of the message is a property
of the network, that the sender can be unaware of. Note that I think here
that the Messenger use of explicit hostnames/ports in "addresses" is
confusing. This might be a particular use case for addresses, but I think
in general the tying of addresses to physical locations is a very bad
thing. Addresses are logical.
I think the issue here is how many fields you expect the sender of a
message to have to populate to route a message. In 0.x the model generally
implied two fields "exchange" and "routing-key" that taken together would
allow the intermediary to route the message. My take on the mapping to
AMQP 1.0 is that one should consider this as a compound address
"exchange/routing-key". As such the AMQP 1.0 address is actually more
opaque and allows more indirection than the 0.x form where the sender of
the message was made aware of a distinction between exchange and
routing-key (and in fact the method of routing was de facto also being made
explicit).
>
>
> If an application is designed around using AMQP 1.0, I would not normally
>> expect them to be putting routing information in the subject field.
>>
>
> That depends on what you define to be routing information. I think it is a
> perfectly sensible approach to use the subject to indicate some sort of
> logical category and then have the broker be configured to 'route' messages
> according to that logical category.
>
I think that's one choice you could make, certainly. I just think it's a
bad choice for the general case. I think using a single field to provide
the information you wish the message to be routed on is, in general,
superior. In practice if one wants to add supplemental fields for routing
I think it would be better to use application properties. By using an
application property it is easier for messages coming from other APIs (such
as JMS) or messages being translated from other protocols (where the notion
of application structured data is common) to also use the same routing
semantics.
-- Rob
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Use of subject for routing - moved thread to user list from earlier
private discussion.
Posted by Gordon Sim <gs...@redhat.com>.
On 08/29/2014 03:12 PM, Rob Godfrey wrote:
> In general, however, I think the subject is not the best fit for a mapping
> of the semantics of the 0.x "routing-key". The AMQP 1.0 spec defines
> subject as "A common field for summary information about the message
> content and purpose", rather than giving it and explicit or implicit
> routing semantics.
In my view, the 'routing key' in the old exchanges model is intended to
be exactly such a logical categorisation of the messages content or purpose.
> The "to" field is defined as "identifies the node that
> is the intended destination of the message" which definitely implies some
> routing semantics but does not make them explicit.
As you state, the to field identifies the destination. That is not what
the old exchange types are about in my view. In the typical use cases
for that old model, the sender doesn't want to describe who gets the
queue, they merely want to indicate the type or purpose of the messages.
It is then through the bindings that this type is mapped to the
appropriate recipient(s).
> If an application is designed around using AMQP 1.0, I would not normally
> expect them to be putting routing information in the subject field.
That depends on what you define to be routing information. I think it is
a perfectly sensible approach to use the subject to indicate some sort
of logical category and then have the broker be configured to 'route'
messages according to that logical category.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Use of subject for routing - moved thread to user list from earlier
private discussion.
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
On 29/08/14 16:15, Rob Godfrey wrote:
> On 29 August 2014 16:50, Fraser Adams <fr...@blueyonder.co.uk> wrote:
>
>> Back to my point about APIs though, one thing that I think is important is
>> to be able to efficiently "multiplex" messages. In the current subject
>> based world I could have a number of consumers to amq.direct called Q1, Q2,
>> Q3 etc. and in JMS I could create a simple producer that had "amq.direct"
>> as the JMS destination. From my client's perspective I can create that at
>> startup time and send different messages to different places just by
>> setting the subject.
>
> Except that there is no way to set the subject in JMS, other than through
> some Qpid specific magic that doesn't actually exist in any AMQP 1.0 JMS
> client to my knowledge :-). The qpid.subject hack (which then had to be
> changed again because qpid.subject is an illegal name in JMS) was an 0-10
> Addressing Syntax specific addition to the Qpid JMS client. You couldn't
> use that if you were using AMQP 0-9-1, for instance.
This sort of thing is exactly why I said that it's really important to
consider the APIs and to hopefully try to consider all the various APIs,
because they all have different quirks/strengths/weaknesses - remember
me suggesting that lists probably weren't a good idea in AMQP 1.0
Management 'cause lists are a pain in the butt with JMS (though dead
easy with my JavaScript Messenger ;->).
TBH I'm quite glad this subject (heh, heh, heh) has come up, it's a good
time to get this sort of discussion in the open, as I said earlier one
of the things I'm keen on is the maximum consistency across the Qpid C++
and Java offerings. if we can't do that then we're probably not setting
the best example in terms of cross-vendor interoperability.
>> I wouldn't want to have to create new Destination objects every time that
>> I wanted to send a message!! I know that "to" is an immutable AMQP 1.0
>> property, but what's the API mapping - and what should it contain??
>>
>>
> So, JMS is clearly rubbish in the way that it does topics :-)
It's pretty pants at handling lists too and MapMessage needs an
alternative that plays better with Collections etc. etc. etc. in
fairness I've still got a bit of a soft spot for JMS, but it's quite
irritating too.
> In my head I am rewriting every AMQP 0.x exchange + routing-key
> combination as the address "exchange/routing-key" in AMQP 1.0 and
> conversely any AMQP 1.0 address of foo/bar as the "exchange" foo and
> "routing-key" bar in AMQP 0.x. This obviously may require the broker
> to dynamically pretend that such an entity as "foo" exists, in order
> to service the requests of an AMQP 0.x client. -- Rob
That's not an unreasonable mental model, but such a model isn't
universally supported at present and there's the potential ambiguity
over whether an address that *looks like* "exchange/routing-key" is
*really* that or a node called "exchange/routing-key", which is equally
valid and
Indeed using an address of that form in Messenger to the C++ Broker
causes it to barf
./qpid-config.js -r queues
amqp://guest:guest@localhost:5673/qmf.default.direct/broker
LINK ERROR (amqp:not-found) Node not found: qmf.default.direct/broker
I've also just managed to add a queue called "exchange/routing-key" :-)
and another called "amq.direct/routing-key"
probably not a good idea, bit it's arguably perfectly valid as an AMQP
1.0 node name.
All this routing stuff is just a wee bit subtle, especially when you
factor in all the various API quirks.
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Use of subject for routing - moved thread to user list from
earlier private discussion.
Posted by Rob Godfrey <ro...@gmail.com>.
On 29 August 2014 16:50, Fraser Adams <fr...@blueyonder.co.uk> wrote:
>
> I guess that from my perspective one of my main concerns is consistency
> between the C++ and Java Brokers (and as much as possible with other
> vendors) without that things get very confusing.
>
> Another thing I guess that I'd want to make sure gets thought about is how
> this stuff gets mapped to various APIs.
>
> I think I've got some sympathy with the logic of Rob's comment "subject as
> "A common field for summary information about the message content and
> purpose" " and "The "to" field is defined as "identifies the node that is
> the intended destination of the message" " those sort of definitions seem
> to fit with traditional email interpretations of subject and to, however
> it's very definitely fairly "traditional" in Messaging to use the term
> "subject" when talking about publishing topics i.e. the traditional
> "UK.NEWS" subject, so it's not unreasonable that people might intuitively
> feel that subject and routing key have a relationship.
>
> Back to my point about APIs though, one thing that I think is important is
> to be able to efficiently "multiplex" messages. In the current subject
> based world I could have a number of consumers to amq.direct called Q1, Q2,
> Q3 etc. and in JMS I could create a simple producer that had "amq.direct"
> as the JMS destination. From my client's perspective I can create that at
> startup time and send different messages to different places just by
> setting the subject.
Except that there is no way to set the subject in JMS, other than through
some Qpid specific magic that doesn't actually exist in any AMQP 1.0 JMS
client to my knowledge :-). The qpid.subject hack (which then had to be
changed again because qpid.subject is an illegal name in JMS) was an 0-10
Addressing Syntax specific addition to the Qpid JMS client. You couldn't
use that if you were using AMQP 0-9-1, for instance.
> I wouldn't want to have to create new Destination objects every time that
> I wanted to send a message!! I know that "to" is an immutable AMQP 1.0
> property, but what's the API mapping - and what should it contain??
>
>
So, JMS is clearly rubbish in the way that it does topics :-)
> what do I mean "what should it contain??" well different things might make
> different sense depending on whether one is using a "connection-oriented"
> API or not. For example in Messenger I "think" that the to would be the
> whole address - say "amqp://guest:guest@localhost:5673/qmf.default.direct/broker"
> what
The on the wire address should never contain amqp://guest:guest@ . The
magic of the Messenger API, of course, is a whole different matter. How a
messenger address is translated to an on the wire address similarly.
Personally I think that while Messenger is useful for some bootstrapping
cases, for a large scale system you would need to separate your connection
address (where you connect to) and the message address (the logical name
which will be used to determine where the message ends up). I believe
Messenger can be set up with some sort of routing table to achieve this...
but I think the base case for Messenger misleads as to how addresses would
work in a larger system (addresses are not going to be scoped and bound to
a DNS address of a connectivity endpoint).
> would it be in a connection-oriented API like JMS/qpid::messaging, would
> it be the same or would it just be "broker". We already have this sort of
> strange inconsistency around replyTo where one interpretation of a replyTo
> might be the fully qualified AMQP address of the originator, but the
> Broker's use the node name, both are reasonable interpretations, but they
> are clearly different and have different implications
>
> Similarly in Messenger, as I say I think that the "to" gets set by
> setAddress() on Message, but I've no idea what caching (if any) takes
> place. Is it more efficient to do something like:
>
> message.setAddress(address + '/' + subscriptionQueue);
>
> where subscriptionQueue might be one of the Q1, Q2, Q3 things I mentioned
> above, or is it more efficient to do
>
> message.setAddress(address);
>
> and pass the subscriptionQueue information as the subject/routing-key
> whatever.
>
>
> So I think it's not just about whether in the AMQP 1.0 spec. the subject
> and to now look more like the email interpretations (and I do have sympathy
> with that view) as I say though it's also about traditional interpretations
> of what "subject" means in messaging systems and also about how the various
> APIs allow multiplexing of messages.
>
>
I agree - this is really a question about how we present features through
APIs and how much our current mapping from APIs should drive our broker
models, vs. how an abstract model can be presented through an API.
Moreover I think it is a question as to how "special" we want dealing with
a broker containing a 0.x model to feel compared to the general case of an
AMQP 1.0 network... and conversely how transparent we can make it for an
0.x client to connect into a messaging network which is actually based on
1.0 (and may contain entities that are neither queues nor exchanges).
My personal view, at a very high level I believe that the simplest way to
do routing is to think of things having a single key which is (the usual
case) for routing. Further that this key can be considered opaque, is
logical, and may refer to an address space rather than a single logical
address.
In my head I am rewriting every AMQP 0.x exchange + routing-key combination
as the address "exchange/routing-key" in AMQP 1.0 and conversely any AMQP
1.0 address of foo/bar as the "exchange" foo and "routing-key" bar in AMQP
0.x. This obviously may require the broker to dynamically pretend that
such an entity as "foo" exists, in order to service the requests of an AMQP
0.x client.
-- Rob
> Frase
>
>
> On 29/08/14 15:12, Rob Godfrey wrote:
>
>> So, I think the issue has come about on the Java Broker because I have
>> been
>> trying to move some of the other behaviours to be in line with the
>> upcoming
>> work on management and addressing. In these cases the "to" field of an
>> AMQP 1.0 message is used for routing purposes. The inadvertent side
>> effect
>> of the changes (that to was promoted "to" be the primary routing attribute
>> of the message) was that the exchanges which previously used the "subject"
>> field of a 1.0 message where they would use "routing-key" of an 0.x
>> message
>> currently use "to". Given the way that the legacy filters were
>> constructed
>> this is an error and should probably be changed so that the Java Broker
>> reverts to routing on subject for the amq.direct and amq.topic exchanges
>> (and any other address which in AMQP 1.0 offers the ability to use the
>> legacy filters).
>>
>> In general, however, I think the subject is not the best fit for a mapping
>> of the semantics of the 0.x "routing-key". The AMQP 1.0 spec defines
>> subject as "A common field for summary information about the message
>> content and purpose", rather than giving it and explicit or implicit
>> routing semantics. The "to" field is defined as "identifies the node that
>> is the intended destination of the message" which definitely implies some
>> routing semantics but does not make them explicit.
>>
>> If an application is designed around using AMQP 1.0, I would not normally
>> expect them to be putting routing information in the subject field.
>>
>> -- Rob
>>
>>
>> On 29 August 2014 15:55, Fraser Adams <fr...@blueyonder.co.uk>
>> wrote:
>>
>> All,
>>> I've moved this thread to the user list. It started out on the back of an
>>> email conversation between Rob Godfrey. Gordon Sim and and myself when I
>>> couldn't get my QMF messages routed on the Java Broker.
>>>
>>> The original context was that I was sending messages to
>>> qmf.default.direct
>>> and setting the message subject to "broker", this is pretty much exactly
>>> what the python qpid tools such as qpid-config do.
>>>
>>> in AMQP 0.10 this works fine because the subject (which in AMQP 0.10 is a
>>> user property called qpid.subject) gets treated as the routing key when
>>> sent to a direct exchange.
>>>
>>> When I did my AMQP 1.0 JavaScript qpid-config port I set the AMQP 1.0
>>
>>> subject (which is now an immutable message property and not an
>>> application
>>> property) to "broker" and sent to amqp://guest:guest@localhost:
>>> 5673/qmf.default.direct
>>>
>>>
>>> This worked fine with the C++ broker, but with the Java Broker the
>>> message
>>> wasn't routed and in subsequent conversations with Rob and Gordon I
>>> discovered that the Java Broker doesn't route on direct exchanges based
>>> on
>>> Subject rather it first tries to use to "to" - in other words if I sent
>>> to
>>> amqp://guest:guest@localhost:5673/qmf.default.direct/broker it'd work or
>>> if the to isn't set it uses the application property "routing-key".
>>>
>>> So basically this thread is around an inconsistency between the C++ and
>>> Java Brokers where the C++ Broker continues to route on Subject for AMQP
>>> 1.0 but the Java Broker does not.
>>>
>>> Rob and Gordon can fill in if I've missed anything, but it's probably
>>> best
>>> to share this discussion on the user list.
>>>
>>> Frase
>>>
>>>
>>>
>>>
>>> On 29/08/14 14:03, Rob Godfrey wrote:
>>>
>>> I think in the context of where AMQP 1.0 is now, routing by subject is
>>>> somewhat counter-intuitive, though when we started we did see the
>>>> subject
>>>> field in a routing-key sort of role.
>>>>
>>>> From my perspective I'd say exactly the opposite, that is routing by
>>> subject seems entirely intuitive and is also consistent with what
>>> happened
>>> with AMQP 0.10. Given the use of subject as the defacto routing key on
>>> all
>>> of the python tools I don't think I'm alone in feeling that's intuitive.
>>> I'm pretty sure that the subject is used for routing on topic exchanges
>>> so
>>> I'm not sure why you think its counter-intuitive on derect exchanges?
>>>
>>>
>>> In terms of current behaviour - the legacy filters in the Java Broker
>>>> just set up the bindings between the queue and the exchange and thus use
>>>> the same routing as previously described (i.e. they're not currently
>>>> using
>>>> subject). Inside the Java Broker an exchange routes based on an abstract
>>>> notion of the "routingAddress" of a message instance. In AMQP 0.x
>>>> that's
>>>> taken to be the routing key. In 1.0 it is as I described in my previous
>>>> mail.
>>>>
>>>> If we collectively (and this discussion should really be on the users
>>>> list) think that 0.x exchanges should route on subject when routing a
>>>> 1.0
>>>> message then I'm happy to change the default behaviour of the Java
>>>> Broker -
>>>> except for the "no name" exchange which will route on "to" because
>>>> that's
>>>> what we're defining on the addressing spec. I'll then probably add
>>>> options
>>>> to the exchange implementation so that a user can configure on a per
>>>> exchange basis to route on something else (except for the non default
>>>> amq.*
>>>> exchanges).
>>>>
>>>> -- Rob
>>>>
>>>>
>>>> On 29 August 2014 13:45, Gordon Sim <gsim@redhat.com <mailto:
>>>> gsim@redhat.com>> wrote:
>>>>
>>>> On 08/29/2014 11:49 AM, Rob Godfrey wrote:
>>>>
>>>> For the moment I guess we'll just have to live with the C++
>>>> and Java
>>>> Brokers having different ideas about how to route 1.0 messages
>>>> at exchanges.
>>>>
>>>>
>>>> The legacy-amqp filters are defined to work on the subject when
>>>> using the exchange as the source. I thought the java broker
>>>> supported those?
>>>>
>>>> If I create a receiving link from amq.direct with a
>>>> apache.org:legacy-amqp-direct-binding:string of foo, then send a
>>>> message to amq.direct whose subject is foo, my expectation would
>>>> be that the receiver will receive that.
>>>>
>>>>
>>>>
>>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Use of subject for routing - moved thread to user list from earlier
private discussion.
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
I guess that from my perspective one of my main concerns is consistency
between the C++ and Java Brokers (and as much as possible with other
vendors) without that things get very confusing.
Another thing I guess that I'd want to make sure gets thought about is
how this stuff gets mapped to various APIs.
I think I've got some sympathy with the logic of Rob's comment "subject
as "A common field for summary information about the message content and
purpose" " and "The "to" field is defined as "identifies the node that
is the intended destination of the message" " those sort of definitions
seem to fit with traditional email interpretations of subject and to,
however it's very definitely fairly "traditional" in Messaging to use
the term "subject" when talking about publishing topics i.e. the
traditional "UK.NEWS" subject, so it's not unreasonable that people
might intuitively feel that subject and routing key have a relationship.
Back to my point about APIs though, one thing that I think is important
is to be able to efficiently "multiplex" messages. In the current
subject based world I could have a number of consumers to amq.direct
called Q1, Q2, Q3 etc. and in JMS I could create a simple producer that
had "amq.direct" as the JMS destination. From my client's perspective I
can create that at startup time and send different messages to different
places just by setting the subject. I wouldn't want to have to create
new Destination objects every time that I wanted to send a message!! I
know that "to" is an immutable AMQP 1.0 property, but what's the API
mapping - and what should it contain??
what do I mean "what should it contain??" well different things might
make different sense depending on whether one is using a
"connection-oriented" API or not. For example in Messenger I "think"
that the to would be the whole address - say
"amqp://guest:guest@localhost:5673/qmf.default.direct/broker" what would
it be in a connection-oriented API like JMS/qpid::messaging, would it be
the same or would it just be "broker". We already have this sort of
strange inconsistency around replyTo where one interpretation of a
replyTo might be the fully qualified AMQP address of the originator, but
the Broker's use the node name, both are reasonable interpretations, but
they are clearly different and have different implications
Similarly in Messenger, as I say I think that the "to" gets set by
setAddress() on Message, but I've no idea what caching (if any) takes
place. Is it more efficient to do something like:
message.setAddress(address + '/' + subscriptionQueue);
where subscriptionQueue might be one of the Q1, Q2, Q3 things I
mentioned above, or is it more efficient to do
message.setAddress(address);
and pass the subscriptionQueue information as the subject/routing-key
whatever.
So I think it's not just about whether in the AMQP 1.0 spec. the subject
and to now look more like the email interpretations (and I do have
sympathy with that view) as I say though it's also about traditional
interpretations of what "subject" means in messaging systems and also
about how the various APIs allow multiplexing of messages.
Frase
On 29/08/14 15:12, Rob Godfrey wrote:
> So, I think the issue has come about on the Java Broker because I have been
> trying to move some of the other behaviours to be in line with the upcoming
> work on management and addressing. In these cases the "to" field of an
> AMQP 1.0 message is used for routing purposes. The inadvertent side effect
> of the changes (that to was promoted "to" be the primary routing attribute
> of the message) was that the exchanges which previously used the "subject"
> field of a 1.0 message where they would use "routing-key" of an 0.x message
> currently use "to". Given the way that the legacy filters were constructed
> this is an error and should probably be changed so that the Java Broker
> reverts to routing on subject for the amq.direct and amq.topic exchanges
> (and any other address which in AMQP 1.0 offers the ability to use the
> legacy filters).
>
> In general, however, I think the subject is not the best fit for a mapping
> of the semantics of the 0.x "routing-key". The AMQP 1.0 spec defines
> subject as "A common field for summary information about the message
> content and purpose", rather than giving it and explicit or implicit
> routing semantics. The "to" field is defined as "identifies the node that
> is the intended destination of the message" which definitely implies some
> routing semantics but does not make them explicit.
>
> If an application is designed around using AMQP 1.0, I would not normally
> expect them to be putting routing information in the subject field.
>
> -- Rob
>
>
> On 29 August 2014 15:55, Fraser Adams <fr...@blueyonder.co.uk> wrote:
>
>> All,
>> I've moved this thread to the user list. It started out on the back of an
>> email conversation between Rob Godfrey. Gordon Sim and and myself when I
>> couldn't get my QMF messages routed on the Java Broker.
>>
>> The original context was that I was sending messages to qmf.default.direct
>> and setting the message subject to "broker", this is pretty much exactly
>> what the python qpid tools such as qpid-config do.
>>
>> in AMQP 0.10 this works fine because the subject (which in AMQP 0.10 is a
>> user property called qpid.subject) gets treated as the routing key when
>> sent to a direct exchange.
>>
> When I did my AMQP 1.0 JavaScript qpid-config port I set the AMQP 1.0
>> subject (which is now an immutable message property and not an application
>> property) to "broker" and sent to amqp://guest:guest@localhost:
>> 5673/qmf.default.direct
>>
>>
>> This worked fine with the C++ broker, but with the Java Broker the message
>> wasn't routed and in subsequent conversations with Rob and Gordon I
>> discovered that the Java Broker doesn't route on direct exchanges based on
>> Subject rather it first tries to use to "to" - in other words if I sent to
>> amqp://guest:guest@localhost:5673/qmf.default.direct/broker it'd work or
>> if the to isn't set it uses the application property "routing-key".
>>
>> So basically this thread is around an inconsistency between the C++ and
>> Java Brokers where the C++ Broker continues to route on Subject for AMQP
>> 1.0 but the Java Broker does not.
>>
>> Rob and Gordon can fill in if I've missed anything, but it's probably best
>> to share this discussion on the user list.
>>
>> Frase
>>
>>
>>
>>
>> On 29/08/14 14:03, Rob Godfrey wrote:
>>
>>> I think in the context of where AMQP 1.0 is now, routing by subject is
>>> somewhat counter-intuitive, though when we started we did see the subject
>>> field in a routing-key sort of role.
>>>
>> From my perspective I'd say exactly the opposite, that is routing by
>> subject seems entirely intuitive and is also consistent with what happened
>> with AMQP 0.10. Given the use of subject as the defacto routing key on all
>> of the python tools I don't think I'm alone in feeling that's intuitive.
>> I'm pretty sure that the subject is used for routing on topic exchanges so
>> I'm not sure why you think its counter-intuitive on derect exchanges?
>>
>>
>>> In terms of current behaviour - the legacy filters in the Java Broker
>>> just set up the bindings between the queue and the exchange and thus use
>>> the same routing as previously described (i.e. they're not currently using
>>> subject). Inside the Java Broker an exchange routes based on an abstract
>>> notion of the "routingAddress" of a message instance. In AMQP 0.x that's
>>> taken to be the routing key. In 1.0 it is as I described in my previous
>>> mail.
>>>
>>> If we collectively (and this discussion should really be on the users
>>> list) think that 0.x exchanges should route on subject when routing a 1.0
>>> message then I'm happy to change the default behaviour of the Java Broker -
>>> except for the "no name" exchange which will route on "to" because that's
>>> what we're defining on the addressing spec. I'll then probably add options
>>> to the exchange implementation so that a user can configure on a per
>>> exchange basis to route on something else (except for the non default amq.*
>>> exchanges).
>>>
>>> -- Rob
>>>
>>>
>>> On 29 August 2014 13:45, Gordon Sim <gsim@redhat.com <mailto:
>>> gsim@redhat.com>> wrote:
>>>
>>> On 08/29/2014 11:49 AM, Rob Godfrey wrote:
>>>
>>> For the moment I guess we'll just have to live with the C++
>>> and Java
>>> Brokers having different ideas about how to route 1.0 messages
>>> at exchanges.
>>>
>>>
>>> The legacy-amqp filters are defined to work on the subject when
>>> using the exchange as the source. I thought the java broker
>>> supported those?
>>>
>>> If I create a receiving link from amq.direct with a
>>> apache.org:legacy-amqp-direct-binding:string of foo, then send a
>>> message to amq.direct whose subject is foo, my expectation would
>>> be that the receiver will receive that.
>>>
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Use of subject for routing - moved thread to user list from
earlier private discussion.
Posted by Rob Godfrey <ro...@gmail.com>.
So, I think the issue has come about on the Java Broker because I have been
trying to move some of the other behaviours to be in line with the upcoming
work on management and addressing. In these cases the "to" field of an
AMQP 1.0 message is used for routing purposes. The inadvertent side effect
of the changes (that to was promoted "to" be the primary routing attribute
of the message) was that the exchanges which previously used the "subject"
field of a 1.0 message where they would use "routing-key" of an 0.x message
currently use "to". Given the way that the legacy filters were constructed
this is an error and should probably be changed so that the Java Broker
reverts to routing on subject for the amq.direct and amq.topic exchanges
(and any other address which in AMQP 1.0 offers the ability to use the
legacy filters).
In general, however, I think the subject is not the best fit for a mapping
of the semantics of the 0.x "routing-key". The AMQP 1.0 spec defines
subject as "A common field for summary information about the message
content and purpose", rather than giving it and explicit or implicit
routing semantics. The "to" field is defined as "identifies the node that
is the intended destination of the message" which definitely implies some
routing semantics but does not make them explicit.
If an application is designed around using AMQP 1.0, I would not normally
expect them to be putting routing information in the subject field.
-- Rob
On 29 August 2014 15:55, Fraser Adams <fr...@blueyonder.co.uk> wrote:
> All,
> I've moved this thread to the user list. It started out on the back of an
> email conversation between Rob Godfrey. Gordon Sim and and myself when I
> couldn't get my QMF messages routed on the Java Broker.
>
> The original context was that I was sending messages to qmf.default.direct
> and setting the message subject to "broker", this is pretty much exactly
> what the python qpid tools such as qpid-config do.
>
> in AMQP 0.10 this works fine because the subject (which in AMQP 0.10 is a
> user property called qpid.subject) gets treated as the routing key when
> sent to a direct exchange.
>
When I did my AMQP 1.0 JavaScript qpid-config port I set the AMQP 1.0
> subject (which is now an immutable message property and not an application
> property) to "broker" and sent to amqp://guest:guest@localhost:
> 5673/qmf.default.direct
>
>
> This worked fine with the C++ broker, but with the Java Broker the message
> wasn't routed and in subsequent conversations with Rob and Gordon I
> discovered that the Java Broker doesn't route on direct exchanges based on
> Subject rather it first tries to use to "to" - in other words if I sent to
> amqp://guest:guest@localhost:5673/qmf.default.direct/broker it'd work or
> if the to isn't set it uses the application property "routing-key".
>
> So basically this thread is around an inconsistency between the C++ and
> Java Brokers where the C++ Broker continues to route on Subject for AMQP
> 1.0 but the Java Broker does not.
>
> Rob and Gordon can fill in if I've missed anything, but it's probably best
> to share this discussion on the user list.
>
> Frase
>
>
>
>
> On 29/08/14 14:03, Rob Godfrey wrote:
>
>> I think in the context of where AMQP 1.0 is now, routing by subject is
>> somewhat counter-intuitive, though when we started we did see the subject
>> field in a routing-key sort of role.
>>
> From my perspective I'd say exactly the opposite, that is routing by
> subject seems entirely intuitive and is also consistent with what happened
> with AMQP 0.10. Given the use of subject as the defacto routing key on all
> of the python tools I don't think I'm alone in feeling that's intuitive.
> I'm pretty sure that the subject is used for routing on topic exchanges so
> I'm not sure why you think its counter-intuitive on derect exchanges?
>
>
>> In terms of current behaviour - the legacy filters in the Java Broker
>> just set up the bindings between the queue and the exchange and thus use
>> the same routing as previously described (i.e. they're not currently using
>> subject). Inside the Java Broker an exchange routes based on an abstract
>> notion of the "routingAddress" of a message instance. In AMQP 0.x that's
>> taken to be the routing key. In 1.0 it is as I described in my previous
>> mail.
>>
>> If we collectively (and this discussion should really be on the users
>> list) think that 0.x exchanges should route on subject when routing a 1.0
>> message then I'm happy to change the default behaviour of the Java Broker -
>> except for the "no name" exchange which will route on "to" because that's
>> what we're defining on the addressing spec. I'll then probably add options
>> to the exchange implementation so that a user can configure on a per
>> exchange basis to route on something else (except for the non default amq.*
>> exchanges).
>>
>> -- Rob
>>
>>
>> On 29 August 2014 13:45, Gordon Sim <gsim@redhat.com <mailto:
>> gsim@redhat.com>> wrote:
>>
>> On 08/29/2014 11:49 AM, Rob Godfrey wrote:
>>
>> For the moment I guess we'll just have to live with the C++
>> and Java
>> Brokers having different ideas about how to route 1.0 messages
>> at exchanges.
>>
>>
>> The legacy-amqp filters are defined to work on the subject when
>> using the exchange as the source. I thought the java broker
>> supported those?
>>
>> If I create a receiving link from amq.direct with a
>> apache.org:legacy-amqp-direct-binding:string of foo, then send a
>> message to amq.direct whose subject is foo, my expectation would
>> be that the receiver will receive that.
>>
>>
>>