AMQP 1.0, Qpid broker+dispatcher, and routing

[tomt <tt...@netapp.com>]

Hello,

I have read through all of the various AMQP 1.0 specs and installed and
configured the proton-c, the dispatch router, and qpid-cpp.  I've got a
prefix in the dispatch router that is connected to the broker via linkRoute
configurations.  Additionally, I have been able to spend time trying out a
few scenarios and I've got some questions.  I would like to primarily use
the 1.0 protocol via the proton libraries and the JMS one as well.

1.  qpid-cpp is a 0-10-based broker, but has 1.0 support via a plugin.  To
me, this appears to have an affect on addressing compared to a pure 1.0
implementation.  I would think that if i had exchanges/topics declared like
/a/b and /a/c, a receiver should be able to enter a wild-carded topic to
look for both /a/b and /a/c by making the address /a/#.  Should this be the
case if everything was 1.0 or am I misinterpreting what I've read?  I've
looked over the proton-c examples broker.py, server.py, and client.py, which
makes me thing this would work if the broker.py was a fully implemented
broker.

2. The qpid-cpp broker does not appear to set remote_target or remote_source
when creating senders and receivers... When would these ever be set?  I
would like the ability to set target and source options for a given client
so that they don't go about creating infinite amounts of durable
subscriptions.  Azure Message Bus looks like it has REST APIs to do some
topic/queue creation that could potentially put these controls in place, but
then it also has similar proton APIs where you can create_receiver/sender.

3. AMQP 1.0 uses / notation for addressing rather than periods... should I
be naming my exchanges/topics this way as well? It doesn't really seem to
buy me anything other than just conforming with one of the specs if I can't
create a receiver as mentioned in question 1.

4. Finally, the dispatch router book mentions that it does both message
routing (default) and link routing but I haven't fully understood what I
have read.  I think I'm using link routing if I have configured linkRoutes
in the router spec, but is that right?

I'm sure I'll have more questions as I go through more. ;)



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: AMQP 1.0, Qpid broker+dispatcher, and routing

[Gordon Sim <gs...@redhat.com>]

On 07/01/2020 5:01 am, tomt wrote:
> As far as the naming of topics in the broker, I had asked last time about
> dot notation vs slashes since I had read the AMQP 1.0 addressing spec.  Does
> it make sense to use slashes in the naming for topics to be inline with the
> addressing spec?

It doesn't make any difference to qpid-cpp (or the router).

> As far as my expected setup goes, I have 4 machines dedicated to my
> exploration effort here and I am running the qpid-cpp broker in HA mode.  I
> also have a router on each machine that points to the active broker.  In the
> event of a broker failover to another node, the router configurations are
> expected to be modified on the fly to point to the newly active broker.  My
> routers have link routing configured so any topic with a given prefix can be
> subscribed to from any machine in my cluster by any number of clients
> unknown to me.  I would think the clients would have no idea that anything
> changed under the covers but have not yet had a chance to test this.
> 
> It looks like this dispatcher config using link routers should work for my
> needs over broker messaging unless I'm missing something.  From what I can
> tell, in broker messaging you must define the ins and outs for each queue in
> qdrouter's configuration where link routing is more flexible by creating the
> virtual tunnel you mentioned for the link.  I can even set link options like
> timeout, which is important next.

Agreed, link routing is what you want for an arbitrary number of 
independent topic subscriptions.

> I have seen in the broker that the defined name for the virtual tunnel is
> <router_id>:<container_id>@<subscription_name> (or something very close to
> that).  If my AMQP client that is connected to one router unexpectedly moves
> over to another router, can it attach to the existing link on the new router
> (if the router names are all the same)?

I don't think the router names *can* be the same, as they need to be 
uniquely identified. For durable (or reliable) subscriptions, that will 
indeed likely pose a problem, not from where the client is attached but 
from which router is connected to the broker (as it is the container id 
of that router that the broker will see).

You can set a 'shared' capability on the link source, in which case the 
c++ broker will use only the link name (not qualified by the container id).


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: AMQP 1.0, Qpid broker+dispatcher, and routing

[tomt <tt...@netapp.com>]

Gordon,

It looks like the java broker cannot do the wildcarding on the address, but
configuring a topic via qpid-config looks promising.

As far as the naming of topics in the broker, I had asked last time about
dot notation vs slashes since I had read the AMQP 1.0 addressing spec.  Does
it make sense to use slashes in the naming for topics to be inline with the
addressing spec?  

As far as my expected setup goes, I have 4 machines dedicated to my
exploration effort here and I am running the qpid-cpp broker in HA mode.  I
also have a router on each machine that points to the active broker.  In the
event of a broker failover to another node, the router configurations are
expected to be modified on the fly to point to the newly active broker.  My
routers have link routing configured so any topic with a given prefix can be
subscribed to from any machine in my cluster by any number of clients
unknown to me.  I would think the clients would have no idea that anything
changed under the covers but have not yet had a chance to test this.

It looks like this dispatcher config using link routers should work for my
needs over broker messaging unless I'm missing something.  From what I can
tell, in broker messaging you must define the ins and outs for each queue in
qdrouter's configuration where link routing is more flexible by creating the
virtual tunnel you mentioned for the link.  I can even set link options like
timeout, which is important next.

I have seen in the broker that the defined name for the virtual tunnel is
<router_id>:<container_id>@<subscription_name> (or something very close to
that).  If my AMQP client that is connected to one router unexpectedly moves
over to another router, can it attach to the existing link on the new router
(if the router names are all the same)?

Any thoughts or concerns?

Thanks,
Tom



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: AMQP 1.0, Qpid broker+dispatcher, and routing

[Gordon Sim <gs...@redhat.com>]

On 17/12/2019 12:31 am, tomt wrote:
> Hello,
> 
> I have read through all of the various AMQP 1.0 specs and installed and
> configured the proton-c, the dispatch router, and qpid-cpp.  I've got a
> prefix in the dispatch router that is connected to the broker via linkRoute
> configurations.  Additionally, I have been able to spend time trying out a
> few scenarios and I've got some questions.  I would like to primarily use
> the 1.0 protocol via the proton libraries and the JMS one as well.
> 
> 1.  qpid-cpp is a 0-10-based broker, but has 1.0 support via a plugin.  To
> me, this appears to have an affect on addressing compared to a pure 1.0
> implementation.  I would think that if i had exchanges/topics declared like
> /a/b and /a/c, a receiver should be able to enter a wild-carded topic to
> look for both /a/b and /a/c by making the address /a/#.  Should this be the
> case if everything was 1.0 or am I misinterpreting what I've read?  I've
> looked over the proton-c examples broker.py, server.py, and client.py, which
> makes me thing this would work if the broker.py was a fully implemented
> broker.

At present the c++ broker does not allow you to set wildcard addresses 
via the source (or target) address. It uses the filters registered with 
the AMQP WG: http://www.amqp.org/specification/1.0/filters which at the 
time was understood to be the desired approach for interoperability.

However I believe the qpid java broker *does* support the approach you 
describe (one of the experts on that component can perhaps confirm or 
deny this).

> 2. The qpid-cpp broker does not appear to set remote_target or remote_source
> when creating senders and receivers... When would these ever be set?

I'm not quite sure what you mean here. The broker should echo various 
aspects of the source or target based on whether it supports the 
semantics indicated.

>  I
> would like the ability to set target and source options for a given client
> so that they don't go about creating infinite amounts of durable
> subscriptions.  Azure Message Bus looks like it has REST APIs to do some
> topic/queue creation that could potentially put these controls in place, but
> then it also has similar proton APIs where you can create_receiver/sender.

There is a notion of a 'topic' (distinct from an 0-10 topic-exchange), 
which is essentially an exchange with a predefined policy to be applied 
to all subscription queues. E.g.

   qpid-config add topic my-topic --argument exchange=amq.topic\
       --argument qpid.max_count=500 --argument 
qpid.policy_type=self-destruct

will mean that receiver links with address my-topic will be handled by 
binding a queue with the specified policy (i.e. a maximum depth of 500 
which will delete itself if that limit is exceeded). You can set most of 
the usual options through that.

> 3. AMQP 1.0 uses / notation for addressing rather than periods... should I
> be naming my exchanges/topics this way as well? It doesn't really seem to
> buy me anything other than just conforming with one of the specs if I can't
> create a receiver as mentioned in question 1.

For the qpid-cpp broker that isn't necessary or useful.

> 4. Finally, the dispatch router book mentions that it does both message
> routing (default) and link routing but I haven't fully understood what I
> have read.  I think I'm using link routing if I have configured linkRoutes
> in the router spec, but is that right?

Yes. Link routing means that there is a 'virtual' link chained all the 
way to the endpoint, i.e. the endpoints explicitly see all the links 
(though not the connections), and all the link traffic goes to that 
endpoint over that virtual link. In message routing messages are 
individually routed so you get loadbalancing at the message level.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org