You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Chris Richardson <ch...@gmail.com> on 2014/03/27 16:33:01 UTC
Server-client message relaying with dynamic routes
Hi mailinglist,
I'm trying to set up a broker federation topology with a server and (for
prototyping) two clients and I need to send messages from one client to the
other, routed via the server broker since the clients will be
firewalled/NATed and can not communicate directly. My understanding is that
the way to do this is with dynamic routing and I've followed the discussion
on the following thread with some success:
http://qpid.2158936.n2.nabble.com/Dynamic-routing-between-disconnected-exchanges-td7598100.html.
That article describes three nodes A, B and C, and relaying from node A to
C via B.
So far so good - I can use "drain" on node C's test-topic/C
exchange/routing key and "spout" to write to node A's test-topic/C
exchange/routing key and the message is transferred via the "server" (node
B).
The problem I have is that this setup relies on TCP links being established
in both directions between each node. In my client-server scenario this is
clearly not possible and with the network restriction in place the dynamic
routing fails. As the documentation states, "A dynamic exchange route is
always a pull route. It can never be a push route.". Does this mean that
the underlying broker link must be established in the same direction as the
route, or is there some way to override this or get the route from the
server to utilize the existing link from the client? Solutions involving
VPNs and tunnels are "not allowed".
Just for info, I've included some more info from my prototype showing
information firstly from the working setup:
Client A - 192.168.0.12
Server - 192.168.0.13
Client B - 192.168.0.14
*Server:*
*~$ qpid-route link list*
Host Port Transport Durable State Last Error
=============================================================================
192.168.0.12 5672 tcp N Operational
192.168.0.14 5672 tcp N Operational
*~$ sudo netstat -np | grep 5672*
tcp 0 0 192.168.0.13:43022 192.168.0.12:5672
ESTABLISHED 1477/qpidd
tcp 0 0 192.168.0.13:5672 192.168.0.12:38576
ESTABLISHED 1477/qpidd
tcp 0 0 192.168.0.13:5672 192.168.0.14:46242
ESTABLISHED 1477/qpidd
tcp 0 0 192.168.0.13:34905 192.168.0.14:5672
ESTABLISHED 1477/qpidd
*Client A:*
*~$ qpid-route link list*
Host Port Transport Durable State Last Error
=============================================================================
192.168.0.13 5672 tcp N Operational
*~$ sudo netstat -np | grep 5672*
tcp 0 0 192.168.0.12:5672 192.168.0.13:43022
ESTABLISHED 1467/qpidd
tcp 0 0 192.168.0.12:38576 192.168.0.13:5672
ESTABLISHED 1467/qpidd
*Client B:*
Equivalent to Client A.
Then, with some (not so) cunning use of iptables, I blocked incoming
connections from the server to the clients:
*~$ sudo iptables -vnL*
Chain INPUT (policy ACCEPT 2157 packets, 424K bytes)
pkts bytes target prot opt in out source
destination
4 240 REJECT tcp -- * * 192.168.0.13
0.0.0.0/0 tcp dpt:5672 reject-with icmp-port-unreachable
and the server-side links show failure, as one would expect:
*~$ qpid-route link list*
Host Port Transport Durable State Last Error
=============================================================================
192.168.0.12 5672 tcp N Waiting Connection
refused
192.168.0.14 5672 tcp N Waiting Connection
refused
Any suggestions on how to solve this problem would be greatly appreciated!
Thanks in advance
Chris
Re: Server-client message relaying with dynamic routes
Posted by Ted Ross <tr...@redhat.com>.
On 04/04/2014 06:01 AM, Tor Rune Skoglund wrote:
> Hi Gordon,
>
> I'm working with Chris on this, as he is off-line for the time being, I
> post some additional questions:
>
> Could you please expand on how the dispatch router could be incorporated
> into our topology? Specifically, we don't see how to set up a route
> between brokers via a dispatch router.
>
> We are particularly interested in your comment "...qpidd (which supports
> establishing basic AMQP 1.0 links to/from other processes)..."; Chris've
> tried things like adding a link from a broker to a router with
> "qpid-route link add..." but this produces errors relating to SASL on
> the router side and version errors on the broker side. We haven't
> pursued this further as we're not sure this is how it's supposed to
> work. Is there any documentation or examples you could refer us to?
>
> But to sum up, are there at all any "best practise" ways to solve
> "dynamic routing" for spontaneously connected clients without public IPs
> with today's (or tomorrow's :-) ) tool-set?
>
> Simplified setup:
>
> [Machine A/Broker A] [Machine C/Broker C]
> [ Private IP ] [ Private IP ]
> [Out and In Queues ] [Out and In Queues ]
> /\ /\
> || ||
> \/ \/
> [ Priv IP GW ] [ Priv IP GW ]
> [ NAT ] [ NAT ]
> ["Random" pub IP ] ["Random" pub IP ]
> /\ /\
> || Unstable connections (e.g. 3G) ||
> \/ \/
> _________________________________________________
> [ S e r v e r P u b l i c IP ]
> [ Out and In Queues per Machine ]
> [ S e r v e r B r o k e r B ]
> [_________________________________________________]
>
> The "IP analogy" would be that Broker A would not know the route to C it
> just sends the message off to B (the "default route"), and lets B
> forward the message to the right outgoing queue for C.
Let me outline some Dispatch Router scenarios based on your setup...
First scenario: There are relatively few clients on Machine A and
Machine C and your messaging patterns don't require store and forward.
In this case you can simply put one Dispatch router on the public
server. The clients on A and C can connect outbound through the NAT
firewalls to the router, subscribe to addresses, and send messages to
each other via addresses.
Second scenario: Many clients on A and C but still no need for store
and forward.
In this case, you can put a router on A, C, and on the public server.
The routers are interconnected such that A and C establish connections
outbound to the public router (if you need help with the details, let me
know). Once these connections are established, the router network
appears flat to all of the clients attached in their various places.
Now traffic between processes on A stays on A and likewise for C and
public. Traffic between A and C flows through the public router.
Note that in the above two scenarios, there are no brokers.
Third scenario: Same as (2) but store-and-forward queuing is needed.
In this case. one or more brokers can be connected to the routers for
message queuing. This is not yet supported in the router, but there is
an experimental implementation soon to be committed that integrates
queuing into routed networks.
>
> Best regards,
> Tor Rune Skoglund, trs@swi.no
>
> Den 01. april 2014 13:27, skrev Chris Richardson:
>> Thanks very much Gordon. I have had some previous endeavours with the
>> dispatch router (in fact I started there before switching to the broker)
>> and will return to that plan of attack.
>>
>> I think we will still need brokers in this topology in order to queue
>> messages for offline clients, otherwise the router would be the better
>> option. I look forward to future releases with the improved broker
>> integration!
>>
>> /Chris
>>
>>
>>
>> 2014-03-28 15:52 GMT+00:00 Gordon Sim <gs...@redhat.com>:
>>
>>> On 03/28/2014 01:36 PM, Chris Richardson wrote:
>>>
>>>> Ah, now we are really opening Pandora's box! ;)
>>>>
>>>> The term "client" here is actually quite a simplified term and actually
>>>> refers loosely speaking to a system managing a number of products on
>>>> that... client. Both the management system and the products should be
>>>> individually identifiable and addressable over AMQP; it should also be
>>>> possible to address them in groups. Furthermore the client system will
>>>> typically be connected over an unreliable network link and the messaging
>>>> system is expected to guarantee delivery. Given these constraints I
>>>> envisage each client system having at least one local exchange (and queues
>>>> to cope with disconnections), and most probably it/they will be topic
>>>> exchanges to support the addressing functionality. However I'm very new to
>>>> the AMQP routing and addressing concepts so I may be misguided.
>>>>
>>> Even with dynamic routing, you need to manually set it up per exchange
>>> (what it then does is manage the set of routing keys in use for
>>> inter-broker links, based on the interest expressed by subscriptions).
>>>
>>> If you are using topic exchanges you might be able to simply forward all
>>> messages for a given exchange through from A->B->C, which you can do by
>>> simple static routes per exchange (not actually any more management
>>> operations than making setting up dynamic routing for the same exchange).
>>> The static routes *should* work ok with 'push' routes (though I will warn
>>> that those are not as well used), allowing you to choose the direction of
>>> the TCP connection independent of the direction of message flow.
>>>
>>> Also, without wanting to confuse the picture too much, I'll just note in
>>> passing that the recent developments around AMQP 1.0 may be of some
>>> interest to your scenario. First off there is the 'Dispatch Router'
>>> component. This is superficially similar to a broker, but it only forwards
>>> messages it never accepts responsibility for them. It could be used as the
>>> 'server' broker in your setup, with the 'clients' both connecting in and
>>> sending or receiving to/from that. You could use this in conjunction with
>>> qpidd (which supports establishing basic AMQP 1.0 links to/from other
>>> processes), or depending on your use case there may not be a real need for
>>> brokers at all.
>>>
>>> (A future release of the router will be able to establish links out to
>>> other 1.0 capable brokers (or other types of 'thing') itself making it even
>>> more flexible/powerful).
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Gordon Sim <gs...@redhat.com>.
On 04/09/2014 07:50 PM, Tor Rune Skoglund wrote:
> There would have to be queueing in and out on all devices and the server
> (B). B can accept responsibility as "storage proxy". If there is a
> message from A to C, B would enqueue it for delivery to C. Then A can
> dequeue it as soon as B holds it.
In that case, one option is to use 0-10 based federation between brokers
on A, B and C (more generally, assuming I understand your setup, have a
broker on all 'client' machines accessible only to local processes, and
one publicly accessible one on B.
For example, lets assume that each client has a single unique address of
its own through which it can be individually contacted. For a client A
you would then setup the routing for incoming messages:
qpid-config -b BrokerA --durable add exchange fanout incoming_x
qpid-config -b BrokerA --durable add queue incoming_q
qpid-config -b BrokerA bind incoming_x incoming_q
qpid-config -b BrokerB --durable add queue A
qpid-config -b BrokerB bind amq.topic A A
qpid-route --durable --ack 10 queue add BrokerA BrokerB incoming_x A
and outgoing messages:
qpid-config -b BrokerA --durable add queue outgoing_q
qpid-config -b BrokerA bind amq.topic outgoing_q '#'
qpid-route --durable --ack 10 --src-local queue add BrokerB BrokerA
amq.topic outgoing_q
Now the application on A can connect to its local broker, receive
incoming messages from 'incoming_q' and send to any other client via a
sender to amq.topic/X where X is the other clients address
(alternatively use one sender to amq.topic, and set the subject of the
message to the name of the other client). The broker on A only needs to
be accessible locally, broker B won't need to connect to it.
Messages will be queued at every stage (on the outgoing_q of sending
'client', then on the 'client' specific queue on the public broker, then
on the incoming_q of the receiving 'client'). I've kept the queues
simple, i.e. using the defaults, but they could be customised to handle
buildup of messages in different ways (e.g. dropping older messages,
blocking senders etc) and sized to the expected needs of the system.
All other clients would repeat the steps above, but replace A with
whatever their identifier was (e.g. C). These should only need to be
done once, i.e. as part of the provisioning of the machine.
It would be fairly easy to add small enhancements to the addressing
scheme - e.g. a broadcast address for message to be received by all clients.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Tor Rune Skoglund <tr...@swi.no>.
Den 09. april 2014 19:41, skrev Gordon Sim:
> On 04/09/2014 03:28 PM, Chris Richardson wrote:
>> I've set this scenario up on my test system (I had to add "--argument
>> domain=BrokerB" to the "qpid-config add incoming|outgoing ..."
>> commands...)
>
> Sorry!
>
>> and it looks promising, except if I understand correctly there is no
>> queuing on the relaying node since it's a dispatch router.
>> Unfortunately we
>> absolutely need this.
>
> Just to be clear, you would have queueing on either side, and the
> message would not be dequeued from A before being enqueued at C.
> However B would not itself accept responsibility, it would relay the
> acknowledgements, but it wouldn't undertake to deliver a message from
> A, even if A went away (i.e. no store-and-forward).
>
> [...]
There would have to be queueing in and out on all devices and the server
(B). B can accept responsibility as "storage proxy". If there is a
message from A to C, B would enqueue it for delivery to C. Then A can
dequeue it as soon as B holds it.
>> We're obviously very interested in these next developments. Can you
>> give us
>> a rough idea of when we can hope to see these implemented, at least in
>> beta?
>
> Ted would be the best person to answer that, My understanding is that
> some of the code is already in place on trunk, albeit in a rudimentary
> form, so its not very far away from some kind of usable alpha I would
> think.
I am sure we can do some heavy testing in case the [NAT'ed
A]<------>[Public B]<------>[NAT'ed C] scenario is meant to be
supported! :-)
BTW, thanks to all for answering!
- Tor Rune Skoglund, trs@swi.no
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Gordon Sim <gs...@redhat.com>.
On 04/09/2014 03:28 PM, Chris Richardson wrote:
> I've set this scenario up on my test system (I had to add "--argument
> domain=BrokerB" to the "qpid-config add incoming|outgoing ..." commands...)
Sorry!
> and it looks promising, except if I understand correctly there is no
> queuing on the relaying node since it's a dispatch router. Unfortunately we
> absolutely need this.
Just to be clear, you would have queueing on either side, and the
message would not be dequeued from A before being enqueued at C. However
B would not itself accept responsibility, it would relay the
acknowledgements, but it wouldn't undertake to deliver a message from A,
even if A went away (i.e. no store-and-forward).
[...]
> We're obviously very interested in these next developments. Can you give us
> a rough idea of when we can hope to see these implemented, at least in
> beta?
Ted would be the best person to answer that, My understanding is that
some of the code is already in place on trunk, albeit in a rudimentary
form, so its not very far away from some kind of usable alpha I would think.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Chris Richardson <cr...@fourc.eu>.
On 8 April 2014 15:41, Gordon Sim <gs...@redhat.com> wrote:
> <snip>
>
>> We are particularly interested in your comment "...qpidd (which supports
>> establishing basic AMQP 1.0 links to/from other processes)..."; Chris've
>> tried things like adding a link from a broker to a router with
>> "qpid-route link add..." but this produces errors relating to SASL on
>> the router side and version errors on the broker side. We haven't
>> pursued this further as we're not sure this is how it's supposed to
>> work. Is there any documentation or examples you could refer us to?
>>
>
> In the case of qpidd this is indeed (as Fraser pointed out) via a slightly
> different mechanism than the existing 'federation' feature (which is more
> tied to AMQP 0-10).
>
> Specifically you would create a 'domain' to represent the router (or
> whatever external AMQP 1.0 compliant process you wish to connect to), then
> create 'incoming' or 'outgoing' links for messages to be transferred over.
>
> I'm afraid although I've described it on this list a few times, it isn't
> properly documented at all yet, sorry! I should also point out that its a
> relatively raw 'feature' at present. Specific known limitations at present
> include no automatic reconnect for broker established 1.0 connections and
> no mechanism for preventing message looping.
>
> However as an example, say you swapped Dispatch Router in for Broker B,
> you would then create a 'domain' for that on both Broker A and Broker C,
> using qpid-config (on latest qpid release):
>
> qpid-config add domain BrokerB --argument url=public-ip
>
> or qpidt test tool[1] on earlier releases:
>
> qpidt create domain url=public-ip
>
> (You can also specify sasl details if you want authentication, but as
> dispatch router doesn't yet support that, I've left it out).
>
> Then you would either add 'incoming' or 'outgoing' links from/to that
> domain, e.g. on BrokerA
>
> qpid-config add outgoing my-test-link source=abc target=def
>
> and on BrokerC
>
> qpid-config add incoming my-test-link source=def target=xyz
>
> which should cause messages sent to abc on BrokerA to be received by
> subscribers to xyz on BrokerC, with messages queued in abc and xyz.
>
I've set this scenario up on my test system (I had to add "--argument
domain=BrokerB" to the "qpid-config add incoming|outgoing ..." commands...)
and it looks promising, except if I understand correctly there is no
queuing on the relaying node since it's a dispatch router. Unfortunately we
absolutely need this.
> (You can also send/receive to/from e.g. my-random-address@BrokerB which
> will dynamically create the links as needed between A/C and B. However in
> this scheme you wouldn't get any store and forward ability, so there may
> not be any value to A and C being qpidd instances at all).
>
>
> But to sum up, are there at all any "best practise" ways to solve
>> "dynamic routing" for spontaneously connected clients without public IPs
>> with today's (or tomorrow's :-) ) tool-set?
>>
>
> You should be able to set this up using the 'old' federation and qpidd,
> though it would only use AMQP 0-10. You would need to use push routes as
> well as the standard pull routes. However even with dynamic routing you
> would need one route in each direction per exchange, and the same would
> apply to 'static' routes if you didn't need the traffic to be optimised to
> avoid unnecessary transfers[2], and static routes do allow push routes
> (where the flow of messages is to the process that originally established
> the tcp connection). I think this is the best solution today if you have to
> have some store-and-forward ability (i.e. allow senders to forget about
> messages they sent and rely on the messaging intermediaries to keep trying
> to deliver until successful).
>
> Tomorrows solution would I think be based either on the new mechanisms
> being added to Dispatch Router, allowing store-and-forward capability to be
> added for addresses through an associated broker to which the router will
> manage connections, or on adding reconnect/redelivery capabilities to
> messenger. You are just a little bit ahead of us at present!
We're obviously very interested in these next developments. Can you give us
a rough idea of when we can hope to see these implemented, at least in
beta?
Thanks, Chris
Re: Server-client message relaying with dynamic routes
Posted by Gordon Sim <gs...@redhat.com>.
On 04/04/2014 11:01 AM, Tor Rune Skoglund wrote:
> Hi Gordon,
>
> I'm working with Chris on this, as he is off-line for the time being, I
> post some additional questions:
>
> Could you please expand on how the dispatch router could be incorporated
> into our topology? Specifically, we don't see how to set up a route
> between brokers via a dispatch router.
As Ted mentioned in his post, the router is not yet able to do this (and
in any case, depending on topology, you might sometimes want the
connections established in the other direction anyway).
However providing the broker used can establish connections (and links
on them) to some external AMQP 1.0 compliant process, you should be able
to get them to connect to the dispatch router.
> We are particularly interested in your comment "...qpidd (which supports
> establishing basic AMQP 1.0 links to/from other processes)..."; Chris've
> tried things like adding a link from a broker to a router with
> "qpid-route link add..." but this produces errors relating to SASL on
> the router side and version errors on the broker side. We haven't
> pursued this further as we're not sure this is how it's supposed to
> work. Is there any documentation or examples you could refer us to?
In the case of qpidd this is indeed (as Fraser pointed out) via a
slightly different mechanism than the existing 'federation' feature
(which is more tied to AMQP 0-10).
Specifically you would create a 'domain' to represent the router (or
whatever external AMQP 1.0 compliant process you wish to connect to),
then create 'incoming' or 'outgoing' links for messages to be
transferred over.
I'm afraid although I've described it on this list a few times, it isn't
properly documented at all yet, sorry! I should also point out that its
a relatively raw 'feature' at present. Specific known limitations at
present include no automatic reconnect for broker established 1.0
connections and no mechanism for preventing message looping.
However as an example, say you swapped Dispatch Router in for Broker B,
you would then create a 'domain' for that on both Broker A and Broker C,
using qpid-config (on latest qpid release):
qpid-config add domain BrokerB --argument url=public-ip
or qpidt test tool[1] on earlier releases:
qpidt create domain url=public-ip
(You can also specify sasl details if you want authentication, but as
dispatch router doesn't yet support that, I've left it out).
Then you would either add 'incoming' or 'outgoing' links from/to that
domain, e.g. on BrokerA
qpid-config add outgoing my-test-link source=abc target=def
and on BrokerC
qpid-config add incoming my-test-link source=def target=xyz
which should cause messages sent to abc on BrokerA to be received by
subscribers to xyz on BrokerC, with messages queued in abc and xyz.
(You can also send/receive to/from e.g. my-random-address@BrokerB which
will dynamically create the links as needed between A/C and B. However
in this scheme you wouldn't get any store and forward ability, so there
may not be any value to A and C being qpidd instances at all).
> But to sum up, are there at all any "best practise" ways to solve
> "dynamic routing" for spontaneously connected clients without public IPs
> with today's (or tomorrow's :-) ) tool-set?
You should be able to set this up using the 'old' federation and qpidd,
though it would only use AMQP 0-10. You would need to use push routes as
well as the standard pull routes. However even with dynamic routing you
would need one route in each direction per exchange, and the same would
apply to 'static' routes if you didn't need the traffic to be optimised
to avoid unnecessary transfers[2], and static routes do allow push
routes (where the flow of messages is to the process that originally
established the tcp connection). I think this is the best solution today
if you have to have some store-and-forward ability (i.e. allow senders
to forget about messages they sent and rely on the messaging
intermediaries to keep trying to deliver until successful).
Tomorrows solution would I think be based either on the new mechanisms
being added to Dispatch Router, allowing store-and-forward capability to
be added for addresses through an associated broker to which the router
will manage connections, or on adding reconnect/redelivery capabilities
to messenger. You are just a little bit ahead of us at present!
> Simplified setup:
>
> [Machine A/Broker A] [Machine C/Broker C]
> [ Private IP ] [ Private IP ]
> [Out and In Queues ] [Out and In Queues ]
> /\ /\
> || ||
> \/ \/
> [ Priv IP GW ] [ Priv IP GW ]
> [ NAT ] [ NAT ]
> ["Random" pub IP ] ["Random" pub IP ]
> /\ /\
> || Unstable connections (e.g. 3G) ||
> \/ \/
> _________________________________________________
> [ S e r v e r P u b l i c IP ]
> [ Out and In Queues per Machine ]
> [ S e r v e r B r o k e r B ]
> [_________________________________________________]
>
> The "IP analogy" would be that Broker A would not know the route to C it
> just sends the message off to B (the "default route"), and lets B
> forward the message to the right outgoing queue for C.
>
> Best regards,
> Tor Rune Skoglund, trs@swi.no
[1]
http://svn.apache.org/viewvc/qpid/branches/0.24/qpid/cpp/src/tests/qpidt?view=markup
[2] 'dynamic' routes only transfer messages for which there is an
interest subscriber on the other end
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Hello Tor Rune
Re: "
We are particularly interested in your comment "...qpidd (which supports
establishing basic AMQP 1.0 links to/from other processes)..."
"
I'm pretty sure that this relates to some features recently added by
Gordon to qpidd for AMQP 1.0 the main place that they have been referred
to currently is in the management-schema.xml in
<qpid>/qpid/cpp/src/qpid/broker/management-schema.xml
I think that the things you are after are the Domain and Outgoing
Management Objects. Unfortunately I'm afraid that I've not yet got round
to playing with these myself and I don't believe that there is any
documentation available on how to use them yet so you might have to wait
for Gordon to give a definitive response, however I have played with the
Topic and TopicPolicy objects and the approach should be similar (I
can't try it out for you at the moment I'm afraid 'cause my build is a
bit broken)
For adding Topics (a Topic is a sort of pseudo-node and allows one to
provide configuration for subscription Queues).
The following syntax will create a topic called fanout that uses the
exchange amq.fanout and
creates circular (ring) subscription queues of size 1000000.
qpid-config add topic fanout --argument exchange=amq.fanout \
--argument qpid.max_size=1000000 --argument qpid.policy_type=ring
qpid-config list topic
gives:
Objects of type 'topic'
name durable properties exchangeRef
============================================================================================
fanout False {u'qpid.max_size': u'1000000', u'qpid.policy_type':
u'ring'} amq.fanout
Now that's not directly going to answer your question, however I think
that the mechanism you'd need is analogous, that is to say
qpid-config add domain --argument name <name> --argument url
<destination connection URL> --argument username <username> --argument
password <password>
As I say I've not actually tried this so YMMV and I'm not totally sure
what the url relates to, but I'd guess that it might simply be the
<host>:<port> of the destination AMQP container.
In order to get messages to the destination container from the broker I
*think* that you need to specify the domain in the address, so for
example if you had set up a queue myqueue on the destination broker and
decided to give it the domain mydomain I believe that you'd use the
address myqueue@mydomain when you sent the message to the source broker.
I'm sure that I read that somewhere - but I'm darned if I can remember
where.
Sorry that this is somewhat second hand and hearsay and hopefully
someone will be able to give a more definitive answer, but it might help
to get you started.
The domain stuff was put in place for AMQP 1.0 and I'm not sure if it
supports AMQP 0.10, my guess is probably not, AMQP 1.0 is quite
different to AMQP 0.10, in particular it is a peer to peer protocol and
primarily specifies how to establish links between AMQP 1.0 nodes. This
is a lot less limiting than AMQP 0.10 and is one of the reasons why
things like the dispatch router can now be created. It also provides
much better interoperability, so whereas in the past you might have had
to build a bespoke message bridge to connect say a Qpid broker to an
ActiveMQ broker with AMQP 1.0 I think that you should be able to
configure a Domain to allow you to directly connect.
Regards,
Frase
On 04/04/14 11:01, Tor Rune Skoglund wrote:
> Hi Gordon,
>
> I'm working with Chris on this, as he is off-line for the time being, I
> post some additional questions:
>
> Could you please expand on how the dispatch router could be incorporated
> into our topology? Specifically, we don't see how to set up a route
> between brokers via a dispatch router.
>
> We are particularly interested in your comment "...qpidd (which supports
> establishing basic AMQP 1.0 links to/from other processes)..."; Chris've
> tried things like adding a link from a broker to a router with
> "qpid-route link add..." but this produces errors relating to SASL on
> the router side and version errors on the broker side. We haven't
> pursued this further as we're not sure this is how it's supposed to
> work. Is there any documentation or examples you could refer us to?
>
> But to sum up, are there at all any "best practise" ways to solve
> "dynamic routing" for spontaneously connected clients without public IPs
> with today's (or tomorrow's :-) ) tool-set?
>
> Simplified setup:
>
> [Machine A/Broker A] [Machine C/Broker C]
> [ Private IP ] [ Private IP ]
> [Out and In Queues ] [Out and In Queues ]
> /\ /\
> || ||
> \/ \/
> [ Priv IP GW ] [ Priv IP GW ]
> [ NAT ] [ NAT ]
> ["Random" pub IP ] ["Random" pub IP ]
> /\ /\
> || Unstable connections (e.g. 3G) ||
> \/ \/
> _________________________________________________
> [ S e r v e r P u b l i c IP ]
> [ Out and In Queues per Machine ]
> [ S e r v e r B r o k e r B ]
> [_________________________________________________]
>
> The "IP analogy" would be that Broker A would not know the route to C it
> just sends the message off to B (the "default route"), and lets B
> forward the message to the right outgoing queue for C.
>
> Best regards,
> Tor Rune Skoglund, trs@swi.no
>
> Den 01. april 2014 13:27, skrev Chris Richardson:
>> Thanks very much Gordon. I have had some previous endeavours with the
>> dispatch router (in fact I started there before switching to the broker)
>> and will return to that plan of attack.
>>
>> I think we will still need brokers in this topology in order to queue
>> messages for offline clients, otherwise the router would be the better
>> option. I look forward to future releases with the improved broker
>> integration!
>>
>> /Chris
>>
>>
>>
>> 2014-03-28 15:52 GMT+00:00 Gordon Sim <gs...@redhat.com>:
>>
>>> On 03/28/2014 01:36 PM, Chris Richardson wrote:
>>>
>>>> Ah, now we are really opening Pandora's box! ;)
>>>>
>>>> The term "client" here is actually quite a simplified term and actually
>>>> refers loosely speaking to a system managing a number of products on
>>>> that... client. Both the management system and the products should be
>>>> individually identifiable and addressable over AMQP; it should also be
>>>> possible to address them in groups. Furthermore the client system will
>>>> typically be connected over an unreliable network link and the messaging
>>>> system is expected to guarantee delivery. Given these constraints I
>>>> envisage each client system having at least one local exchange (and queues
>>>> to cope with disconnections), and most probably it/they will be topic
>>>> exchanges to support the addressing functionality. However I'm very new to
>>>> the AMQP routing and addressing concepts so I may be misguided.
>>>>
>>> Even with dynamic routing, you need to manually set it up per exchange
>>> (what it then does is manage the set of routing keys in use for
>>> inter-broker links, based on the interest expressed by subscriptions).
>>>
>>> If you are using topic exchanges you might be able to simply forward all
>>> messages for a given exchange through from A->B->C, which you can do by
>>> simple static routes per exchange (not actually any more management
>>> operations than making setting up dynamic routing for the same exchange).
>>> The static routes *should* work ok with 'push' routes (though I will warn
>>> that those are not as well used), allowing you to choose the direction of
>>> the TCP connection independent of the direction of message flow.
>>>
>>> Also, without wanting to confuse the picture too much, I'll just note in
>>> passing that the recent developments around AMQP 1.0 may be of some
>>> interest to your scenario. First off there is the 'Dispatch Router'
>>> component. This is superficially similar to a broker, but it only forwards
>>> messages it never accepts responsibility for them. It could be used as the
>>> 'server' broker in your setup, with the 'clients' both connecting in and
>>> sending or receiving to/from that. You could use this in conjunction with
>>> qpidd (which supports establishing basic AMQP 1.0 links to/from other
>>> processes), or depending on your use case there may not be a real need for
>>> brokers at all.
>>>
>>> (A future release of the router will be able to establish links out to
>>> other 1.0 capable brokers (or other types of 'thing') itself making it even
>>> more flexible/powerful).
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Tor Rune Skoglund <tr...@swi.no>.
Hi Gordon,
I'm working with Chris on this, as he is off-line for the time being, I
post some additional questions:
Could you please expand on how the dispatch router could be incorporated
into our topology? Specifically, we don't see how to set up a route
between brokers via a dispatch router.
We are particularly interested in your comment "...qpidd (which supports
establishing basic AMQP 1.0 links to/from other processes)..."; Chris've
tried things like adding a link from a broker to a router with
"qpid-route link add..." but this produces errors relating to SASL on
the router side and version errors on the broker side. We haven't
pursued this further as we're not sure this is how it's supposed to
work. Is there any documentation or examples you could refer us to?
But to sum up, are there at all any "best practise" ways to solve
"dynamic routing" for spontaneously connected clients without public IPs
with today's (or tomorrow's :-) ) tool-set?
Simplified setup:
[Machine A/Broker A] [Machine C/Broker C]
[ Private IP ] [ Private IP ]
[Out and In Queues ] [Out and In Queues ]
/\ /\
|| ||
\/ \/
[ Priv IP GW ] [ Priv IP GW ]
[ NAT ] [ NAT ]
["Random" pub IP ] ["Random" pub IP ]
/\ /\
|| Unstable connections (e.g. 3G) ||
\/ \/
_________________________________________________
[ S e r v e r P u b l i c IP ]
[ Out and In Queues per Machine ]
[ S e r v e r B r o k e r B ]
[_________________________________________________]
The "IP analogy" would be that Broker A would not know the route to C it
just sends the message off to B (the "default route"), and lets B
forward the message to the right outgoing queue for C.
Best regards,
Tor Rune Skoglund, trs@swi.no
Den 01. april 2014 13:27, skrev Chris Richardson:
> Thanks very much Gordon. I have had some previous endeavours with the
> dispatch router (in fact I started there before switching to the broker)
> and will return to that plan of attack.
>
> I think we will still need brokers in this topology in order to queue
> messages for offline clients, otherwise the router would be the better
> option. I look forward to future releases with the improved broker
> integration!
>
> /Chris
>
>
>
> 2014-03-28 15:52 GMT+00:00 Gordon Sim <gs...@redhat.com>:
>
>> On 03/28/2014 01:36 PM, Chris Richardson wrote:
>>
>>> Ah, now we are really opening Pandora's box! ;)
>>>
>>> The term "client" here is actually quite a simplified term and actually
>>> refers loosely speaking to a system managing a number of products on
>>> that... client. Both the management system and the products should be
>>> individually identifiable and addressable over AMQP; it should also be
>>> possible to address them in groups. Furthermore the client system will
>>> typically be connected over an unreliable network link and the messaging
>>> system is expected to guarantee delivery. Given these constraints I
>>> envisage each client system having at least one local exchange (and queues
>>> to cope with disconnections), and most probably it/they will be topic
>>> exchanges to support the addressing functionality. However I'm very new to
>>> the AMQP routing and addressing concepts so I may be misguided.
>>>
>>
>> Even with dynamic routing, you need to manually set it up per exchange
>> (what it then does is manage the set of routing keys in use for
>> inter-broker links, based on the interest expressed by subscriptions).
>>
>> If you are using topic exchanges you might be able to simply forward all
>> messages for a given exchange through from A->B->C, which you can do by
>> simple static routes per exchange (not actually any more management
>> operations than making setting up dynamic routing for the same exchange).
>> The static routes *should* work ok with 'push' routes (though I will warn
>> that those are not as well used), allowing you to choose the direction of
>> the TCP connection independent of the direction of message flow.
>>
>> Also, without wanting to confuse the picture too much, I'll just note in
>> passing that the recent developments around AMQP 1.0 may be of some
>> interest to your scenario. First off there is the 'Dispatch Router'
>> component. This is superficially similar to a broker, but it only forwards
>> messages it never accepts responsibility for them. It could be used as the
>> 'server' broker in your setup, with the 'clients' both connecting in and
>> sending or receiving to/from that. You could use this in conjunction with
>> qpidd (which supports establishing basic AMQP 1.0 links to/from other
>> processes), or depending on your use case there may not be a real need for
>> brokers at all.
>>
>> (A future release of the router will be able to establish links out to
>> other 1.0 capable brokers (or other types of 'thing') itself making it even
>> more flexible/powerful).
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Chris Richardson <ch...@gmail.com>.
Thanks very much Gordon. I have had some previous endeavours with the
dispatch router (in fact I started there before switching to the broker)
and will return to that plan of attack.
I think we will still need brokers in this topology in order to queue
messages for offline clients, otherwise the router would be the better
option. I look forward to future releases with the improved broker
integration!
/Chris
2014-03-28 15:52 GMT+00:00 Gordon Sim <gs...@redhat.com>:
> On 03/28/2014 01:36 PM, Chris Richardson wrote:
>
>> Ah, now we are really opening Pandora's box! ;)
>>
>> The term "client" here is actually quite a simplified term and actually
>> refers loosely speaking to a system managing a number of products on
>> that... client. Both the management system and the products should be
>> individually identifiable and addressable over AMQP; it should also be
>> possible to address them in groups. Furthermore the client system will
>> typically be connected over an unreliable network link and the messaging
>> system is expected to guarantee delivery. Given these constraints I
>> envisage each client system having at least one local exchange (and queues
>> to cope with disconnections), and most probably it/they will be topic
>> exchanges to support the addressing functionality. However I'm very new to
>> the AMQP routing and addressing concepts so I may be misguided.
>>
>
> Even with dynamic routing, you need to manually set it up per exchange
> (what it then does is manage the set of routing keys in use for
> inter-broker links, based on the interest expressed by subscriptions).
>
> If you are using topic exchanges you might be able to simply forward all
> messages for a given exchange through from A->B->C, which you can do by
> simple static routes per exchange (not actually any more management
> operations than making setting up dynamic routing for the same exchange).
> The static routes *should* work ok with 'push' routes (though I will warn
> that those are not as well used), allowing you to choose the direction of
> the TCP connection independent of the direction of message flow.
>
> Also, without wanting to confuse the picture too much, I'll just note in
> passing that the recent developments around AMQP 1.0 may be of some
> interest to your scenario. First off there is the 'Dispatch Router'
> component. This is superficially similar to a broker, but it only forwards
> messages it never accepts responsibility for them. It could be used as the
> 'server' broker in your setup, with the 'clients' both connecting in and
> sending or receiving to/from that. You could use this in conjunction with
> qpidd (which supports establishing basic AMQP 1.0 links to/from other
> processes), or depending on your use case there may not be a real need for
> brokers at all.
>
> (A future release of the router will be able to establish links out to
> other 1.0 capable brokers (or other types of 'thing') itself making it even
> more flexible/powerful).
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Server-client message relaying with dynamic routes
Posted by Gordon Sim <gs...@redhat.com>.
On 03/28/2014 01:36 PM, Chris Richardson wrote:
> Ah, now we are really opening Pandora's box! ;)
>
> The term "client" here is actually quite a simplified term and actually
> refers loosely speaking to a system managing a number of products on
> that... client. Both the management system and the products should be
> individually identifiable and addressable over AMQP; it should also be
> possible to address them in groups. Furthermore the client system will
> typically be connected over an unreliable network link and the messaging
> system is expected to guarantee delivery. Given these constraints I
> envisage each client system having at least one local exchange (and queues
> to cope with disconnections), and most probably it/they will be topic
> exchanges to support the addressing functionality. However I'm very new to
> the AMQP routing and addressing concepts so I may be misguided.
Even with dynamic routing, you need to manually set it up per exchange
(what it then does is manage the set of routing keys in use for
inter-broker links, based on the interest expressed by subscriptions).
If you are using topic exchanges you might be able to simply forward all
messages for a given exchange through from A->B->C, which you can do by
simple static routes per exchange (not actually any more management
operations than making setting up dynamic routing for the same
exchange). The static routes *should* work ok with 'push' routes (though
I will warn that those are not as well used), allowing you to choose the
direction of the TCP connection independent of the direction of message
flow.
Also, without wanting to confuse the picture too much, I'll just note in
passing that the recent developments around AMQP 1.0 may be of some
interest to your scenario. First off there is the 'Dispatch Router'
component. This is superficially similar to a broker, but it only
forwards messages it never accepts responsibility for them. It could be
used as the 'server' broker in your setup, with the 'clients' both
connecting in and sending or receiving to/from that. You could use this
in conjunction with qpidd (which supports establishing basic AMQP 1.0
links to/from other processes), or depending on your use case there may
not be a real need for brokers at all.
(A future release of the router will be able to establish links out to
other 1.0 capable brokers (or other types of 'thing') itself making it
even more flexible/powerful).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Chris Richardson <ch...@gmail.com>.
Ah, now we are really opening Pandora's box! ;)
The term "client" here is actually quite a simplified term and actually
refers loosely speaking to a system managing a number of products on
that... client. Both the management system and the products should be
individually identifiable and addressable over AMQP; it should also be
possible to address them in groups. Furthermore the client system will
typically be connected over an unreliable network link and the messaging
system is expected to guarantee delivery. Given these constraints I
envisage each client system having at least one local exchange (and queues
to cope with disconnections), and most probably it/they will be topic
exchanges to support the addressing functionality. However I'm very new to
the AMQP routing and addressing concepts so I may be misguided.
2014-03-28 13:00 GMT+00:00 Gordon Sim <gs...@redhat.com>:
> On 03/28/2014 12:53 PM, Gordon Sim wrote:
>
>> On 03/28/2014 11:22 AM, Chris Richardson wrote:
>>
>>> Static routes might be ok for a prototype, but a production system would
>>> have many hundreds or even thousands of clients frequently being added
>>> and
>>> removed. My assumption is that a static configuration would incur a much
>>> higher management overhead?
>>>
>>
>> It really depends on the messaging patterns between the clients. At one
>> extreme, if each client was using the same 'topic' (whether sending or
>> receiving), then the static configuration would be very simple. At the
>> other extreme if each client would require extra routes to be defined,
>> then yes, that would be an unwieldy solution most likely.
>>
>
> Putting the question a better way, how many different exchanges will you
> be using and what will be the type of the exchange(s)?
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Server-client message relaying with dynamic routes
Posted by Gordon Sim <gs...@redhat.com>.
On 03/28/2014 12:53 PM, Gordon Sim wrote:
> On 03/28/2014 11:22 AM, Chris Richardson wrote:
>> Static routes might be ok for a prototype, but a production system would
>> have many hundreds or even thousands of clients frequently being added
>> and
>> removed. My assumption is that a static configuration would incur a much
>> higher management overhead?
>
> It really depends on the messaging patterns between the clients. At one
> extreme, if each client was using the same 'topic' (whether sending or
> receiving), then the static configuration would be very simple. At the
> other extreme if each client would require extra routes to be defined,
> then yes, that would be an unwieldy solution most likely.
Putting the question a better way, how many different exchanges will you
be using and what will be the type of the exchange(s)?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Gordon Sim <gs...@redhat.com>.
On 03/28/2014 11:22 AM, Chris Richardson wrote:
> Static routes might be ok for a prototype, but a production system would
> have many hundreds or even thousands of clients frequently being added and
> removed. My assumption is that a static configuration would incur a much
> higher management overhead?
It really depends on the messaging patterns between the clients. At one
extreme, if each client was using the same 'topic' (whether sending or
receiving), then the static configuration would be very simple. At the
other extreme if each client would require extra routes to be defined,
then yes, that would be an unwieldy solution most likely.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Server-client message relaying with dynamic routes
Posted by Chris Richardson <ch...@gmail.com>.
Static routes might be ok for a prototype, but a production system would
have many hundreds or even thousands of clients frequently being added and
removed. My assumption is that a static configuration would incur a much
higher management overhead?
2014-03-28 10:51 GMT+00:00 Gordon Sim <gs...@redhat.com>:
> On 03/27/2014 03:33 PM, Chris Richardson wrote:
>
>> Hi mailinglist,
>>
>> I'm trying to set up a broker federation topology with a server and (for
>> prototyping) two clients and I need to send messages from one client to
>> the
>> other, routed via the server broker since the clients will be
>> firewalled/NATed and can not communicate directly. My understanding is
>> that
>> the way to do this is with dynamic routing and I've followed the
>> discussion
>> on the following thread with some success:
>> http://qpid.2158936.n2.nabble.com/Dynamic-routing-between-
>> disconnected-exchanges-td7598100.html.
>> That article describes three nodes A, B and C, and relaying from node A to
>> C via B.
>>
>> So far so good - I can use "drain" on node C's test-topic/C
>> exchange/routing key and "spout" to write to node A's test-topic/C
>> exchange/routing key and the message is transferred via the "server" (node
>> B).
>>
>> The problem I have is that this setup relies on TCP links being
>> established
>> in both directions between each node. In my client-server scenario this is
>> clearly not possible and with the network restriction in place the dynamic
>> routing fails. As the documentation states, "A dynamic exchange route is
>> always a pull route. It can never be a push route.". Does this mean that
>> the underlying broker link must be established in the same direction as
>> the
>> route, or is there some way to override this or get the route from the
>> server to utilize the existing link from the client? Solutions involving
>> VPNs and tunnels are "not allowed".
>>
>
> What are the message flows here, i.e. what 'topics' or 'queues' will be
> involved? Do you really need dynamic federation or could a static
> configuration work?
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Server-client message relaying with dynamic routes
Posted by Gordon Sim <gs...@redhat.com>.
On 03/27/2014 03:33 PM, Chris Richardson wrote:
> Hi mailinglist,
>
> I'm trying to set up a broker federation topology with a server and (for
> prototyping) two clients and I need to send messages from one client to the
> other, routed via the server broker since the clients will be
> firewalled/NATed and can not communicate directly. My understanding is that
> the way to do this is with dynamic routing and I've followed the discussion
> on the following thread with some success:
> http://qpid.2158936.n2.nabble.com/Dynamic-routing-between-disconnected-exchanges-td7598100.html.
> That article describes three nodes A, B and C, and relaying from node A to
> C via B.
>
> So far so good - I can use "drain" on node C's test-topic/C
> exchange/routing key and "spout" to write to node A's test-topic/C
> exchange/routing key and the message is transferred via the "server" (node
> B).
>
> The problem I have is that this setup relies on TCP links being established
> in both directions between each node. In my client-server scenario this is
> clearly not possible and with the network restriction in place the dynamic
> routing fails. As the documentation states, "A dynamic exchange route is
> always a pull route. It can never be a push route.". Does this mean that
> the underlying broker link must be established in the same direction as the
> route, or is there some way to override this or get the route from the
> server to utilize the existing link from the client? Solutions involving
> VPNs and tunnels are "not allowed".
What are the message flows here, i.e. what 'topics' or 'queues' will be
involved? Do you really need dynamic federation or could a static
configuration work?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org