You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Noel OConnor <no...@gmail.com> on 2015/11/15 11:51:50 UTC
Waypoints and phases in Dispatch Router
Hi,
I've struggling with understanding the context of phases in waypoints and
how you would use them.
Are phases an AMQP or dispatch routes concept ?
I've taken a look at the code and found the following
>>* A waypoint sends/receives messages to/from an external entity such as a
>>* broker as part of a multi-phase address.
Is a multi-phase address something line a/b/c/d ?
>>* An address can have multiple phases. Each phase acts like a separate
address,
>>* but sharing the same address string.
>>* Phases are not visible to normal senders/receivers, they are set by
>>* waypoints. Messages from normal senders go to the phase=0 address.
Normal
>>* subscribers subscribe to the highest phase defined for the address.
>>* A waypoint takes messages for its in-phase and sends them to the
external
>>* entity. Messages received from the external entity are given the
waypoint's
>>* out-phase. Waypoints can be "chained" with the out-phase of one equal
to the
>>* in-phase for the next. Thus waypoints provide a way to route messages
via
>>* multiple external entities between a sender and a subscriber using the
same
>>* address.
An example of the above would be great ?
Are waypoints a kind of orchestration scheme i.e. routing from one broker
to another ?
thanks for your help
Noel
Re: Waypoints and phases in Dispatch Router
Posted by Noel OConnor <no...@gmail.com>.
Bridge works but it has other connotations within messaging systems
i.e. jms bridges, which might conflict or confuse the intention of the
feature.
I also like "diverts" out of the hornetq/artemis world.
The address-rewriter sounds great but it'd have to be clear where in
the routing pipeline it would be used.
With all the address-rewriting and chaining it might be an idea to be
able to output the processing pipeline thats in place when processing
messages from an address. I suspect that this would be useful in
non-trivial use cases.
On Sat, Nov 21, 2015 at 3:54 AM, aconway <ac...@redhat.com> wrote:
> On Fri, 2015-11-20 at 04:03 +0000, Noel OConnor wrote:
>> Thanks guys, I think this explains it nicely but I'll play around
>> with
>> waypoints some more to ensure I fully understand it. Having read your
>> description I think that it's a nomenclature issue.
>>
>> To me prefixing "waypoints" and "phases" with "processing" i.e.
>> processing
>> phases and processing waypoints makes them clearer in my mind.
>>
>> Cheers
>> Noel
>>
>
> I wrote a user description of the waypoint functionality substituting
> "FOO" for waypoint to try to see what word would fit.
>
> ----
>
> A FOO forwards messages between dispatch and an external AMQP process.
> The connection can be established in either direction, the FOO can
> create multiple sender and/or receiver links to addresses in the
> external process.
>
> For example suppose you want a queue on an external broker to appear as
> the dispatch address 'myqueue'. You would configure a FOO from address
> 'myqueue' to create sender and receiver links to the broker queue.
>
> When dispatch receives a message for 'myqueue' it goes to the FOO,
> which forwards it to the external broker queue. The FOO "consumes" the
> message, it is *not* routed to dispatch network subscribers of
> 'myqueue'. Instead, when the FOO receives a message from its external
> queue subscription, *that* message is forwarded to dispatch
> subscribers. The relationship (if any) between messages out and
> messages in on the bridge is controlled by the external process.
>
> This means you can use arbitrary external processes (provided they
> speak AMQP) to queue, filter, transform or otherwise manipulate
> messages sent via a dispatch address.
>
> In advanced cases, FOO's can be chained to so a message passes through
> multiple external processes before it is finally sent to dispatch
> subscribers.
>
> ----
>
> I think FOO=bridge works pretty well, any other ideas?
>
> Observations:
>
> The non-chained case does not require any phase configuration, it
> should be the default (equivalent to in-phase=0, out-phase=255)
>
> { waypoint: address=myqueue; connector=broker }
>
> Address rewriting: waypoints and link-routes need to derive external
> addresses from internal ones. They are inconsistent (fixedAddrss,
> prefix etc.) We should make address-rewriter an entity and re-use it
> for all such needs. address-rewriter should provide options like
> strip/add prefix/suffix, hard-coded X->Y mappings and maybe pattern
> substitutions.
>
> Chaining: Using integer phases for chaining allows implied fan-out and
> fan-in which I'm not sure was intended or makes sense. I suggest we
> give chained waypoints explicit names or internal addresses to refer to
> each other. I have a feeling that we can generalize "address chaining"
> beyond waypoints but not sure how yet.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Waypoints and phases in Dispatch Router
Posted by aconway <ac...@redhat.com>.
On Fri, 2015-11-20 at 04:03 +0000, Noel OConnor wrote:
> Thanks guys, I think this explains it nicely but I'll play around
> with
> waypoints some more to ensure I fully understand it. Having read your
> description I think that it's a nomenclature issue.
>
> To me prefixing "waypoints" and "phases" with "processing" i.e.
> processing
> phases and processing waypoints makes them clearer in my mind.
>
> Cheers
> Noel
>
I wrote a user description of the waypoint functionality substituting
"FOO" for waypoint to try to see what word would fit.
----
A FOO forwards messages between dispatch and an external AMQP process.
The connection can be established in either direction, the FOO can
create multiple sender and/or receiver links to addresses in the
external process.
For example suppose you want a queue on an external broker to appear as
the dispatch address 'myqueue'. You would configure a FOO from address
'myqueue' to create sender and receiver links to the broker queue.
When dispatch receives a message for 'myqueue' it goes to the FOO,
which forwards it to the external broker queue. The FOO "consumes" the
message, it is *not* routed to dispatch network subscribers of
'myqueue'. Instead, when the FOO receives a message from its external
queue subscription, *that* message is forwarded to dispatch
subscribers. The relationship (if any) between messages out and
messages in on the bridge is controlled by the external process.
This means you can use arbitrary external processes (provided they
speak AMQP) to queue, filter, transform or otherwise manipulate
messages sent via a dispatch address.
In advanced cases, FOO's can be chained to so a message passes through
multiple external processes before it is finally sent to dispatch
subscribers.
----
I think FOO=bridge works pretty well, any other ideas?
Observations:
The non-chained case does not require any phase configuration, it
should be the default (equivalent to in-phase=0, out-phase=255)
{ waypoint: address=myqueue; connector=broker }
Address rewriting: waypoints and link-routes need to derive external
addresses from internal ones. They are inconsistent (fixedAddrss,
prefix etc.) We should make address-rewriter an entity and re-use it
for all such needs. address-rewriter should provide options like
strip/add prefix/suffix, hard-coded X->Y mappings and maybe pattern
substitutions.
Chaining: Using integer phases for chaining allows implied fan-out and
fan-in which I'm not sure was intended or makes sense. I suggest we
give chained waypoints explicit names or internal addresses to refer to
each other. I have a feeling that we can generalize "address chaining"
beyond waypoints but not sure how yet.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Waypoints and phases in Dispatch Router
Posted by Noel OConnor <no...@gmail.com>.
Thanks guys, I think this explains it nicely but I'll play around with
waypoints some more to ensure I fully understand it. Having read your
description I think that it's a nomenclature issue.
To me prefixing "waypoints" and "phases" with "processing" i.e. processing
phases and processing waypoints makes them clearer in my mind.
Cheers
Noel
On Thu, Nov 19, 2015, 08:00 aconway <ac...@redhat.com> wrote:
> On Sun, 2015-11-15 at 10:51 +0000, Noel OConnor wrote:
> > Hi,
> > I've struggling with understanding the context of phases in waypoints
> > and
> > how you would use them.
> > Are phases an AMQP or dispatch routes concept ?
>
> Dispatch only, more below...
>
> > I've taken a look at the code and found the following
> >
> > >>* A waypoint sends/receives messages to/from an external entity
> > such as a
> > >>* broker as part of a multi-phase address.
> >
> > Is a multi-phase address something line a/b/c/d ?
> >
> > >>* An address can have multiple phases. Each phase acts like a
> > separate
> > address,
> > >>* but sharing the same address string.
> >
> > >>* Phases are not visible to normal senders/receivers, they are set
> > by
> > >>* waypoints. Messages from normal senders go to the phase=0
> > address.
> > Normal
> > >>* subscribers subscribe to the highest phase defined for the
> > address.
> >
> > >>* A waypoint takes messages for its in-phase and sends them to the
> > external
> > >>* entity. Messages received from the external entity are given the
> > waypoint's
> > >>* out-phase. Waypoints can be "chained" with the out-phase of one
> > equal
> > to the
> > >>* in-phase for the next. Thus waypoints provide a way to route
> > messages
> > via
> > >>* multiple external entities between a sender and a subscriber
> > using the
> > same
> > >>* address.
> >
> > An example of the above would be great ?
> > Are waypoints a kind of orchestration scheme i.e. routing from one
> > broker
> > to another ?
>
> Yes, exactly! Waypoints are a powerful feature but lots of people find
> them confusing so we may change the way they are expressed in future to
> be more easily understood.
>
> Phases are internal to dispatch, they represent different "stages" in
> the lifecycle of a message associated with just *one* externally
> -visible AMQP address.
>
> Simple example: exposing an AMQP queue as a dispatch address "Q". You
> want messages to the address to go onto the queue, and subscribers to
> the address to subscribe to the queue. You don't want messages sent to
> the dispatch address to be sent directly to the subscribers, bypassing
> the queue. So you have a waypoint with 2 phases. Messages arriving from
> outside of dispatch are Q-0, so they get sent to the queue by the
> waypoint. The waypoint defines a second phase Q-1, for messages coming
> *from* the queue. Subscribers subscribe to Q-1. The external processes
> only know about "Q", the phases are internal to the dispatch network.
>
> More complex example is a "distributed queue", there's code in
> tests/system_tests_broker.py. Senders and subscribers to a single
> external address get load-balanced over a set of broker queues
> automatically.
>
> All the examples so far end up needing only 2 phases, so it's possible
> that allowing N phases is overkill and we can simplify the presentation
> a bit.
>
> Cheers,
> Alan.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Waypoints and phases in Dispatch Router
Posted by aconway <ac...@redhat.com>.
On Sun, 2015-11-15 at 10:51 +0000, Noel OConnor wrote:
> Hi,
> I've struggling with understanding the context of phases in waypoints
> and
> how you would use them.
> Are phases an AMQP or dispatch routes concept ?
Dispatch only, more below...
> I've taken a look at the code and found the following
>
> >>* A waypoint sends/receives messages to/from an external entity
> such as a
> >>* broker as part of a multi-phase address.
>
> Is a multi-phase address something line a/b/c/d ?
>
> >>* An address can have multiple phases. Each phase acts like a
> separate
> address,
> >>* but sharing the same address string.
>
> >>* Phases are not visible to normal senders/receivers, they are set
> by
> >>* waypoints. Messages from normal senders go to the phase=0
> address.
> Normal
> >>* subscribers subscribe to the highest phase defined for the
> address.
>
> >>* A waypoint takes messages for its in-phase and sends them to the
> external
> >>* entity. Messages received from the external entity are given the
> waypoint's
> >>* out-phase. Waypoints can be "chained" with the out-phase of one
> equal
> to the
> >>* in-phase for the next. Thus waypoints provide a way to route
> messages
> via
> >>* multiple external entities between a sender and a subscriber
> using the
> same
> >>* address.
>
> An example of the above would be great ?
> Are waypoints a kind of orchestration scheme i.e. routing from one
> broker
> to another ?
Yes, exactly! Waypoints are a powerful feature but lots of people find
them confusing so we may change the way they are expressed in future to
be more easily understood.
Phases are internal to dispatch, they represent different "stages" in
the lifecycle of a message associated with just *one* externally
-visible AMQP address.
Simple example: exposing an AMQP queue as a dispatch address "Q". You
want messages to the address to go onto the queue, and subscribers to
the address to subscribe to the queue. You don't want messages sent to
the dispatch address to be sent directly to the subscribers, bypassing
the queue. So you have a waypoint with 2 phases. Messages arriving from
outside of dispatch are Q-0, so they get sent to the queue by the
waypoint. The waypoint defines a second phase Q-1, for messages coming
*from* the queue. Subscribers subscribe to Q-1. The external processes
only know about "Q", the phases are internal to the dispatch network.
More complex example is a "distributed queue", there's code in
tests/system_tests_broker.py. Senders and subscribers to a single
external address get load-balanced over a set of broker queues
automatically.
All the examples so far end up needing only 2 phases, so it's possible
that allowing N phases is overkill and we can simplify the presentation
a bit.
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Waypoints and phases in Dispatch Router
Posted by aconway <ac...@redhat.com>.
Agreed with all Ted says below about waypoints. I think we currently
lack a clear overview of all the dispatch concepts - the implementation
is on track, we just need to improve the explanation. Here's a quick
thinking-out-loud attempt to summarize:
----
The key to dispatch's flexibility is the "address". For clients that
produce or consume messages, an address is (mostly) an opaque string.
The power of dispatch is that the meaning of addresses can be
statically or dynamically changed in the dispatch network configuration
without updating large numbers of clients. So it is important to
explain exactly what an "address" is.
First problem: there are two sometimes separate, sometimes overlapping
parts to an "address". First you need a network connection to a
dispatch router, so you need a "connection address" typically an IP
host and port along with SSL or SASL configuration. This part of the
address can be managed with standard IP tricks: from static
configuration to DNS balancing, dynamic IP etc.
Next, dispatch operates on a "routable address". The line between these
is blurred. Often we use a URL-style `amqp://host:port/path` that
contains both. To add to the confusion, it is application-dependent
whether we use only `path` as the routable address, or whether we
include some or all of the `host:port`. Dispatch works either way but
we need some clarification and guidance for users.
For the remainder of this discussion "address" means the dispatch
"routable address". There are several address behaviors that can be
configured.
*Special* addresses:
- addresses starting with `_local/` and `_topo/` are reserved special
dispatch use.
- `$management` is reserved for AMQP standard management
[Question should we follow the AMQP convention and use $local/$topo?]
*Mobile* addresses: these are arbitrary application-defined strings.
There have policy associated with them (e.g. mutlicast, load-balanced
etc.) but are not attached to any point in the network. Any client can
send or receive from anywhere the network dynamically routes according
to policy.
*Link-Routed Addresses*: These addresses are associated with an
external AMQP system such as a broker. You can send or receive from
anywhere in the network, but doing so establishes an AMQP "link" which
acts like a "tunnel" through the network directly to the external
system. No dispatch-level policies apply - dispatch simply acts as a go
-between, the external system decides what to do with the messages.
*Waypoint Addresses*: These addresses are also associated with an
external system or systems but in a more loosely coupled way. Waypoints
can have policy associated and be connected to more than one external
AMQP system on the same logical address.
Link-routes vs. waypoints: link routes are best when you want flexible
but direct connections to existing AMQP systems, for example to connect
directly to a broker without knowing exactly where it is hosted.
Waypoints allow you to create more flexible orchestrations of AMQP
systems. For example you can distribute the load to/from a single
logical address over queues or topics on multiple brokers, without the
clients or the brokers being aware of it.
I have some thoughts about making the 3 address types more consistent,
will send another email
On Mon, 2015-11-16 at 08:29 -0500, Ted Ross wrote:
> Hi Noel,
>
> Phased addresses are a Dispatch Router concept, not an AMQP concept.
>
> To illustrate, think of a queue in a broker called "orders". From
> the
> AMQP standpoint, the address of the queue is "orders", but from the
> router-network standpoint, there are actually two distinct addresses
> that need to be independently routed. Messages produced _to_
> "orders"
> can originate anywhere in the network and must be routed to the
> broker.
> Messages _from_ "orders" originate from the broker and must be
> routed
> to consumers which can be anywhere in the network. So we use the
> convention that produced messages are phase-0 and consumed messages
> are
> phase-1.
>
> To get technical, phases are an annotation on an address that allow
> the
> router fast-path in Dispatch to treat each phase of an address as a
> completely distinct address, with its own routing destinations and
> semantics.
>
> In the above example, a waypoint is provisioned to represent and
> manage
> message flow to and from the queue. Each waypoint has an address and
> (typically) two phases, one for deliveries flowing to the waypoint
> and
> another for deliveries flowing from the waypoint.
>
> It's important to note that from the broker's perspective and also
> that
> of the clients that produce and consume messages, this is all
> transparent. Clients and brokers only use the base address
> ("orders")
> and are unaware of the phased message routing.
>
> So here are a few implications:
>
> 1) You can have multiple waypoints on the same address and with the
> same
> phases (i.e. multiple brokers with queues named "orders" using phases
> 0
> and 1). Dispatch can be configured to anycast (balance) messages to
> and
> from the waypoints, thus balancing the order load across multiple
> brokers in a way that is transparent to clients. Furthermore, the
> number of waypoints can be changed dynamically without the knowledge
> of
> the clients (or brokers) to react to changes in order load. (Note
> that
> the ability to quiesce and remove waypoints is not yet implemented).
>
> 2) Waypoints can be arranged serially (i.e. a 0->1 waypoint can feed
> a
> 1->2 waypoint, etc.). Waypoints can be broker queues or topics.
> They
> can also be arbitrary processes that receive and send messages,
> perhaps
> doing some form of transformation, filtering, or coalescing of data.
>
> 3) Waypoints can be single-phase. You can have a "source" waypoint
> that
> only has a "from" phase. Messages come out but don't go in.
> Likewise,
> a "sink" waypoint can be created that only receives messages.
>
> 4) Different phases of an address can have different semantics (i.e.
> multicast vs. anycast). You can set up a waypoint such that produced
> messages are balanced (anycast) across instances of the waypoint and
> that messages from the waypoint are broadcast to all attached
> consumers.
>
> I hope this helps. Waypoints are still in active development, so
> your
> feedback and ideas are most welcome.
>
> -Ted
>
>
> On 11/15/2015 05:51 AM, Noel OConnor wrote:
> > Hi,
> > I've struggling with understanding the context of phases in
> > waypoints and
> > how you would use them.
> > Are phases an AMQP or dispatch routes concept ?
> >
> > I've taken a look at the code and found the following
> >
> > >>* A waypoint sends/receives messages to/from an external entity
> > such as a
> > >>* broker as part of a multi-phase address.
> >
> > Is a multi-phase address something line a/b/c/d ?
> >
> > >>* An address can have multiple phases. Each phase acts like a
> > separate
> > address,
> > >>* but sharing the same address string.
> >
> > >>* Phases are not visible to normal senders/receivers, they are
> > set by
> > >>* waypoints. Messages from normal senders go to the phase=0
> > address.
> > Normal
> > >>* subscribers subscribe to the highest phase defined for the
> > address.
> >
> > >>* A waypoint takes messages for its in-phase and sends them to
> > the
> > external
> > >>* entity. Messages received from the external entity are given
> > the
> > waypoint's
> > >>* out-phase. Waypoints can be "chained" with the out-phase of
> > one equal
> > to the
> > >>* in-phase for the next. Thus waypoints provide a way to route
> > messages
> > via
> > >>* multiple external entities between a sender and a subscriber
> > using the
> > same
> > >>* address.
> >
> > An example of the above would be great ?
> > Are waypoints a kind of orchestration scheme i.e. routing from one
> > broker
> > to another ?
> >
> > thanks for your help
> > Noel
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Waypoints and phases in Dispatch Router
Posted by Ted Ross <tr...@redhat.com>.
Hi Noel,
Phased addresses are a Dispatch Router concept, not an AMQP concept.
To illustrate, think of a queue in a broker called "orders". From the
AMQP standpoint, the address of the queue is "orders", but from the
router-network standpoint, there are actually two distinct addresses
that need to be independently routed. Messages produced _to_ "orders"
can originate anywhere in the network and must be routed to the broker.
Messages _from_ "orders" originate from the broker and must be routed
to consumers which can be anywhere in the network. So we use the
convention that produced messages are phase-0 and consumed messages are
phase-1.
To get technical, phases are an annotation on an address that allow the
router fast-path in Dispatch to treat each phase of an address as a
completely distinct address, with its own routing destinations and
semantics.
In the above example, a waypoint is provisioned to represent and manage
message flow to and from the queue. Each waypoint has an address and
(typically) two phases, one for deliveries flowing to the waypoint and
another for deliveries flowing from the waypoint.
It's important to note that from the broker's perspective and also that
of the clients that produce and consume messages, this is all
transparent. Clients and brokers only use the base address ("orders")
and are unaware of the phased message routing.
So here are a few implications:
1) You can have multiple waypoints on the same address and with the same
phases (i.e. multiple brokers with queues named "orders" using phases 0
and 1). Dispatch can be configured to anycast (balance) messages to and
from the waypoints, thus balancing the order load across multiple
brokers in a way that is transparent to clients. Furthermore, the
number of waypoints can be changed dynamically without the knowledge of
the clients (or brokers) to react to changes in order load. (Note that
the ability to quiesce and remove waypoints is not yet implemented).
2) Waypoints can be arranged serially (i.e. a 0->1 waypoint can feed a
1->2 waypoint, etc.). Waypoints can be broker queues or topics. They
can also be arbitrary processes that receive and send messages, perhaps
doing some form of transformation, filtering, or coalescing of data.
3) Waypoints can be single-phase. You can have a "source" waypoint that
only has a "from" phase. Messages come out but don't go in. Likewise,
a "sink" waypoint can be created that only receives messages.
4) Different phases of an address can have different semantics (i.e.
multicast vs. anycast). You can set up a waypoint such that produced
messages are balanced (anycast) across instances of the waypoint and
that messages from the waypoint are broadcast to all attached consumers.
I hope this helps. Waypoints are still in active development, so your
feedback and ideas are most welcome.
-Ted
On 11/15/2015 05:51 AM, Noel OConnor wrote:
> Hi,
> I've struggling with understanding the context of phases in waypoints and
> how you would use them.
> Are phases an AMQP or dispatch routes concept ?
>
> I've taken a look at the code and found the following
>
> >>* A waypoint sends/receives messages to/from an external entity such as a
> >>* broker as part of a multi-phase address.
>
> Is a multi-phase address something line a/b/c/d ?
>
> >>* An address can have multiple phases. Each phase acts like a separate
> address,
> >>* but sharing the same address string.
>
> >>* Phases are not visible to normal senders/receivers, they are set by
> >>* waypoints. Messages from normal senders go to the phase=0 address.
> Normal
> >>* subscribers subscribe to the highest phase defined for the address.
>
> >>* A waypoint takes messages for its in-phase and sends them to the
> external
> >>* entity. Messages received from the external entity are given the
> waypoint's
> >>* out-phase. Waypoints can be "chained" with the out-phase of one equal
> to the
> >>* in-phase for the next. Thus waypoints provide a way to route messages
> via
> >>* multiple external entities between a sender and a subscriber using the
> same
> >>* address.
>
> An example of the above would be great ?
> Are waypoints a kind of orchestration scheme i.e. routing from one broker
> to another ?
>
> thanks for your help
> Noel
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org