You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by James Strachan <ja...@gmail.com> on 2010/10/19 12:06:29 UTC
implementing 'Protocols' or a way to make it easier to black box
routes and compose them with other routes
I've seen quite a few Camel riders doing very similar things over the
years; particularly in cases where teams are sharing routes with other
team or folks want to wrap up 'protocols' as something thats a black
box and easy to cookie-cut. The following is admittedly advanced usage
but seems common enough to be a valid pattern I'd like to describe.
So I was chatting with a nice chap at the recent Paris FuseDay (sorry
I forgot your name, if you're reading this please say Hi and I
apologise for my lousy memory!). We were chatting about what he was
doing and it started ringing lots of bells then on further reflection
we came up with this idea...
So often folks:
* wrap up a number of routes into a protocol bean (say a RouteBuilder)
and parameterize it with fields/properties so that the same 'shape' of
route can be instantiated many times with just different values (e.g.
the names of the queues change or whatever)
* the protocol bean is then instantiated many times using, say,
dependency injection
* sometimes the protocols are totally stand alone as a black box -
however quite often there are some public inputs and outputs for the
black box. i.e. folks want to hide the internal implementation detail
and routes inside the protocol - but want to expose the public inputs
and outputs. Then others can compose the protocols into higher level
protocols.
For example if two teams create protocol A and protocol B then a
different user might wish to route A and B together; without
necessarily having to know all about the internal implementation
detail of A and B along with whatever middleware was used etc.
Basically this is just a description of composing new protocols out of
existing protocols; not unlike Object Orientation where we can nest
things together and have public interface versus private
implementation and so forth. (I guess its not unlike the SCA concept
of composing a hierarchy of assemblies though I've never been much of
an SCA fan ;-).
I was wondering how we can make it easier for the person wiring
together A and B though; without having to dive inside the code of A
and B to figure stuff out. e.g. often A and B just have a single input
and output. Wouldn't it be nice to just do...
from("a://output").to("b://input")
where "input" and "output" are just logical names relative to the
protocols A and B which then map to the actual physical implementation
detail endpoints that the creators of A and B decided to choose. So
think data hiding in OO and having logical alias names for things
inside protocols.
In the above using Camel terminology, we've just got A and B
implemented as Camel Components which then act as a factory of
endpoints (i.e. taking a logical endpoint name and returning the
actual physical endpoint name that the protocol is using)
Implementing Protocols
=================
So the realisation we came to was, we just need to combine a protocol
bean RouteBuilder and a Component into a single, simple bean that we
use Dependency Injection to create and configure. i.e. a single
RouteBuilder can also be a Component as well which can then expose its
public endpoints.
e.g. here's a strawman...
/**
* a simple protocol that does something but has a public
* input and output endpoint if you wish to communicate with the protocol
*/
public class MyProtocol extends ProtocolBuilder {
// variables injected..
private String input;
private String output;
protected void configure() {
// lets expose the public endpoints
alias("input", input);
alias("output", input);
// regular implementation detail routes go here...
from(input).
beanRef("foo", "bar").
to("foo:bar").
to(output);
}
// properties....
...
}
Then folks could expose protocol A and B using Spring XML (these
protocols don't have to be the same class BTW, just reusing it for
brevity)...
<bean id="a" class="MyProtocol">
<property name="input" value="activemq:Some.A.Input"/>
<property name="output" value="activemq:Some.A.Output"/>
</bean>
<bean id="b" class="MyProtocol">
<property name="input" value="mq:BlahBlah"/>
<property name="output" value="file:somethingOrOther"/>
</bean>
Then to route these two 'protocols' together we could refer to the
public aliases (kinda like exported public methods in OO terms) via...
from("a://output").to("b://input")
(I kinda wanted to use the term "export" for these alias endpoints,
but we use export in the DSL for exporting a service to an endpoint -
the server side of "Spring Remoting" type stuff so didn't want to
confuse folks with overusing the same term).
So what does ProtocolBuilder do? Well its just a RouteBuilder
extension which implements the "alias" methods. (Not 100% sure on the
name "alias" yet).
public abstract ProtocolBuilder alias(String logicalName, String
physicalEndpointUri);
public abstract ProtocolBuilder alias(String logicalName, Endpoint
physicalEndpoint);
Then it either needs to implement Component directly or have some kind
of Type Converter to convert ProtocolBuilder to be a Component. The
Component implementation just maps the logical endpoint name like
"input" to whatever physical URI its using.
The basic result is, protocol developers just have a slightly
different base class then an easy way to expose logical endpoints.
Then folks who assemble the protocols get to reuse simple logical URIs
relative to the protocol instances; making the protocol appear a very
simple protocol, hiding all the internal complexity (and protecting
the higher order route from the lower level implementation details).
Thoughts?
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://fusesource.com/
Re: implementing 'Protocols' or a way to make it easier to black box
routes and compose them with other routes
Posted by James Strachan <ja...@gmail.com>.
On 19 October 2010 13:19, James Strachan <ja...@gmail.com> wrote:
> On 19 October 2010 12:54, Willem Jiang <wi...@gmail.com> wrote:
>> On 10/19/10 6:06 PM, James Strachan wrote:
> [snip]
>>> Implementing Protocols
>>> =================
>>>
>>> So the realisation we came to was, we just need to combine a protocol
>>> bean RouteBuilder and a Component into a single, simple bean that we
>>> use Dependency Injection to create and configure. i.e. a single
>>> RouteBuilder can also be a Component as well which can then expose its
>>> public endpoints.
>>>
>>> e.g. here's a strawman...
>>>
>>> /**
>>> * a simple protocol that does something but has a public
>>> * input and output endpoint if you wish to communicate with the protocol
>>> */
>>> public class MyProtocol extends ProtocolBuilder {
>>> // variables injected..
>>> private String input;
>>> private String output;
>>>
>>> protected void configure() {
>>> // lets expose the public endpoints
>>> alias("input", input);
>>> alias("output", input);
>>>
>>> // regular implementation detail routes go here...
>>> from(input).
>>> beanRef("foo", "bar").
>>> to("foo:bar").
>>> to(output);
>>> }
>>>
>>> // properties....
>>> ...
>>> }
>>>
>>> Then folks could expose protocol A and B using Spring XML (these
>>> protocols don't have to be the same class BTW, just reusing it for
>>> brevity)...
>>>
>>> <bean id="a" class="MyProtocol">
>>> <property name="input" value="activemq:Some.A.Input"/>
>>> <property name="output" value="activemq:Some.A.Output"/>
>>> </bean>
>>>
>>> <bean id="b" class="MyProtocol">
>>> <property name="input" value="mq:BlahBlah"/>
>>> <property name="output" value="file:somethingOrOther"/>
>>> </bean>
>>>
>>>
>>> Then to route these two 'protocols' together we could refer to the
>>> public aliases (kinda like exported public methods in OO terms) via...
>>>
>>> from("a://output").to("b://input")
>>
>> With this DSL, I don't think protocol-a's output is connect to the
>> protocol-b's input automatically.
>>
>> Maybe we should override their input or output definition by looking up the
>> position of these two protocols.
>
>
> So the idea is the ProtocolBuilder implements the
> Component.createEndpoint method
> http://camel.apache.org/maven/camel-2.2.0/camel-core/apidocs/org/apache/camel/Component.html#createEndpoint(java.lang.String)
>
> taking the "output" String and aliasing it to whatever real physical
> endpoint the ProtocolBuilder decides to map it to via the
> alias("output", something) method call. Its up to the ProtocolBuilder
> implementation to decide how to map a physical endpoint to a logical
> name.
>
> i.e. the alias() method in this strawman creates the map of logical
> names to physical endpoints used inside the routes; which can then be
> used to resolve endpoint URIs within the protocol instances namespace.
>
>
>>> (I kinda wanted to use the term "export" for these alias endpoints,
>>> but we use export in the DSL for exporting a service to an endpoint -
>>> the server side of "Spring Remoting" type stuff so didn't want to
>>> confuse folks with overusing the same term).
>>>
>>>
>>> So what does ProtocolBuilder do? Well its just a RouteBuilder
>>> extension which implements the "alias" methods. (Not 100% sure on the
>>> name "alias" yet).
>>>
>>> public abstract ProtocolBuilder alias(String logicalName, String
>>> physicalEndpointUri);
>>> public abstract ProtocolBuilder alias(String logicalName, Endpoint
>>> physicalEndpoint);
>>>
>>> Then it either needs to implement Component directly or have some kind
>>> of Type Converter to convert ProtocolBuilder to be a Component. The
>>> Component implementation just maps the logical endpoint name like
>>> "input" to whatever physical URI its using.
>>
>> Maybe a ProtocolComponent can do this job, and the DSL could be
>>
>> from(protocol://a).to(protocol://b);
>
> If there's only one input and output for protocols thats another
> option; though I can imagine many protocols having multiple inputs or
> outputs.
>
> e.g. there might be a dead letter error output (e.g. like stdin /
> stdout / stderr). Some protocols might have multiple inputs (e.g.
> doing a BAM protocol to correlate messages together).
>
> I guess having a protocol endpoint could just assume input and output
> as the default endpoint names. Or maybe we could name the logical
> endpoints within the protocols...
>
> from("protocol://a/input").to("protocol://b/output")
>
> Though given how we need some API to look up in the RouteBuilder to
> find the logical -> physical mappings, I figured it was easiest to
> just add 1 new class which is-a RouteBuilder and is-a Component and so
> knows how to do the URI mapping from public name -> physical endpoint
> and can implement the alias() methods. It would also lead to simpler,
> shorter URIs...
>
> from("a://input").to("b://output")
>
> Then folks can instantiate as many instances of a ProtocolBuilder as
> required - each instance getting its own scheme in Camel and so its
> own logical set of endpoint names which map to a separate set of
> physical endpoints.
>
> Camel already binds a URI scheme to a bean if it can convert itself to
> a Component; I figured protocols was a good use case of using that
> same behaviour, leading to short concise URIs.
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://fusesource.com/
>
Just to close this thread off; if anyone finds it searching: its
implemented here:
http://camel.apache.org/context.html
--
James
-------
FuseSource
Email: james@fusesource.com
Web: http://fusesource.com
Twitter: jstrachan
Blog: http://macstrac.blogspot.com/
Open Source Integration
Re: implementing 'Protocols' or a way to make it easier to black box
routes and compose them with other routes
Posted by James Strachan <ja...@gmail.com>.
On 19 October 2010 12:54, Willem Jiang <wi...@gmail.com> wrote:
> On 10/19/10 6:06 PM, James Strachan wrote:
[snip]
>> Implementing Protocols
>> =================
>>
>> So the realisation we came to was, we just need to combine a protocol
>> bean RouteBuilder and a Component into a single, simple bean that we
>> use Dependency Injection to create and configure. i.e. a single
>> RouteBuilder can also be a Component as well which can then expose its
>> public endpoints.
>>
>> e.g. here's a strawman...
>>
>> /**
>> * a simple protocol that does something but has a public
>> * input and output endpoint if you wish to communicate with the protocol
>> */
>> public class MyProtocol extends ProtocolBuilder {
>> // variables injected..
>> private String input;
>> private String output;
>>
>> protected void configure() {
>> // lets expose the public endpoints
>> alias("input", input);
>> alias("output", input);
>>
>> // regular implementation detail routes go here...
>> from(input).
>> beanRef("foo", "bar").
>> to("foo:bar").
>> to(output);
>> }
>>
>> // properties....
>> ...
>> }
>>
>> Then folks could expose protocol A and B using Spring XML (these
>> protocols don't have to be the same class BTW, just reusing it for
>> brevity)...
>>
>> <bean id="a" class="MyProtocol">
>> <property name="input" value="activemq:Some.A.Input"/>
>> <property name="output" value="activemq:Some.A.Output"/>
>> </bean>
>>
>> <bean id="b" class="MyProtocol">
>> <property name="input" value="mq:BlahBlah"/>
>> <property name="output" value="file:somethingOrOther"/>
>> </bean>
>>
>>
>> Then to route these two 'protocols' together we could refer to the
>> public aliases (kinda like exported public methods in OO terms) via...
>>
>> from("a://output").to("b://input")
>
> With this DSL, I don't think protocol-a's output is connect to the
> protocol-b's input automatically.
>
> Maybe we should override their input or output definition by looking up the
> position of these two protocols.
So the idea is the ProtocolBuilder implements the
Component.createEndpoint method
http://camel.apache.org/maven/camel-2.2.0/camel-core/apidocs/org/apache/camel/Component.html#createEndpoint(java.lang.String)
taking the "output" String and aliasing it to whatever real physical
endpoint the ProtocolBuilder decides to map it to via the
alias("output", something) method call. Its up to the ProtocolBuilder
implementation to decide how to map a physical endpoint to a logical
name.
i.e. the alias() method in this strawman creates the map of logical
names to physical endpoints used inside the routes; which can then be
used to resolve endpoint URIs within the protocol instances namespace.
>> (I kinda wanted to use the term "export" for these alias endpoints,
>> but we use export in the DSL for exporting a service to an endpoint -
>> the server side of "Spring Remoting" type stuff so didn't want to
>> confuse folks with overusing the same term).
>>
>>
>> So what does ProtocolBuilder do? Well its just a RouteBuilder
>> extension which implements the "alias" methods. (Not 100% sure on the
>> name "alias" yet).
>>
>> public abstract ProtocolBuilder alias(String logicalName, String
>> physicalEndpointUri);
>> public abstract ProtocolBuilder alias(String logicalName, Endpoint
>> physicalEndpoint);
>>
>> Then it either needs to implement Component directly or have some kind
>> of Type Converter to convert ProtocolBuilder to be a Component. The
>> Component implementation just maps the logical endpoint name like
>> "input" to whatever physical URI its using.
>
> Maybe a ProtocolComponent can do this job, and the DSL could be
>
> from(protocol://a).to(protocol://b);
If there's only one input and output for protocols thats another
option; though I can imagine many protocols having multiple inputs or
outputs.
e.g. there might be a dead letter error output (e.g. like stdin /
stdout / stderr). Some protocols might have multiple inputs (e.g.
doing a BAM protocol to correlate messages together).
I guess having a protocol endpoint could just assume input and output
as the default endpoint names. Or maybe we could name the logical
endpoints within the protocols...
from("protocol://a/input").to("protocol://b/output")
Though given how we need some API to look up in the RouteBuilder to
find the logical -> physical mappings, I figured it was easiest to
just add 1 new class which is-a RouteBuilder and is-a Component and so
knows how to do the URI mapping from public name -> physical endpoint
and can implement the alias() methods. It would also lead to simpler,
shorter URIs...
from("a://input").to("b://output")
Then folks can instantiate as many instances of a ProtocolBuilder as
required - each instance getting its own scheme in Camel and so its
own logical set of endpoint names which map to a separate set of
physical endpoints.
Camel already binds a URI scheme to a bean if it can convert itself to
a Component; I figured protocols was a good use case of using that
same behaviour, leading to short concise URIs.
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://fusesource.com/
Re: implementing 'Protocols' or a way to make it easier to black
box routes and compose them with other routes
Posted by Willem Jiang <wi...@gmail.com>.
On 10/19/10 6:06 PM, James Strachan wrote:
> I've seen quite a few Camel riders doing very similar things over the
> years; particularly in cases where teams are sharing routes with other
> team or folks want to wrap up 'protocols' as something thats a black
> box and easy to cookie-cut. The following is admittedly advanced usage
> but seems common enough to be a valid pattern I'd like to describe.
>
> So I was chatting with a nice chap at the recent Paris FuseDay (sorry
> I forgot your name, if you're reading this please say Hi and I
> apologise for my lousy memory!). We were chatting about what he was
> doing and it started ringing lots of bells then on further reflection
> we came up with this idea...
>
> So often folks:
>
> * wrap up a number of routes into a protocol bean (say a RouteBuilder)
> and parameterize it with fields/properties so that the same 'shape' of
> route can be instantiated many times with just different values (e.g.
> the names of the queues change or whatever)
>
> * the protocol bean is then instantiated many times using, say,
> dependency injection
>
> * sometimes the protocols are totally stand alone as a black box -
> however quite often there are some public inputs and outputs for the
> black box. i.e. folks want to hide the internal implementation detail
> and routes inside the protocol - but want to expose the public inputs
> and outputs. Then others can compose the protocols into higher level
> protocols.
>
> For example if two teams create protocol A and protocol B then a
> different user might wish to route A and B together; without
> necessarily having to know all about the internal implementation
> detail of A and B along with whatever middleware was used etc.
>
> Basically this is just a description of composing new protocols out of
> existing protocols; not unlike Object Orientation where we can nest
> things together and have public interface versus private
> implementation and so forth. (I guess its not unlike the SCA concept
> of composing a hierarchy of assemblies though I've never been much of
> an SCA fan ;-).
>
> I was wondering how we can make it easier for the person wiring
> together A and B though; without having to dive inside the code of A
> and B to figure stuff out. e.g. often A and B just have a single input
> and output. Wouldn't it be nice to just do...
>
> from("a://output").to("b://input")
>
> where "input" and "output" are just logical names relative to the
> protocols A and B which then map to the actual physical implementation
> detail endpoints that the creators of A and B decided to choose. So
> think data hiding in OO and having logical alias names for things
> inside protocols.
>
> In the above using Camel terminology, we've just got A and B
> implemented as Camel Components which then act as a factory of
> endpoints (i.e. taking a logical endpoint name and returning the
> actual physical endpoint name that the protocol is using)
>
>
> Implementing Protocols
> =================
>
> So the realisation we came to was, we just need to combine a protocol
> bean RouteBuilder and a Component into a single, simple bean that we
> use Dependency Injection to create and configure. i.e. a single
> RouteBuilder can also be a Component as well which can then expose its
> public endpoints.
>
> e.g. here's a strawman...
>
> /**
> * a simple protocol that does something but has a public
> * input and output endpoint if you wish to communicate with the protocol
> */
> public class MyProtocol extends ProtocolBuilder {
> // variables injected..
> private String input;
> private String output;
>
> protected void configure() {
> // lets expose the public endpoints
> alias("input", input);
> alias("output", input);
>
> // regular implementation detail routes go here...
> from(input).
> beanRef("foo", "bar").
> to("foo:bar").
> to(output);
> }
>
> // properties....
> ...
> }
>
> Then folks could expose protocol A and B using Spring XML (these
> protocols don't have to be the same class BTW, just reusing it for
> brevity)...
>
> <bean id="a" class="MyProtocol">
> <property name="input" value="activemq:Some.A.Input"/>
> <property name="output" value="activemq:Some.A.Output"/>
> </bean>
>
> <bean id="b" class="MyProtocol">
> <property name="input" value="mq:BlahBlah"/>
> <property name="output" value="file:somethingOrOther"/>
> </bean>
>
>
> Then to route these two 'protocols' together we could refer to the
> public aliases (kinda like exported public methods in OO terms) via...
>
> from("a://output").to("b://input")
With this DSL, I don't think protocol-a's output is connect to the
protocol-b's input automatically.
Maybe we should override their input or output definition by looking up
the position of these two protocols.
>
> (I kinda wanted to use the term "export" for these alias endpoints,
> but we use export in the DSL for exporting a service to an endpoint -
> the server side of "Spring Remoting" type stuff so didn't want to
> confuse folks with overusing the same term).
>
>
> So what does ProtocolBuilder do? Well its just a RouteBuilder
> extension which implements the "alias" methods. (Not 100% sure on the
> name "alias" yet).
>
> public abstract ProtocolBuilder alias(String logicalName, String
> physicalEndpointUri);
> public abstract ProtocolBuilder alias(String logicalName, Endpoint
> physicalEndpoint);
>
> Then it either needs to implement Component directly or have some kind
> of Type Converter to convert ProtocolBuilder to be a Component. The
> Component implementation just maps the logical endpoint name like
> "input" to whatever physical URI its using.
Maybe a ProtocolComponent can do this job, and the DSL could be
from(protocol://a).to(protocol://b);
>
> The basic result is, protocol developers just have a slightly
> different base class then an easy way to expose logical endpoints.
> Then folks who assemble the protocols get to reuse simple logical URIs
> relative to the protocol instances; making the protocol appear a very
> simple protocol, hiding all the internal complexity (and protecting
> the higher order route from the lower level implementation details).
>
> Thoughts?
>
--
Willem
----------------------------------
Open Source Integration: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: http://twitter.com/willemjiang