Routing Use Case

[Alex Boisvert <bo...@intalio.com>]

I'd like to pick up on Charles' use-case to validate my understanding 
and possibly define some kind of best practice within JBI since I see 
this pattern come up quite frequently.

To summarize, we have the following environment:

* One consumer component (MyConsumer)
* One routing component (MyRouter)
* Several service provider components (ServiceA, ServiceB, ...) 
implementing the same service type (SameInterface)

What we want is to have a processing pipeline that dynamically picks up 
one of the service provider components based on a policy defined in the 
routing component:

MyConsumer -->  MyRouter --> { ServiceA or ServiceB or ... }

My questions are 1) what are the requirements for each components
and 2) what's the best way to achieve this?

For the sake of identifying the bounds of the JBI specification and the 
added-value of ServiceMix, I'd like to answer those question both for a 
"plain-vanilla" JBI container and for ServiceMix v2.x.

First, I assume all components must handle the message exchange pattern 
defined by the service type (e.g. one-way, request-response, ...). 
Otherwise, I don't think the exchange could be carried out successfully.

Now, is it a requirement of the JBI spec that MyRouter expose 
SameInterface on the JBI bus using ComponentContext#activateEndpoint and 
subsequently Component#getServiceDescription?  If so, does ServiceMix 
require this as well?

I'm going to go out on a limb now and say that MyConsumer cannot be 
(deterministically) linked to MyRouter by relying on service type 
routing because we have more than one component exposing SameInterface. 
  Thus, MyConsumer must specify MyRouter's fully qualified service 
endpoint name.  Is this right?

Based on section 5.4.3.3 of the JBI specification, the service endpoint 
name can be a hard link, a soft link or a standard link as determined by 
in the service unit's meta-data.  I guess the choice between those is a 
deployment preference.  I'd be curious to hear from people who have 
experience about which approach work best for them.

I'll assume MyRouter is driven by some sort of business policy, such as 
content-based routing.  For example, the policy could be "route all 
transactions under $1000 to ServiceA and those equal or over $1000 to 
ServiceB."  MyRouter must therefore be configured with rules and with 
service endpoint names of ServiceA, ServiceB, etc.

Once again, is it a requirement of the JBI spec that service provider 
components (ServiceA, ServiceB, ...) expose SameInterface on the JBI 
bus?  And if so, does ServiceMix enforce this?

Is what I described here a good design?  Is there a simpler way of going 
about it using plain JBI?  I understand ServiceMix simplifies the 
configuration of such pipeline using Spring+XBean but I'd like to 
understand as well the actual requirements if the same components were 
to be moved to another JBI container.

Another question I have is whether it would be advantageous to hook a 
policy at the NMR level instead of having a routing component on the JBI 
bus.

Thoughts or comments anyone?

alex



Guillaume Nodet wrote:
> JBI provides some way to specify routing at deployment time, but not at 
> runtime using the jbi.xml configuration file.
> 
> If you want to do advanced routing in servicemix, there are two ways:
>  * either implement an EndpointChooser and set it on the activation 
> spec, or use it as the default one : the main problem is that
>      they can not be updated at runtime.  We currently have very basic 
> policies : FirstChoicePolicy and RandomChoicePolicy.
>     We may be able provide more advanced policies, but it is imho quite 
> difficult to come up with generic policies.
>     If you have ideas about generic policies to implement, please tell 
> us so that we can discuss them.
>     You can also raise a jira and attach a patch when you have written 
> one :)
> 
>  * using a service engine as a router : the main benefit is that by 
> undeploying / redeploying, you can change the routing policy.
>     This can be done using rules engines, such as drools, or xslt 
> routers.     We can also imagine using an external store and retrieve 
> the updated policies on a regular basis.  The deployment would only
>     consist on a pointer to a set of rules in the external store.
> 
> I think that if the routing rules contains business logic, this should 
> be handled by a service engine and not a policy, to be able to
> manage them easily using jmx.  Policies imho, should be kept for simple 
> policies that can not fail, but this may be argued I guess.
> 
> Cheers,
> Guillaume Nodet
> 
> Charles Souillard wrote:
> 
>> I now well understand your explanation.
>> It helps me.
>>
>> But I am surprised there no way to map OilProvider one time to Total 
>> and another time to Elf...
>> Perhap I will work with Total for 1day and then move for 2 days to Elf 
>> etc...
>> I am surprised to only have the solution to remove the SE !
>> I can also imagine I have many oil products and that I want to ship 
>> 15W40 oil to Elf and 15W50 to Total so I need to have both at the same 
>> time !
>>
>> What I was thinking is a way to configure JBI container to move from 
>> one to the other very quickly. Perhaps this info could be carried in 
>> the message sent through the NMR. Both Elf and Total implement 
>> OilProvider...
>>
>> What do you think about that ?
>>
>> Regards,
>> Charles

Re: Routing Use Case

[Alex Boisvert <bo...@intalio.com>]

Thanks Guillaume for your answers.   A few more comments in-line.

Guillaume Nodet wrote:
>> First, I assume all components must handle the message exchange 
>> pattern defined by the service type (e.g. one-way, request-response, 
>> ...). Otherwise, I don't think the exchange could be carried out 
>> successfully.
> 
> Yes.  Note that such limitations could be expressed by the component by 
> using the isExchangeWithProviderOk and isExchangeWithConsumerOk 
> methods.  But these methods are only called by the NMR and can not be 
> accessed by another component (the router).  If such information may be 
> usefull to you, please raise a JIRA.

Understood.  I don't have a need for this right now.

>> Now, is it a requirement of the JBI spec that MyRouter expose 
>> SameInterface on the JBI bus using ComponentContext#activateEndpoint 
>> and subsequently Component#getServiceDescription?  If so, does 
>> ServiceMix require this as well?
> 
> The JBI spec, in 5.5.2.2, says: " the component MUST supply service 
> metadata concerning services it provides in its implementation of the 
> getServiceDescription(ServiceEndpoint) method ".   ServiceMix does not 
> require such information, but if provided uses the wsdl to retrieve the 
> interfaces implemented by the endpoint.  Note that ServiceMix can only 
> handle WSDL 1 documents.  We are planning to use Apache Woden in the 
> future to parse and use wsdl 2 documents.

Ok, that clarifies it.   I'm looking forward to WSDL 2 support for my 
Axis2 binding component.   More on this later since I don't know if it's 
going to be a strict requirement.

>> I'm going to go out on a limb now and say that MyConsumer cannot be 
>> (deterministically) linked to MyRouter by relying on service type 
>> routing because we have more than one component exposing 
>> SameInterface.  Thus, MyConsumer must specify MyRouter's fully 
>> qualified service endpoint name.  Is this right?
> 
> Depending on how you design your components, you could also use the 
> service name, without the full endpoint, provided that each of your 
> component have a different service name.  But using the ServiceEndpoint 
> is the only way to be sure about the target endpoint.  Note that if your 
> router want to access the list of activated endpoints dynamically (with 
> the JBI API methods) it will receive a list of ServiceEndpoint.

That's also good to know.  I was not clear whether the routing could 
happen with the service name alone, or if a full service endpoint was 
required.  Service name alone works as well (but may resolve to multiple 
endpoints).

>> Based on section 5.4.3.3 of the JBI specification, the service 
>> endpoint name can be a hard link, a soft link or a standard link as 
>> determined by in the service unit's meta-data.  I guess the choice 
>> between those is a deployment preference.  I'd be curious to hear from 
>> people who have experience about which approach work best for them.
> 
> This is a not well tested area of ServiceMix.  Maybe someone has already 
> used that, but I am not sure.

Alright, I'll thread carefully.

>> I'll assume MyRouter is driven by some sort of business policy, such 
>> as content-based routing.  For example, the policy could be "route all 
>> transactions under $1000 to ServiceA and those equal or over $1000 to 
>> ServiceB."  MyRouter must therefore be configured with rules and with 
>> service endpoint names of ServiceA, ServiceB, etc.
>>
>> Once again, is it a requirement of the JBI spec that service provider 
>> components (ServiceA, ServiceB, ...) expose SameInterface on the JBI 
>> bus?  And if so, does ServiceMix enforce this?
> 
> I am not sure of what you are saying: I do not see how ServiceMix (or 
> any other JBI container) can know that two endpoints should expose  the 
> same interface.  The JBI spec says that each endpoint should have a wsdl 
> description, but this description is given by the component.  If your 
> endpoints do not implement the same interface, then routing will behave 
> according to this fact.

You answered my question above.  I wanted to know if ServiceA and 
ServiceB needed to implement getServiceDescription(ServiceEndpoint) and 
they do.

>> Another question I have is whether it would be advantageous to hook a 
>> policy at the NMR level instead of having a routing component on the 
>> JBI bus.
> 
> The pros of using a policy are imho,
>  * you do not have to worry about selecting the possible endpoints : you 
> can leverage interface based routing and the use of isExchangeWithXXXOk 
> methods (if your components implements them)
>  * easy to write
> Cons:
>  * ServiceMix specific
>  * unmanageable (installation / deployment / stats ...)
> The last point is imho the one that makes me think, that if the policy 
> becomes involves business rules, it may be better to use a router.  This 
> may be argued, especially if the rules are not hard-coded, but stored 
> externally.

Good points.  I'll try the component-based router approach and see how 
it goes.

thanks again!
alex

Re: Routing Use Case

[Guillaume Nodet <gu...@worldonline.fr>]

Hi Alex !

See other comments inline

Alex Boisvert wrote:

>
> I'd like to pick up on Charles' use-case to validate my understanding 
> and possibly define some kind of best practice within JBI since I see 
> this pattern come up quite frequently.
>
> To summarize, we have the following environment:
>
> * One consumer component (MyConsumer)
> * One routing component (MyRouter)
> * Several service provider components (ServiceA, ServiceB, ...) 
> implementing the same service type (SameInterface)
>
> What we want is to have a processing pipeline that dynamically picks 
> up one of the service provider components based on a policy defined in 
> the routing component:
>
> MyConsumer -->  MyRouter --> { ServiceA or ServiceB or ... }
>
> My questions are 1) what are the requirements for each components
> and 2) what's the best way to achieve this?
>
> For the sake of identifying the bounds of the JBI specification and 
> the added-value of ServiceMix, I'd like to answer those question both 
> for a "plain-vanilla" JBI container and for ServiceMix v2.x.
>
> First, I assume all components must handle the message exchange 
> pattern defined by the service type (e.g. one-way, request-response, 
> ...). Otherwise, I don't think the exchange could be carried out 
> successfully.

Yes.  Note that such limitations could be expressed by the component by 
using the isExchangeWithProviderOk and isExchangeWithConsumerOk 
methods.  But these methods are only called by the NMR and can not be 
accessed by another component (the router).  If such information may be 
usefull to you, please raise a JIRA.

>
> Now, is it a requirement of the JBI spec that MyRouter expose 
> SameInterface on the JBI bus using ComponentContext#activateEndpoint 
> and subsequently Component#getServiceDescription?  If so, does 
> ServiceMix require this as well?

The JBI spec, in 5.5.2.2, says: " the component MUST supply service 
metadata concerning services it provides in its implementation of the 
getServiceDescription(ServiceEndpoint) method ".   ServiceMix does not 
require such information, but if provided uses the wsdl to retrieve the 
interfaces implemented by the endpoint.  Note that ServiceMix can only 
handle WSDL 1 documents.  We are planning to use Apache Woden in the 
future to parse and use wsdl 2 documents.

>
> I'm going to go out on a limb now and say that MyConsumer cannot be 
> (deterministically) linked to MyRouter by relying on service type 
> routing because we have more than one component exposing 
> SameInterface.  Thus, MyConsumer must specify MyRouter's fully 
> qualified service endpoint name.  Is this right?

Depending on how you design your components, you could also use the 
service name, without the full endpoint, provided that each of your 
component have a different service name.  But using the ServiceEndpoint 
is the only way to be sure about the target endpoint.  Note that if your 
router want to access the list of activated endpoints dynamically (with 
the JBI API methods) it will receive a list of ServiceEndpoint.

>
> Based on section 5.4.3.3 of the JBI specification, the service 
> endpoint name can be a hard link, a soft link or a standard link as 
> determined by in the service unit's meta-data.  I guess the choice 
> between those is a deployment preference.  I'd be curious to hear from 
> people who have experience about which approach work best for them.

This is a not well tested area of ServiceMix.  Maybe someone has already 
used that, but I am not sure. 

>
> I'll assume MyRouter is driven by some sort of business policy, such 
> as content-based routing.  For example, the policy could be "route all 
> transactions under $1000 to ServiceA and those equal or over $1000 to 
> ServiceB."  MyRouter must therefore be configured with rules and with 
> service endpoint names of ServiceA, ServiceB, etc.
>
> Once again, is it a requirement of the JBI spec that service provider 
> components (ServiceA, ServiceB, ...) expose SameInterface on the JBI 
> bus?  And if so, does ServiceMix enforce this?

I am not sure of what you are saying: I do not see how ServiceMix (or 
any other JBI container) can know that two endpoints should expose  the 
same interface.  The JBI spec says that each endpoint should have a wsdl 
description, but this description is given by the component.  If your 
endpoints do not implement the same interface, then routing will behave 
according to this fact.

>
> Is what I described here a good design?  Is there a simpler way of 
> going about it using plain JBI?  I understand ServiceMix simplifies 
> the configuration of such pipeline using Spring+XBean but I'd like to 
> understand as well the actual requirements if the same components were 
> to be moved to another JBI container.
>
> Another question I have is whether it would be advantageous to hook a 
> policy at the NMR level instead of having a routing component on the 
> JBI bus.

The pros of using a policy are imho,
  * you do not have to worry about selecting the possible endpoints : 
you can leverage interface based routing and the use of 
isExchangeWithXXXOk methods (if your components implements them)
  * easy to write
Cons:
  * ServiceMix specific
  * unmanageable (installation / deployment / stats ...)
The last point is imho the one that makes me think, that if the policy 
becomes involves business rules, it may be better to use a router.  This 
may be argued, especially if the rules are not hard-coded, but stored 
externally.

Cheers,
Guillaume Nodet

>
> Thoughts or comments anyone?
>
> alex
>
>
>
> Guillaume Nodet wrote:
>
>> JBI provides some way to specify routing at deployment time, but not 
>> at runtime using the jbi.xml configuration file.
>>
>> If you want to do advanced routing in servicemix, there are two ways:
>>  * either implement an EndpointChooser and set it on the activation 
>> spec, or use it as the default one : the main problem is that
>>      they can not be updated at runtime.  We currently have very 
>> basic policies : FirstChoicePolicy and RandomChoicePolicy.
>>     We may be able provide more advanced policies, but it is imho 
>> quite difficult to come up with generic policies.
>>     If you have ideas about generic policies to implement, please 
>> tell us so that we can discuss them.
>>     You can also raise a jira and attach a patch when you have 
>> written one :)
>>
>>  * using a service engine as a router : the main benefit is that by 
>> undeploying / redeploying, you can change the routing policy.
>>     This can be done using rules engines, such as drools, or xslt 
>> routers.     We can also imagine using an external store and retrieve 
>> the updated policies on a regular basis.  The deployment would only
>>     consist on a pointer to a set of rules in the external store.
>>
>> I think that if the routing rules contains business logic, this 
>> should be handled by a service engine and not a policy, to be able to
>> manage them easily using jmx.  Policies imho, should be kept for 
>> simple policies that can not fail, but this may be argued I guess.
>>
>> Cheers,
>> Guillaume Nodet
>>
>> Charles Souillard wrote:
>>
>>> I now well understand your explanation.
>>> It helps me.
>>>
>>> But I am surprised there no way to map OilProvider one time to Total 
>>> and another time to Elf...
>>> Perhap I will work with Total for 1day and then move for 2 days to 
>>> Elf etc...
>>> I am surprised to only have the solution to remove the SE !
>>> I can also imagine I have many oil products and that I want to ship 
>>> 15W40 oil to Elf and 15W50 to Total so I need to have both at the 
>>> same time !
>>>
>>> What I was thinking is a way to configure JBI container to move from 
>>> one to the other very quickly. Perhaps this info could be carried in 
>>> the message sent through the NMR. Both Elf and Total implement 
>>> OilProvider...
>>>
>>> What do you think about that ?
>>>
>>> Regards,
>>> Charles
>>
>
>
>