[DISCUSS] Clustering in ServiceMix 4

[Guillaume Nodet <gn...@gmail.com>]

I've commited my ongoing work about servicemix 4 clustering at
https://svn.apache.org/repos/asf/servicemix/smx4/nmr/trunk/jbi/cluster/
It's not in the build yet and I committed it for discussion purpose.

This work has two goals:
  * provide some persistence in the JBI layer
  * provide some transparent remoting between JBI endpoints

The way I've began implementing that is to use an ExchangeListener in
the NMR to re-route exchanges to a cluster endpoint (I guess it should
be renamed to something like "cluster engine" to avoid being confused
with "clustered endpoints").

The org.apache.servicemix.cluster.requestor package dervies from the
spring message listener container and implements a jms layer which is
able to provide request / response in an asynchronous way.

I've experimented different things, and the one i've been focusing lately
is to use a single JMS queue and selectors.  Let me explain a bit.
The JMS flow in servicemix 3 was using lots of different destinations
(one per container + one per endpoint + one per service qname + one per
interface qname).  The problem with such a design is that a jms consumer
can easily consume only from one destination (unless we use some specific
activemq features).  Another problem is that if not using activemq,
setting up lots of JMS destinations can be really tedious.  The use of a
single destination leads to fewer consumers, at the expense of using jms
selectors.   Previsouly, i've tried to use two queues (one for requests
and another one for responses) but there's no real benefits in doing this
imho.

The other thing i've been focusing on is to make sure that processing a
jms message does not block a thread, and yet be able to use jms or xa
transactions.  This is not so easy.  For example the spring jms listener
containers do use a thread for consuming the jms message and process it,
expecting the processing to happen synchronously.  However, in servicemix
synchronous processing is a bad idea, as if it involves sending an http
request and waiting for a response, this means blocking a thread for nothing.
For scalability, we need to not block threads if possible.  But spring
message listener containers only support synchronous processing, so I've
hacked two new containers, one being JMS compliant, and another one specific
to ActiveMQ which is much more performant.  It uses a MessageAvailableListener
to be notified when consumers have messages to be processed instead of wasting
threads to poll actively for messages.
Anyway, both containers can support client ack, jms local transactions or
xa transactions in asynchronous mode.

I haven't really worked on how to register such endpoints (from the
user point of
view).  At the moment, we need to register a ClusterRegistration in the OSGi
registry.  Such registrations contains a filter that will be used to
decide if a
new active / consumer exchange should be re-routed to the cluster engine or not.
The most simple filter would be a filter that checks the source endpoint and
will thus cluster all exchanges outgoing from a given endpoint.
As for how to register such objects, one way would be to put that on
the endpoint
exporter that is used in smx4 to register jbi endpoints with the OSGi packaging,
but this would not work with JBI packaging (such registrations would have to be
deployed in a separate osgi bundle).  I was also thinking about adding a simple
boolean property on all endpoints, something like clustered="true".

Sorry for the long rant, but I should have sent this email way earlier ...
Feedback welcome.



-- 
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com