You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@servicemix.apache.org by Guillaume Nodet <gn...@gmail.com> on 2007/10/11 11:37:17 UTC
Re: ServiceMix 4 as a distributed OSGi spring container....
Bringing this thread to dev list
On 10/11/07, James Strachan <ja...@gmail.com> wrote:
> > We need to find a way to discover other ServiceMix instances (through
> > ActiveMQ ? any other idea ?)
>
> Am pondering if we
> should have a simple POJO based discovery/heartbeat mechanism thats
> transport agnostic so we could use multicast, jmdns, ActiveMQ, HTTP
> etc. Was pondering about making something really simple in Camel/SMX4
> and we take it from there etc.
>
> e.g. for a heartbeat we really just wanna do a timed send of a status
> message to some endpoint. This could be JMS or a POST to some URI (or
> even an AtomPub entry we POST).
>
> Then the server side of this thing, keeps a RAM cache of all these
> things and times the entires out after a while so they disappear.
> (Doesn't need any persistence or replication really; as if the server
> side fails, another one can start somewhere and after a few heartbeats
> its back in sync again).
>
> Then the server side - lets call it a registry (but a simple
> registry); can just expose these heartbeats as data in different
> queries. So folks can query using some mechanism for nodes of a
> certain type or whatnot.
Let's rephrase to check there is no misunderstanding.
The purpose here is to have a discovery mechanism which is somewhat
independant of the protocol. So there is one server and multiple
clients, each client sending its own information to the server that
keeps track of them.
This implies, that either
* there is a single point of failure for the master which is
statically configured
* the client elect a master amongst themselves
Once the master / registry has been identified, the client send
heatbeats to it and can query it for services. I'm not sure if a push
or pull model is better here: having the master send new information
to the clients, or have the clients polling for new informations or
simply looking up the service on demand.
The problem with the pull approach is that we won't be able to expose
remote services in the OSGi registry without hacking the registry
itself. For this particular use case, it would be better to be
notified of new services (or subscribing somehow) so that a client can
register a new remote service in the OSGi registry, which would be
available through a proxy.
>
> This heartbeat/discovery thing is pretty generic stuff really; it'd be
> nice if the message/data could be anything at all then we can reuse
> the same thing in many circumstances. e.g. we could use this to
> discover AMQ brokers; or CXF services of a kind of WSDL or SMX
> instances or OSGi containers or whatnot.
+1
>
> The blob of data that is heartbeated; and what kinds of queries on the
> data we need to make is kinda speciifc to different problems. e.g. in
> AMQ we mostly just need to get the URL to connect to the broker. We
> may want to include some kinda network-hop type info so we try to
> connect to the nearest node or something - but generally a URL is
> enough in AMQ-land for discovery.
>
> For SMX containers, we might as well just expose the URL to talk to it.
>
> For distributed OSGi stuff we might wanna use more complex data for
> the heartbeat data; so folks could search (rather like the NMR does)
> by service QName, interface name, endpoint name or whatever.
>
>
> From a low level I was wondering about a Camel factory bean in spring
>
> <camel:heartbeat bean="someBeanRef" property="myStatusEventObject"
> millis="2000" uri="http://someServer.com/cheese"/>
>
> This would call someBeanRef.getMyStatusEventObject() every 2 seconds
> and send the POJO to the endpoint URI. We could maybe wrap up this
> using some route? e.g. if the bean endpoint allowed for polling we
> could maybe do
>
> from("bean:someBeanRef?method=getMyStatusEventObject&period=2000").to("http://someServer.com/cheese")
>
> though maybe a factory bean (XML element) is a bit easier - it does
> have better smart completion at least :)
>
I like the idea of using camel to configure the discovery mechanism.
This would be very flexible.
>
> BTW since servicemix is using activemq; probably the easiest discovery
> we should use out of th ebox is using the JMS network - i.e. just to
> use a topic for the heartbeats; then each node can have a local
> in-memory 'registry'. Another option is to have a web app which is the
> registry and clients POST heartbeats and then GET data back - which is
> more useful for folks with firewalls or multi-language clients and so
> forth. I guess those are the big 2 implementations; web based or JMS
> based.
>
Sounds good. But these are very different topologies, the first one
has no centralized registry, whereas the second one has. How can we
bridge both approaches transparently ?
>
>
> > and thus make the OSGi registries
> > distributed. Services that are marked somehow would be exported as a
> > service available remotely (I don't think it would be a good idea to
> > make all services available remotely).
>
> Yeah. We could require "export=true" or something in the service
> metadata or something. Maybe allow custom predicates on the service
> metadata to define profiles of things to distribute.
>
Sounds good.
>
> > As for JAXWS, we need some strategies to deploy them (maybe we could
> > rely on spring), as I'm not sure auto-discovering the beans it always
> > a good idea (might be pluggable though).
>
> Yeah; am thinking if folks use spring to deploy them we could maybe
> hook in to, say, CXF to grok the WSDL bits and expose those on the NMR
> automagically if you see what I mean.
>
Yeah.
> What would be awesome is if we can deploy various things (JAX-WS, EJB3
> and so forth) using Spring+OSGi and we have a way to auto-discover
> things and expose them on the OSGi NMR type stuff (and so in JBI too).
>
> i.e. rather than folks having to put their JAX-WS / EJB3 / SCA stuff
> into a service engine; it'd be nice if we had a way of kinda binding
> to them direclty via OSGi. Maybe wishful thinking though - I'm just
> wondering if we can kinda make the OSGiNMR/JBI stuff invisible and for
> it to auto-hook into whatever folks are actually using. Even if we can
> just find a way of getting CXF services when deployed as OSGi bundles
> to be discoverable so we can export 'em on the OSGi NMR thingy it'd be
> a big win for CXF users.
Agreed. What I was thinking was along those lines. That's the main
reason why the ServiceMix api does not have any notion of "component"
as in JBI. Instead we would create OSGi deployers (baically a bundle
listener or service listener). For example to activate a ServiceMix
endpoint, it's just a matter of registering it in the OSGi registry.
For JAX-WS, we could add a spring extension for deploying a bean,
kinda like the jaxws namespace for cxf
(http://cwiki.apache.org/CXF20DOC/jax-ws-configuration.html).
So exposing a bean in that way would perform all the necessary steps
to register on as a JBI endpoint and make it available remotely (maybe
we need a boolean to say if we want to export it remotely or not)
>
> BTW - here's a thought - should the only thing we distribute remotely
> be the entries in the OSGi NMR? i.e. stuff that we have a kinda WSDL
> for and so forth? i.e. rather than any old Java object in OSGi, we
> kinda define a contract of what can go onto OSGi NMR (so there's some
> kinda WSDL port, binding and so forth) then focus our distribution
> stuff on the OSGi NMRs in each vm?
>
This is a good question. But going through a common way to distribute
the services as some pros. But we could easily hide all the
ServiceMix stuff as I described above. But I agree WSDL and all
should be optional, but they are already in the ServiceMix api. I
think we can use Aegis data binding for non-jaxws POJOs that we want
to expose internally or externally.
>
> > Such beans could be exposed
> > on both Smx registry as an endpoint and externally through the above
> > mechanism as a soap http service.
>
> Agreed
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source SOA
> http://open.iona.com
>
--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by James Strachan <ja...@gmail.com>.
BTW there's a simple example of how you could write a pure POJO
registry without any dependency on any particular middleware or API
and wire it together with services & heartbeating with Camel here...
https://svn.apache.org/repos/asf/activemq/camel/trunk/components/camel-jms/src/test/java/org/apache/camel/component/jms/discovery/
(though it doesn't do the timeout of heartbeat messages - it was just
a simple example to show how things could be wired together really)
On 11/10/2007, James Strachan <ja...@gmail.com> wrote:
> On 11/10/2007, Guillaume Nodet <gn...@gmail.com> wrote:
> > I was thinking that ldap may be handy for the registry, but hopefully
> > Chris will join the discussion at this point... Though camel does
> > not support ldap (yet).
> >
> > So your snippet would actually solve the heartbeat problem. But I'm
> > not sure we can send the whole data at each heartbeat. I guess it
> > depends how bit this data is, but if we have lots of services in the
> > OSGi registry, it may not be very scalable. So we would have to
> > default to send only updates or find another mechanism to send the
> > data (the heartbeat could just contain the url of our container, and
> > the data would be retrieved by another mechanism).
>
> Yeah; it does depend on how much data we're talking about. We could
> slice and dice the data using URLs.
>
> e.g. the full list of services available could be posted to some, say,
> Atom document on some shared server; this list can be updated
> incrementally or in total as and when services are added or removed.
>
> Then the heartbeat could just send around a URL to the detailed information.
>
> Or the list of services available could just be dynamically generated
> on demand if the container exposed a web front end. (push v pull)
>
> From a client perspective, it doesn't much care - the heartbeat
> message contains a URL to the detailed list of actual services
> provided if it wishes to get more information etc.
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source SOA
> http://open.iona.com
>
--
James
-------
http://macstrac.blogspot.com/
Open Source SOA
http://open.iona.com
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by Gert Vanthienen <ge...@skynet.be>.
L.S.,
I really like the way ServiceMix 4.0 will be built around a set of core
technologies (OSGi, ActiveMQ, Camel, CXF). It's a very good idea to
leverage these technologies also for our own 'internal' operations
wherever possible and the Camel snippet that was shown has a very nice
touch of simplicity/readability to it.
I also like the suggestion of sending only a URL of the container in the
heartbeat, thus limiting the size of the heartbeat message, while still
allowing the other node to retrieve the data it needs. Perhaps I'm
completely missing the point here, but ... Wouldn't it be possible to
also include 'update notifications' in the heartbeat message this way?
Whenever a new service (or a bundle of services) becomes available, an
update notification is sent with a URL to allow the other nodes to only
retrieve information about what has changed. This way, the node
receiving the heartbeat can update it's own internal registry to reflect
changes that have occurred in the other nodes.
Gert
James Strachan wrote:
> On 11/10/2007, Guillaume Nodet <gn...@gmail.com> wrote:
>
>> I was thinking that ldap may be handy for the registry, but hopefully
>> Chris will join the discussion at this point... Though camel does
>> not support ldap (yet).
>>
>> So your snippet would actually solve the heartbeat problem. But I'm
>> not sure we can send the whole data at each heartbeat. I guess it
>> depends how bit this data is, but if we have lots of services in the
>> OSGi registry, it may not be very scalable. So we would have to
>> default to send only updates or find another mechanism to send the
>> data (the heartbeat could just contain the url of our container, and
>> the data would be retrieved by another mechanism).
>>
>
> Yeah; it does depend on how much data we're talking about. We could
> slice and dice the data using URLs.
>
> e.g. the full list of services available could be posted to some, say,
> Atom document on some shared server; this list can be updated
> incrementally or in total as and when services are added or removed.
>
> Then the heartbeat could just send around a URL to the detailed information.
>
> Or the list of services available could just be dynamically generated
> on demand if the container exposed a web front end. (push v pull)
>
> >From a client perspective, it doesn't much care - the heartbeat
> message contains a URL to the detailed list of actual services
> provided if it wishes to get more information etc.
>
>
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by James Strachan <ja...@gmail.com>.
On 11/10/2007, Guillaume Nodet <gn...@gmail.com> wrote:
> I was thinking that ldap may be handy for the registry, but hopefully
> Chris will join the discussion at this point... Though camel does
> not support ldap (yet).
>
> So your snippet would actually solve the heartbeat problem. But I'm
> not sure we can send the whole data at each heartbeat. I guess it
> depends how bit this data is, but if we have lots of services in the
> OSGi registry, it may not be very scalable. So we would have to
> default to send only updates or find another mechanism to send the
> data (the heartbeat could just contain the url of our container, and
> the data would be retrieved by another mechanism).
Yeah; it does depend on how much data we're talking about. We could
slice and dice the data using URLs.
e.g. the full list of services available could be posted to some, say,
Atom document on some shared server; this list can be updated
incrementally or in total as and when services are added or removed.
Then the heartbeat could just send around a URL to the detailed information.
Or the list of services available could just be dynamically generated
on demand if the container exposed a web front end. (push v pull)
>From a client perspective, it doesn't much care - the heartbeat
message contains a URL to the detailed list of actual services
provided if it wishes to get more information etc.
--
James
-------
http://macstrac.blogspot.com/
Open Source SOA
http://open.iona.com
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by Bruce Snyder <br...@gmail.com>.
On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
> On 10/11/07, Bruce Snyder <br...@gmail.com> wrote:
> > On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
> > > I was thinking that ldap may be handy for the registry, but hopefully
> > > Chris will join the discussion at this point... Though camel does
> > > not support ldap (yet).
> >
> > This was exactly the road I started going down in order to solve a
> > couple of problems:
> >
> > 1) Distributing a consistent configuration across groups of nodes
> > 2) Providing for a central registry that is replicated to other
> > directory server instances
> >
> > This would optionally require a master directory server with other
> > backup or slave servers in order to replicate the registry data. The
> > size of the network and the criticality of the data would determine if
> > you need to run slave servers or not.
> >
> > The other thing I began thinking about was using the AMQ master/slave
> > functionality and just embedding the directory server in the master
> > and the slaves. This would mean less moving parts and we could make
> > any LDAP replications take place via AMQ transports.
>
> Doesn't apacheds comes with a clustering / replication solution already ?
Yes, it does, but I'm not sure of it's level of efficiency at all as
I've not yet looked at it. I have spoken with Ersin Er who works on
directory about doing directory replication via ActiveMQ, but we
haven't done any work toward it yet.
> > > So your snippet would actually solve the heartbeat problem. But I'm
> > > not sure we can send the whole data at each heartbeat. I guess it
> > > depends how bit this data is, but if we have lots of services in the
> > > OSGi registry, it may not be very scalable. So we would have to
> > > default to send only updates or find another mechanism to send the
> > > data (the heartbeat could just contain the url of our container, and
> > > the data would be retrieved by another mechanism).
> >
> > How about just sending the URL and a flag stating that there is an
> > update? If one of the other servers wants the update, then they can
> > poll that server's URL and a known topic for the actual data updates.
>
> Yeah. However, for performance reasons, it may be interesting to be
> able to only publish the delta in addition to provide an easy access
> to the whole data.
Yeah, I really like that idea. It should cut down on traffic significantly.
> We could also expose the registry as a REST interface, but it may not
> be the most efficient way.
Do we really need to expose the whole registry though?
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache ServiceMix - http://servicemix.org/
Apache Geronimo - http://geronimo.apache.org/
Castor - http://castor.org/
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by Chris Custine <cc...@apache.org>.
On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
>
> On 10/11/07, Bruce Snyder <br...@gmail.com> wrote:
> > On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
> > > I was thinking that ldap may be handy for the registry, but hopefully
> > > Chris will join the discussion at this point... Though camel does
> > > not support ldap (yet).
> >
> > This was exactly the road I started going down in order to solve a
> > couple of problems:
> >
> > 1) Distributing a consistent configuration across groups of nodes
> > 2) Providing for a central registry that is replicated to other
> > directory server instances
What I am trying to get my head around, is if this configuration data can be
abstracted up to the OSGi level or if it is specific to ServiceMix? I see
the specific requirement for this data within ServiceMix itself, but with
OSGi there is also the configuration admin service which is a standardized
way to provide data to instances of services. I have been thinking about
implementing a distributed version of these kinds of OSGi services for a
long time so this could be another option. If nothing else, we could
support a pluggable registry that could have different implementations
depending on the deployment scenario. Just a thought.
>
> > This would optionally require a master directory server with other
> > backup or slave servers in order to replicate the registry data. The
> > size of the network and the criticality of the data would determine if
> > you need to run slave servers or not.
> >
> > The other thing I began thinking about was using the AMQ master/slave
> > functionality and just embedding the directory server in the master
> > and the slaves. This would mean less moving parts and we could make
> > any LDAP replications take place via AMQ transports.
>
> Doesn't apacheds comes with a clustering / replication solution already ?
Yes, and I can look into modifying this to use ActiveMQ to do the
replication so that we can keep things consistent (current replication is
direct TCP with Mina on each node). ApacheDS also allows replication
without exposing the actual LDAP server... so it can be embedded and expose
a direct JNDIContext without exposing a listener externally yet still
achieve replication, which is a common use case for embedded users.
>
> > > So your snippet would actually solve the heartbeat problem. But I'm
> > > not sure we can send the whole data at each heartbeat. I guess it
> > > depends how bit this data is, but if we have lots of services in the
> > > OSGi registry, it may not be very scalable. So we would have to
> > > default to send only updates or find another mechanism to send the
> > > data (the heartbeat could just contain the url of our container, and
> > > the data would be retrieved by another mechanism).
> >
> > How about just sending the URL and a flag stating that there is an
> > update? If one of the other servers wants the update, then they can
> > poll that server's URL and a known topic for the actual data updates.
>
> Yeah. However, for performance reasons, it may be interesting to be
> able to only publish the delta in addition to provide an easy access
> to the whole data.
I think ApacheDS does this now, but I am sure that it could be optimized to
fit our needs if it doesn't already.
We could also expose the registry as a REST interface, but it may not
> be the most efficient way.
>
> >
> > Bruce
> > --
> > perl -e 'print
> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> > );'
> >
> > Apache ActiveMQ - http://activemq.org/
> > Apache ServiceMix - http://servicemix.org/
> > Apache Geronimo - http://geronimo.apache.org/
> > Castor - http://castor.org/
> >
>
>
> --
> Cheers,
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
>
I certainly think that ApacheDS could be of some use here, but I don't want
to be pushy or biased ;-) There will certainly be some customizations for
this use case, but ApacheDS is extensible via interceptors and other similar
means. In addition, it might not be well known that ApacheDS is going to
start moving to an OSGi container based deployment as well, so that might be
useful too. I think these discussions are great and will help expose the
requirements as we toss around these ideas and we can keep fine tuning the
solution.
Chris
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by Guillaume Nodet <gn...@gmail.com>.
On 10/11/07, Bruce Snyder <br...@gmail.com> wrote:
> On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
> > I was thinking that ldap may be handy for the registry, but hopefully
> > Chris will join the discussion at this point... Though camel does
> > not support ldap (yet).
>
> This was exactly the road I started going down in order to solve a
> couple of problems:
>
> 1) Distributing a consistent configuration across groups of nodes
> 2) Providing for a central registry that is replicated to other
> directory server instances
>
> This would optionally require a master directory server with other
> backup or slave servers in order to replicate the registry data. The
> size of the network and the criticality of the data would determine if
> you need to run slave servers or not.
>
> The other thing I began thinking about was using the AMQ master/slave
> functionality and just embedding the directory server in the master
> and the slaves. This would mean less moving parts and we could make
> any LDAP replications take place via AMQ transports.
Doesn't apacheds comes with a clustering / replication solution already ?
>
> > So your snippet would actually solve the heartbeat problem. But I'm
> > not sure we can send the whole data at each heartbeat. I guess it
> > depends how bit this data is, but if we have lots of services in the
> > OSGi registry, it may not be very scalable. So we would have to
> > default to send only updates or find another mechanism to send the
> > data (the heartbeat could just contain the url of our container, and
> > the data would be retrieved by another mechanism).
>
> How about just sending the URL and a flag stating that there is an
> update? If one of the other servers wants the update, then they can
> poll that server's URL and a known topic for the actual data updates.
Yeah. However, for performance reasons, it may be interesting to be
able to only publish the delta in addition to provide an easy access
to the whole data.
We could also expose the registry as a REST interface, but it may not
be the most efficient way.
>
> Bruce
> --
> perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> );'
>
> Apache ActiveMQ - http://activemq.org/
> Apache ServiceMix - http://servicemix.org/
> Apache Geronimo - http://geronimo.apache.org/
> Castor - http://castor.org/
>
--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by James Strachan <ja...@gmail.com>.
On 11/10/2007, Bruce Snyder <br...@gmail.com> wrote:
> On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
> > So your snippet would actually solve the heartbeat problem. But I'm
> > not sure we can send the whole data at each heartbeat. I guess it
> > depends how bit this data is, but if we have lots of services in the
> > OSGi registry, it may not be very scalable. So we would have to
> > default to send only updates or find another mechanism to send the
> > data (the heartbeat could just contain the url of our container, and
> > the data would be retrieved by another mechanism).
>
> How about just sending the URL and a flag stating that there is an
> update? If one of the other servers wants the update, then they can
> poll that server's URL and a known topic for the actual data updates.
Great idea
--
James
-------
http://macstrac.blogspot.com/
Open Source SOA
http://open.iona.com
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by Bruce Snyder <br...@gmail.com>.
On 10/11/07, David Jencks <da...@yahoo.com> wrote:
> > The other thing I began thinking about was using the AMQ master/slave
> > functionality and just embedding the directory server in the master
> > and the slaves. This would mean less moving parts and we could make
> > any LDAP replications take place via AMQ transports.
>
> why would this be better than using the built in ldap replication?
> (mitosis in apacheds) This is not a leading question, I really don't
> know :-)
The only reason I said this is because I haven't yet looked at the
replication code in ApacheDS. I have looked at the marshaling code in
ActiveMQ and I know it is highly optimized and very well tested.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache ServiceMix - http://servicemix.org/
Apache Geronimo - http://geronimo.apache.org/
Castor - http://castor.org/
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by David Jencks <da...@yahoo.com>.
On Oct 11, 2007, at 12:42 PM, Bruce Snyder wrote:
> On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
>> I was thinking that ldap may be handy for the registry, but hopefully
>> Chris will join the discussion at this point... Though camel does
>> not support ldap (yet).
>
> This was exactly the road I started going down in order to solve a
> couple of problems:
>
> 1) Distributing a consistent configuration across groups of nodes
> 2) Providing for a central registry that is replicated to other
> directory server instances
>
> This would optionally require a master directory server with other
> backup or slave servers in order to replicate the registry data. The
> size of the network and the criticality of the data would determine if
> you need to run slave servers or not.
>
> The other thing I began thinking about was using the AMQ master/slave
> functionality and just embedding the directory server in the master
> and the slaves. This would mean less moving parts and we could make
> any LDAP replications take place via AMQ transports.
why would this be better than using the built in ldap replication?
(mitosis in apacheds) This is not a leading question, I really don't
know :-)
thanks
david jencks
>
>> So your snippet would actually solve the heartbeat problem. But I'm
>> not sure we can send the whole data at each heartbeat. I guess it
>> depends how bit this data is, but if we have lots of services in the
>> OSGi registry, it may not be very scalable. So we would have to
>> default to send only updates or find another mechanism to send the
>> data (the heartbeat could just contain the url of our container, and
>> the data would be retrieved by another mechanism).
>
> How about just sending the URL and a flag stating that there is an
> update? If one of the other servers wants the update, then they can
> poll that server's URL and a known topic for the actual data updates.
>
> Bruce
> --
> perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!
> G;6%I;\"YC;VT*"
> );'
>
> Apache ActiveMQ - http://activemq.org/
> Apache ServiceMix - http://servicemix.org/
> Apache Geronimo - http://geronimo.apache.org/
> Castor - http://castor.org/
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by Bruce Snyder <br...@gmail.com>.
On 10/11/07, Guillaume Nodet <gn...@gmail.com> wrote:
> I was thinking that ldap may be handy for the registry, but hopefully
> Chris will join the discussion at this point... Though camel does
> not support ldap (yet).
This was exactly the road I started going down in order to solve a
couple of problems:
1) Distributing a consistent configuration across groups of nodes
2) Providing for a central registry that is replicated to other
directory server instances
This would optionally require a master directory server with other
backup or slave servers in order to replicate the registry data. The
size of the network and the criticality of the data would determine if
you need to run slave servers or not.
The other thing I began thinking about was using the AMQ master/slave
functionality and just embedding the directory server in the master
and the slaves. This would mean less moving parts and we could make
any LDAP replications take place via AMQ transports.
> So your snippet would actually solve the heartbeat problem. But I'm
> not sure we can send the whole data at each heartbeat. I guess it
> depends how bit this data is, but if we have lots of services in the
> OSGi registry, it may not be very scalable. So we would have to
> default to send only updates or find another mechanism to send the
> data (the heartbeat could just contain the url of our container, and
> the data would be retrieved by another mechanism).
How about just sending the URL and a flag stating that there is an
update? If one of the other servers wants the update, then they can
poll that server's URL and a known topic for the actual data updates.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache ServiceMix - http://servicemix.org/
Apache Geronimo - http://geronimo.apache.org/
Castor - http://castor.org/
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by Guillaume Nodet <gn...@gmail.com>.
I was thinking that ldap may be handy for the registry, but hopefully
Chris will join the discussion at this point... Though camel does
not support ldap (yet).
So your snippet would actually solve the heartbeat problem. But I'm
not sure we can send the whole data at each heartbeat. I guess it
depends how bit this data is, but if we have lots of services in the
OSGi registry, it may not be very scalable. So we would have to
default to send only updates or find another mechanism to send the
data (the heartbeat could just contain the url of our container, and
the data would be retrieved by another mechanism).
On 10/11/07, James Strachan <ja...@gmail.com> wrote:
> Interesting discussion :)
>
> On 11/10/2007, James Strachan <ja...@gmail.com> wrote:
> > On 11/10/2007, Guillaume Nodet <gn...@gmail.com> wrote:
> > > Bringing this thread to dev list
> > >
> > > On 10/11/07, James Strachan <ja...@gmail.com> wrote:
> > > > > We need to find a way to discover other ServiceMix instances (through
> > > > > ActiveMQ ? any other idea ?)
>
> [big snip]
>
> > > > From a low level I was wondering about a Camel factory bean in spring
> > > >
> > > > <camel:heartbeat bean="someBeanRef" property="myStatusEventObject"
> > > > millis="2000" uri="http://someServer.com/cheese"/>
> > > >
> > > > This would call someBeanRef.getMyStatusEventObject() every 2 seconds
> > > > and send the POJO to the endpoint URI. We could maybe wrap up this
> > > > using some route? e.g. if the bean endpoint allowed for polling we
> > > > could maybe do
> > > >
> > > > from("bean:someBeanRef?method=getMyStatusEventObject&period=2000").to("http://someServer.com/cheese")
> > > >
> > > > though maybe a factory bean (XML element) is a bit easier - it does
> > > > have better smart completion at least :)
> > > >
> > >
> > > I like the idea of using camel to configure the discovery mechanism.
> > > This would be very flexible.
> >
> > Yeah; am thinking we could do content based routing or message
> > translator stuff to format the POJO of the heartbeat data into
> > different formats, sending it to different locations etc
>
> As an experiment in implementing a POJO based heartbeat mechanism
> thats protocol and data format independent (and can support the easy
> integration of EIP) I thought I'd try write a little test case in
> Camel.
>
> It turned out to be surprisingly easy :)
>
> from("bean:myService?methodName=status").to("mock:result");
>
> where the status() method is invoked periodically on the myService
> bean and then sent to the registry endpoint (a mock in this case as
> its a test case, but this could be activemq:topic:foo.bar etc)
>
> Here's the test case if you're interested...
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/BeanMethodHeartbeatTest.java
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source SOA
> http://open.iona.com
>
--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by James Strachan <ja...@gmail.com>.
Interesting discussion :)
On 11/10/2007, James Strachan <ja...@gmail.com> wrote:
> On 11/10/2007, Guillaume Nodet <gn...@gmail.com> wrote:
> > Bringing this thread to dev list
> >
> > On 10/11/07, James Strachan <ja...@gmail.com> wrote:
> > > > We need to find a way to discover other ServiceMix instances (through
> > > > ActiveMQ ? any other idea ?)
[big snip]
> > > From a low level I was wondering about a Camel factory bean in spring
> > >
> > > <camel:heartbeat bean="someBeanRef" property="myStatusEventObject"
> > > millis="2000" uri="http://someServer.com/cheese"/>
> > >
> > > This would call someBeanRef.getMyStatusEventObject() every 2 seconds
> > > and send the POJO to the endpoint URI. We could maybe wrap up this
> > > using some route? e.g. if the bean endpoint allowed for polling we
> > > could maybe do
> > >
> > > from("bean:someBeanRef?method=getMyStatusEventObject&period=2000").to("http://someServer.com/cheese")
> > >
> > > though maybe a factory bean (XML element) is a bit easier - it does
> > > have better smart completion at least :)
> > >
> >
> > I like the idea of using camel to configure the discovery mechanism.
> > This would be very flexible.
>
> Yeah; am thinking we could do content based routing or message
> translator stuff to format the POJO of the heartbeat data into
> different formats, sending it to different locations etc
As an experiment in implementing a POJO based heartbeat mechanism
thats protocol and data format independent (and can support the easy
integration of EIP) I thought I'd try write a little test case in
Camel.
It turned out to be surprisingly easy :)
from("bean:myService?methodName=status").to("mock:result");
where the status() method is invoked periodically on the myService
bean and then sent to the registry endpoint (a mock in this case as
its a test case, but this could be activemq:topic:foo.bar etc)
Here's the test case if you're interested...
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/BeanMethodHeartbeatTest.java
--
James
-------
http://macstrac.blogspot.com/
Open Source SOA
http://open.iona.com
Re: ServiceMix 4 as a distributed OSGi spring container....
Posted by James Strachan <ja...@gmail.com>.
On 11/10/2007, Guillaume Nodet <gn...@gmail.com> wrote:
> Bringing this thread to dev list
>
> On 10/11/07, James Strachan <ja...@gmail.com> wrote:
> > > We need to find a way to discover other ServiceMix instances (through
> > > ActiveMQ ? any other idea ?)
> >
> > Am pondering if we
> > should have a simple POJO based discovery/heartbeat mechanism thats
> > transport agnostic so we could use multicast, jmdns, ActiveMQ, HTTP
> > etc. Was pondering about making something really simple in Camel/SMX4
> > and we take it from there etc.
> >
> > e.g. for a heartbeat we really just wanna do a timed send of a status
> > message to some endpoint. This could be JMS or a POST to some URI (or
> > even an AtomPub entry we POST).
> >
> > Then the server side of this thing, keeps a RAM cache of all these
> > things and times the entires out after a while so they disappear.
> > (Doesn't need any persistence or replication really; as if the server
> > side fails, another one can start somewhere and after a few heartbeats
> > its back in sync again).
> >
> > Then the server side - lets call it a registry (but a simple
> > registry); can just expose these heartbeats as data in different
> > queries. So folks can query using some mechanism for nodes of a
> > certain type or whatnot.
>
> Let's rephrase to check there is no misunderstanding.
> The purpose here is to have a discovery mechanism which is somewhat
> independant of the protocol. So there is one server and multiple
> clients, each client sending its own information to the server that
> keeps track of them.
> This implies, that either
> * there is a single point of failure for the master which is
> statically configured
> * the client elect a master amongst themselves
Yeah.
I guess the two common approaches would be;
* use a web server as the server via a URL (which you can have some
kinda DNS/IP load balancer to deal with the master failing - though we
really don't want to load balance across the master and slaves; all
traffic should go to the master server at all times). On failover from
master to slave it would just look like all the services died and came
back again shortly afterwards.
* use a JMS topic - so the 'server' is in RAM in each client (e.g. a
List of the non-expired heartbeat POJOs)
The latter is probably much easier to do; the former has the benefit
of being able to be RESTful and so work across languages and corporate
boundaries etc You could always add a RESTful facade ontop of the
latter too - which would support load balancing and failover :)
> Once the master / registry has been identified, the client send
> heatbeats to it and can query it for services. I'm not sure if a push
> or pull model is better here: having the master send new information
> to the clients, or have the clients polling for new informations or
> simply looking up the service on demand.
Yeah. I can see both being useful in different circumstances
> The problem with the pull approach is that we won't be able to expose
> remote services in the OSGi registry without hacking the registry
> itself. For this particular use case, it would be better to be
> notified of new services (or subscribing somehow) so that a client can
> register a new remote service in the OSGi registry, which would be
> available through a proxy.
Yeah
> >
> > This heartbeat/discovery thing is pretty generic stuff really; it'd be
> > nice if the message/data could be anything at all then we can reuse
> > the same thing in many circumstances. e.g. we could use this to
> > discover AMQ brokers; or CXF services of a kind of WSDL or SMX
> > instances or OSGi containers or whatnot.
>
> +1
>
> >
> > The blob of data that is heartbeated; and what kinds of queries on the
> > data we need to make is kinda speciifc to different problems. e.g. in
> > AMQ we mostly just need to get the URL to connect to the broker. We
> > may want to include some kinda network-hop type info so we try to
> > connect to the nearest node or something - but generally a URL is
> > enough in AMQ-land for discovery.
> >
> > For SMX containers, we might as well just expose the URL to talk to it.
> >
> > For distributed OSGi stuff we might wanna use more complex data for
> > the heartbeat data; so folks could search (rather like the NMR does)
> > by service QName, interface name, endpoint name or whatever.
> >
> >
> > From a low level I was wondering about a Camel factory bean in spring
> >
> > <camel:heartbeat bean="someBeanRef" property="myStatusEventObject"
> > millis="2000" uri="http://someServer.com/cheese"/>
> >
> > This would call someBeanRef.getMyStatusEventObject() every 2 seconds
> > and send the POJO to the endpoint URI. We could maybe wrap up this
> > using some route? e.g. if the bean endpoint allowed for polling we
> > could maybe do
> >
> > from("bean:someBeanRef?method=getMyStatusEventObject&period=2000").to("http://someServer.com/cheese")
> >
> > though maybe a factory bean (XML element) is a bit easier - it does
> > have better smart completion at least :)
> >
>
> I like the idea of using camel to configure the discovery mechanism.
> This would be very flexible.
Yeah; am thinking we could do content based routing or message
translator stuff to format the POJO of the heartbeat data into
different formats, sending it to different locations etc
> > BTW since servicemix is using activemq; probably the easiest discovery
> > we should use out of th ebox is using the JMS network - i.e. just to
> > use a topic for the heartbeats; then each node can have a local
> > in-memory 'registry'. Another option is to have a web app which is the
> > registry and clients POST heartbeats and then GET data back - which is
> > more useful for folks with firewalls or multi-language clients and so
> > forth. I guess those are the big 2 implementations; web based or JMS
> > based.
> >
>
> Sounds good. But these are very different topologies, the first one
> has no centralized registry, whereas the second one has. How can we
> bridge both approaches transparently ?
I guess the main thing to figure out is; how would folks query this
registry of beans? e.g. if we used some query language (EL, OGNL,
Groovy/Ruby, maybe even XPath/XQuery on POJOs) then we could add that
query to a GET URI for the REST/web based version - or just apply the
query to in RAM POJOs in a collection.
I guess from the Java client perspective we probably just need some
simple API to be notified of things appearing or disappearing matching
some filter; so we can respond (e.g. add/remove connectors in an
ActiveMQ client or broker - or add/remove service proxies in OSGi or
whatever the use case is)
Then how that API is implemented (watching in memory POJOs populated
from a JMS topic or polling a RESTful service etc) shouldn't matter to
the developer.
> > > and thus make the OSGi registries
> > > distributed. Services that are marked somehow would be exported as a
> > > service available remotely (I don't think it would be a good idea to
> > > make all services available remotely).
> >
> > Yeah. We could require "export=true" or something in the service
> > metadata or something. Maybe allow custom predicates on the service
> > metadata to define profiles of things to distribute.
> >
>
> Sounds good.
>
> >
> > > As for JAXWS, we need some strategies to deploy them (maybe we could
> > > rely on spring), as I'm not sure auto-discovering the beans it always
> > > a good idea (might be pluggable though).
> >
> > Yeah; am thinking if folks use spring to deploy them we could maybe
> > hook in to, say, CXF to grok the WSDL bits and expose those on the NMR
> > automagically if you see what I mean.
> >
>
> Yeah.
>
> > What would be awesome is if we can deploy various things (JAX-WS, EJB3
> > and so forth) using Spring+OSGi and we have a way to auto-discover
> > things and expose them on the OSGi NMR type stuff (and so in JBI too).
> >
> > i.e. rather than folks having to put their JAX-WS / EJB3 / SCA stuff
> > into a service engine; it'd be nice if we had a way of kinda binding
> > to them direclty via OSGi. Maybe wishful thinking though - I'm just
> > wondering if we can kinda make the OSGiNMR/JBI stuff invisible and for
> > it to auto-hook into whatever folks are actually using. Even if we can
> > just find a way of getting CXF services when deployed as OSGi bundles
> > to be discoverable so we can export 'em on the OSGi NMR thingy it'd be
> > a big win for CXF users.
>
> Agreed. What I was thinking was along those lines. That's the main
> reason why the ServiceMix api does not have any notion of "component"
> as in JBI. Instead we would create OSGi deployers (baically a bundle
> listener or service listener). For example to activate a ServiceMix
> endpoint, it's just a matter of registering it in the OSGi registry.
> For JAX-WS, we could add a spring extension for deploying a bean,
> kinda like the jaxws namespace for cxf
> (http://cwiki.apache.org/CXF20DOC/jax-ws-configuration.html).
>
> So exposing a bean in that way would perform all the necessary steps
> to register on as a JBI endpoint and make it available remotely (maybe
> we need a boolean to say if we want to export it remotely or not)
Great!
--
James
-------
http://macstrac.blogspot.com/
Open Source SOA
http://open.iona.com