You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@servicemix.apache.org by Nodet Guillaume <gn...@gmail.com> on 2007/08/23 10:53:13 UTC
ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Hi Kit,
I'm quite sure you would have a very valuable input there, given your
experience
on ServiceMix. So I'm starting this new thread. Would you mind
throwing a few
ideas there ?
Cheers,
Guillaume Nodet
On Aug 23, 2007, at 5:39 AM, Kit Plummer wrote:
> On 8/22/07, Terry Cox <te...@meta-concepts.com> wrote:
>>
>> Interesting.
>>
>> We need to have a very serious chat about application lifecycles and
>> governance...
>>
>> Terry
>>
>
>
> And Federating...distribution of the NMR across n-platforms!
>
> --
> Kit Plummer
> Nobody-in-Charge @ Black:Hole:Logic
> http://www.blackholelogic.com
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by Nodet Guillaume <gn...@gmail.com>.
On Aug 24, 2007, at 7:28 PM, Kit Plummer wrote:
> On 8/24/07, Nodet Guillaume <gn...@gmail.com> wrote:
>>
>> So if i understand you correctly, you are mostly concerned of
>> enhancing
>> the JMS flow in the following areas:
>> * avoid ping/pong and lower bandwidth requirement
>> (avoid sending the whole exchange and only send the actual
>> data)
>> * enhance security (authentication, encryption ?)
>> * enhance the endoint registry wrt to services or instances going
>> up and down
>
>
>
> Did I catch you correcty ?
>
>
> Yes. The registry piece is a little interesting for us to.
> Because we
> are integrating "hardware" via service components we've had to
> abstract the
> notion of capabilities. It would be nice if the registry were
> extensible.
If we leverage the OSGi registry, you can associate a bunch of metadata
to each service registered in the registry (in our cases, endpoints).
Cheers,
Guillaume Nodet
>
> For the bandwidh requirement, I'm sure we can do that and reconstruct
>> a fake
>> exchange on the other side. We would loose informations wrt to the
>> input message
>> but I don't think this would be too much a problem. For the ping/
>> pong stuff, I'm sure
>> we can find a way to optimize it somehow. We may have troubles
>> handling the
>> InOptionalOut pattern though, as you don't really know if you will
>> receive an Out
>> message or nothing, but for the other simple patterns (InOnly and
>> InOut) it should
>> be quite easy to send back the exchange with a DONE status just after
>> having send
>> the jms message containing the In or Out.
>
>
> I think as long as everything is optional wrt to optimizations the
> fact that
> it may exclude various patterns is acceptable. We've done
> everything in an
> InOnly world.
>
>
> On the security subject, I know there is lots to do, but this is an
>> area i'm not so familiar
>> with. My biggest concern is that we security can be applied at the
>> connection level
>> or at the message level. NMR-NMR security for the JMS flow could be
>> delegated
>> to ActiveMQ i guess (using AMQ security features).
>
>
> Agreed. It is now...the Network of Brokers feature. But, there is
> not
> really any concept of policy.
>
> On the registry side, I think one of the main problem is that there
>> is no way to tell the
>> difference between an endpoint that goes down because the server is
>> no more
>> accessibe (it will be up again at a later time) or because the
>> endpoint has been
>> undeployed. Imho, this is a key requirement to be able to make
>> routing decisions.
>> I don't know yet how to handle this problem: if a server has been
>> shutdown, it may
>> never go up again...
>
>
> Yeh, like if it has been blown up by an IED...
>
>> So i'm still not sure how to handle the problem :-(
> Yep, you are right. Right now if we undeploy a service assembly, its
> endpoint still exists, and messages are still routed to it. Could
> just be a
> bug. ; }
>
> I'm sure more discussion on this will follow.
>
>
> Cheers,
>> Guillaume Nodet
>>
>> On Aug 23, 2007, at 3:35 PM, Kit Plummer wrote:
>>
>>> Sure Guillaume.
>>>
>>> Maybe the best thing to do is explain the concept...and what we've
>>> done to
>>> meet our requirements.
>>>
>>> It is actually quite simple. We needed to be able to connect two
>>> computers
>>> together via TCP/IP, and have a publisher on one system, the
>>> consumer on the
>>> other. Granted we've got lot's of both on each - but, the premise
>>> is that
>>> link between is transparent.
>>>
>>> Currently, we are using a feature of ActiveMQ called "Network of
>>> Brokers"
>>> (NoB) to create a mapping of destinations/endpoints.
>>>
>>> Where it gets really complicated is when we only want to allow a
>>> specific
>>> MEPs to cross the NoB connection. In this example, bandwidth is
>>> not a
>>> commodity and must be tightly constrained. We were tolerant of all
>>> the SEDA
>>> flow handshaking, but I believe it would be nice if InOnly MEPS
>>> really were
>>> just a single transmission (turning off levels of reliability/
>>> durability).
>>> Also, in our environment multicast isn't possible, and the networks
>>> are
>>> fairly ad-hoc...meaning not stable. Plus, we need to know about
>>> the state
>>> of the link.
>>>
>>> Service registration happens also in different configurations. For
>>> example,
>>> one topology we support is a hierarchical flow (master-slaves).
>>> Imagine a
>>> simple sensor net. There would be a single point at the top, where
>>> are data
>>> were to be aggregated. So, in this example the NoBs need to support
>>> "followers" only communicating with their "leader"...and the
>>> "leader" only
>>> communicating with its "leader". But, there might also be a need
>>> to have
>>> "shared" data that is available on all platforms in network
>>> (health, state,
>>> etc.). Ding lifecycle.
>>>
>>> I could keep going...but, am curious if anyone else looks at it
>>> this way.
>>> Obviously, the notion of simple performance scalability is one way
>>> to look
>>> at. There is a lot of capability in the NoB, but I think it falls
>>> a bit
>>> short. There are a few features that we'd like to see, that would
>>> help us
>>> federate better. BC/SE/SA-level authentication to the bus, as
>>> well as
>>> platform-to-platform, or NMR-to-NMR authentication would be very
>>> helpful.
>>> We've been looking at grid/cluster-like capabilities too - for
>>> example, if
>>> one platform is maxed out from a processing perspective, sending
>>> the SA and
>>> the message/task to another platform in network automatically.
>>>
>>> Thanks for taking the time to do this.
>>>
>>> On 8/23/07, Nodet Guillaume <gn...@gmail.com> wrote:
>>>>
>>>> Hi Kit,
>>>>
>>>> I'm quite sure you would have a very valuable input there, given
>>>> your
>>>> experience
>>>> on ServiceMix. So I'm starting this new thread. Would you mind
>>>> throwing a few
>>>> ideas there ?
>>>>
>>>> Cheers,
>>>> Guillaume Nodet
>>>>
>>>>
>>>> On Aug 23, 2007, at 5:39 AM, Kit Plummer wrote:
>>>>
>>>>> On 8/22/07, Terry Cox <te...@meta-concepts.com> wrote:
>>>>>>
>>>>>> Interesting.
>>>>>>
>>>>>> We need to have a very serious chat about application lifecycles
>>>>>> and
>>>>>> governance...
>>>>>>
>>>>>> Terry
>>>>>>
>>>>>
>>>>>
>>>>> And Federating...distribution of the NMR across n-platforms!
>>>>>
>>>>> --
>>>>> Kit Plummer
>>>>> Nobody-in-Charge @ Black:Hole:Logic
>>>>> http://www.blackholelogic.com
>>>>
>>>>
>>
>>
>
>
> --
> Kit Plummer
> Nobody-in-Charge @ Black:Hole:Logic
> http://www.blackholelogic.com
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by Kit Plummer <ki...@gmail.com>.
On 8/24/07, Nodet Guillaume <gn...@gmail.com> wrote:
>
> So if i understand you correctly, you are mostly concerned of enhancing
> the JMS flow in the following areas:
> * avoid ping/pong and lower bandwidth requirement
> (avoid sending the whole exchange and only send the actual data)
> * enhance security (authentication, encryption ?)
> * enhance the endoint registry wrt to services or instances going
> up and down
Did I catch you correcty ?
Yes. The registry piece is a little interesting for us to. Because we
are integrating "hardware" via service components we've had to abstract the
notion of capabilities. It would be nice if the registry were extensible.
For the bandwidh requirement, I'm sure we can do that and reconstruct
> a fake
> exchange on the other side. We would loose informations wrt to the
> input message
> but I don't think this would be too much a problem. For the ping/
> pong stuff, I'm sure
> we can find a way to optimize it somehow. We may have troubles
> handling the
> InOptionalOut pattern though, as you don't really know if you will
> receive an Out
> message or nothing, but for the other simple patterns (InOnly and
> InOut) it should
> be quite easy to send back the exchange with a DONE status just after
> having send
> the jms message containing the In or Out.
I think as long as everything is optional wrt to optimizations the fact that
it may exclude various patterns is acceptable. We've done everything in an
InOnly world.
On the security subject, I know there is lots to do, but this is an
> area i'm not so familiar
> with. My biggest concern is that we security can be applied at the
> connection level
> or at the message level. NMR-NMR security for the JMS flow could be
> delegated
> to ActiveMQ i guess (using AMQ security features).
Agreed. It is now...the Network of Brokers feature. But, there is not
really any concept of policy.
On the registry side, I think one of the main problem is that there
> is no way to tell the
> difference between an endpoint that goes down because the server is
> no more
> accessibe (it will be up again at a later time) or because the
> endpoint has been
> undeployed. Imho, this is a key requirement to be able to make
> routing decisions.
> I don't know yet how to handle this problem: if a server has been
> shutdown, it may
> never go up again...
Yeh, like if it has been blown up by an IED...
> So i'm still not sure how to handle the problem :-(
Yep, you are right. Right now if we undeploy a service assembly, its
endpoint still exists, and messages are still routed to it. Could just be a
bug. ; }
I'm sure more discussion on this will follow.
Cheers,
> Guillaume Nodet
>
> On Aug 23, 2007, at 3:35 PM, Kit Plummer wrote:
>
> > Sure Guillaume.
> >
> > Maybe the best thing to do is explain the concept...and what we've
> > done to
> > meet our requirements.
> >
> > It is actually quite simple. We needed to be able to connect two
> > computers
> > together via TCP/IP, and have a publisher on one system, the
> > consumer on the
> > other. Granted we've got lot's of both on each - but, the premise
> > is that
> > link between is transparent.
> >
> > Currently, we are using a feature of ActiveMQ called "Network of
> > Brokers"
> > (NoB) to create a mapping of destinations/endpoints.
> >
> > Where it gets really complicated is when we only want to allow a
> > specific
> > MEPs to cross the NoB connection. In this example, bandwidth is not a
> > commodity and must be tightly constrained. We were tolerant of all
> > the SEDA
> > flow handshaking, but I believe it would be nice if InOnly MEPS
> > really were
> > just a single transmission (turning off levels of reliability/
> > durability).
> > Also, in our environment multicast isn't possible, and the networks
> > are
> > fairly ad-hoc...meaning not stable. Plus, we need to know about
> > the state
> > of the link.
> >
> > Service registration happens also in different configurations. For
> > example,
> > one topology we support is a hierarchical flow (master-slaves).
> > Imagine a
> > simple sensor net. There would be a single point at the top, where
> > are data
> > were to be aggregated. So, in this example the NoBs need to support
> > "followers" only communicating with their "leader"...and the
> > "leader" only
> > communicating with its "leader". But, there might also be a need
> > to have
> > "shared" data that is available on all platforms in network
> > (health, state,
> > etc.). Ding lifecycle.
> >
> > I could keep going...but, am curious if anyone else looks at it
> > this way.
> > Obviously, the notion of simple performance scalability is one way
> > to look
> > at. There is a lot of capability in the NoB, but I think it falls
> > a bit
> > short. There are a few features that we'd like to see, that would
> > help us
> > federate better. BC/SE/SA-level authentication to the bus, as well as
> > platform-to-platform, or NMR-to-NMR authentication would be very
> > helpful.
> > We've been looking at grid/cluster-like capabilities too - for
> > example, if
> > one platform is maxed out from a processing perspective, sending
> > the SA and
> > the message/task to another platform in network automatically.
> >
> > Thanks for taking the time to do this.
> >
> > On 8/23/07, Nodet Guillaume <gn...@gmail.com> wrote:
> >>
> >> Hi Kit,
> >>
> >> I'm quite sure you would have a very valuable input there, given your
> >> experience
> >> on ServiceMix. So I'm starting this new thread. Would you mind
> >> throwing a few
> >> ideas there ?
> >>
> >> Cheers,
> >> Guillaume Nodet
> >>
> >>
> >> On Aug 23, 2007, at 5:39 AM, Kit Plummer wrote:
> >>
> >>> On 8/22/07, Terry Cox <te...@meta-concepts.com> wrote:
> >>>>
> >>>> Interesting.
> >>>>
> >>>> We need to have a very serious chat about application lifecycles
> >>>> and
> >>>> governance...
> >>>>
> >>>> Terry
> >>>>
> >>>
> >>>
> >>> And Federating...distribution of the NMR across n-platforms!
> >>>
> >>> --
> >>> Kit Plummer
> >>> Nobody-in-Charge @ Black:Hole:Logic
> >>> http://www.blackholelogic.com
> >>
> >>
>
>
--
Kit Plummer
Nobody-in-Charge @ Black:Hole:Logic
http://www.blackholelogic.com
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by Bruce Snyder <br...@gmail.com>.
On 8/24/07, Nodet Guillaume <gn...@gmail.com> wrote:
>
> On Aug 24, 2007, at 6:44 PM, Bruce Snyder wrote:
>
> > On 8/24/07, Nodet Guillaume <gn...@gmail.com> wrote:
> >>
> >> On the registry side, I think one of the main problem is that there
> >> is no way to tell the
> >> difference between an endpoint that goes down because the server is
> >> no more
> >> accessibe (it will be up again at a later time) or because the
> >> endpoint has been
> >> undeployed. Imho, this is a key requirement to be able to make
> >> routing decisions.
> >> I don't know yet how to handle this problem: if a server has been
> >> shutdown, it may
> >> never go up again... So i'm still not sure how to handle the
> >> problem :-(
> >
> > This is one place where a proper registry might be useful. If the
> > registry handles metadata for a given component, it can provide this
> > type of finer grained information to anyone that is interested. E.g.,
> > if a component is undeployed, the deployer tells the registry that
> > componentX is being undeployed and the registry would store that
> > information. When componentX comes back online it tells the registry
> > and the registry updates the metadata.
> >
>
> I agree, but it may not even be sufficient. Let's take the example
> of Kit.
> This environment is very dynamic and instances will not even say
> that the endpoints are undeployed. They are just gone and may never
> appear again. Another example: if you use a cluster of servers and you
> want to bring one down, will you really undeploy everything or just shut
> ServiceMix down ?
>
> So I'm not sure you can really know wether a service will come up at a
> later time. This requires much more knowledge that ServiceMix can
> have. I guess we could have a default behavior that would say when a
> server goes down if the endpoints should be considered temporarily
> unavailable or permanently lost.
Well, if it's gone, it's gone. That component cannot be used. I was
assuming that this status would be used to determine how to handle
messages destined for a component that is not online. E.g.:
a) componentX is marked as undeployed in the registry so it's gone and
messages should not be held for it
b) componentX is marked as stopped in the registry which means it will
be started up again at a later time, so hold messages for it until it
comes back online
This means that certain states assume certain behavior that might work
in a manner similar to a lifecycle. E.g., if a component is in the
process of being started up, messages destined for that component
should be held until the component is fully started.
> We currently don't have such a feature in the JMS/JCA flows, but we
> definitely need something around that...
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache ServiceMix - http://servicemix.org/
Apache Geronimo - http://geronimo.apache.org/
Castor - http://castor.org/
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by Nodet Guillaume <gn...@gmail.com>.
On Aug 24, 2007, at 6:44 PM, Bruce Snyder wrote:
> On 8/24/07, Nodet Guillaume <gn...@gmail.com> wrote:
>>
>> On the registry side, I think one of the main problem is that there
>> is no way to tell the
>> difference between an endpoint that goes down because the server is
>> no more
>> accessibe (it will be up again at a later time) or because the
>> endpoint has been
>> undeployed. Imho, this is a key requirement to be able to make
>> routing decisions.
>> I don't know yet how to handle this problem: if a server has been
>> shutdown, it may
>> never go up again... So i'm still not sure how to handle the
>> problem :-(
>
> This is one place where a proper registry might be useful. If the
> registry handles metadata for a given component, it can provide this
> type of finer grained information to anyone that is interested. E.g.,
> if a component is undeployed, the deployer tells the registry that
> componentX is being undeployed and the registry would store that
> information. When componentX comes back online it tells the registry
> and the registry updates the metadata.
>
I agree, but it may not even be sufficient. Let's take the example
of Kit.
This environment is very dynamic and instances will not even say
that the endpoints are undeployed. They are just gone and may never
appear again. Another example: if you use a cluster of servers and you
want to bring one down, will you really undeploy everything or just shut
ServiceMix down ?
So I'm not sure you can really know wether a service will come up at a
later time. This requires much more knowledge that ServiceMix can
have. I guess we could have a default behavior that would say when a
server goes down if the endpoints should be considered temporarily
unavailable or permanently lost.
We currently don't have such a feature in the JMS/JCA flows, but we
definitely need something around that...
Cheers,
Guillaume Nodet
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by Bruce Snyder <br...@gmail.com>.
On 8/24/07, Nodet Guillaume <gn...@gmail.com> wrote:
> So if i understand you correctly, you are mostly concerned of enhancing
> the JMS flow in the following areas:
> * avoid ping/pong and lower bandwidth requirement
> (avoid sending the whole exchange and only send the actual data)
> * enhance security (authentication, encryption ?)
> * enhance the endoint registry wrt to services or instances going
> up and down
>
>
> Did I catch you correcty ?
>
> For the bandwidh requirement, I'm sure we can do that and reconstruct
> a fake
> exchange on the other side. We would loose informations wrt to the
> input message
> but I don't think this would be too much a problem. For the ping/
> pong stuff, I'm sure
> we can find a way to optimize it somehow. We may have troubles
> handling the
> InOptionalOut pattern though, as you don't really know if you will
> receive an Out
> message or nothing, but for the other simple patterns (InOnly and
> InOut) it should
> be quite easy to send back the exchange with a DONE status just after
> having send
> the jms message containing the In or Out.
>
> On the security subject, I know there is lots to do, but this is an
> area i'm not so familiar
> with. My biggest concern is that we security can be applied at the
> connection level
> or at the message level. NMR-NMR security for the JMS flow could be
> delegated
> to ActiveMQ i guess (using AMQ security features).
An interceptor framework might help this situation where the necessary
authentication and authorization interceptors could be added to handle
the situation. Then it's just up to the implementation of each.
> On the registry side, I think one of the main problem is that there
> is no way to tell the
> difference between an endpoint that goes down because the server is
> no more
> accessibe (it will be up again at a later time) or because the
> endpoint has been
> undeployed. Imho, this is a key requirement to be able to make
> routing decisions.
> I don't know yet how to handle this problem: if a server has been
> shutdown, it may
> never go up again... So i'm still not sure how to handle the problem :-(
This is one place where a proper registry might be useful. If the
registry handles metadata for a given component, it can provide this
type of finer grained information to anyone that is interested. E.g.,
if a component is undeployed, the deployer tells the registry that
componentX is being undeployed and the registry would store that
information. When componentX comes back online it tells the registry
and the registry updates the metadata.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache ServiceMix - http://servicemix.org/
Apache Geronimo - http://geronimo.apache.org/
Castor - http://castor.org/
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by David Jencks <da...@yahoo.com>.
On Aug 24, 2007, at 3:03 AM, Nodet Guillaume wrote:
> So if i understand you correctly, you are mostly concerned of
> enhancing
> the JMS flow in the following areas:
> * avoid ping/pong and lower bandwidth requirement
> (avoid sending the whole exchange and only send the actual data)
> * enhance security (authentication, encryption ?)
> * enhance the endoint registry wrt to services or instances going
> up and down
>
>
> Did I catch you correcty ?
>
> For the bandwidh requirement, I'm sure we can do that and
> reconstruct a fake
> exchange on the other side. We would loose informations wrt to the
> input message
> but I don't think this would be too much a problem. For the ping/
> pong stuff, I'm sure
> we can find a way to optimize it somehow. We may have troubles
> handling the
> InOptionalOut pattern though, as you don't really know if you will
> receive an Out
> message or nothing, but for the other simple patterns (InOnly and
> InOut) it should
> be quite easy to send back the exchange with a DONE status just
> after having send
> the jms message containing the In or Out.
>
> On the security subject, I know there is lots to do, but this is an
> area i'm not so familiar
> with. My biggest concern is that we security can be applied at the
> connection level
> or at the message level. NMR-NMR security for the JMS flow could
> be delegated
> to ActiveMQ i guess (using AMQ security features).
You may want to look into the CORBA csiv2 framework for some ideas on
how to simultaneously deal with authentication of both the client/
server and user. I'm not suggesting that you use CORBA as a
transport but the concepts might be useful.
thanks
david jencks
>
> On the registry side, I think one of the main problem is that there
> is no way to tell the
> difference between an endpoint that goes down because the server is
> no more
> accessibe (it will be up again at a later time) or because the
> endpoint has been
> undeployed. Imho, this is a key requirement to be able to make
> routing decisions.
> I don't know yet how to handle this problem: if a server has been
> shutdown, it may
> never go up again... So i'm still not sure how to handle the
> problem :-(
>
> Cheers,
> Guillaume Nodet
>
> On Aug 23, 2007, at 3:35 PM, Kit Plummer wrote:
>
>> Sure Guillaume.
>>
>> Maybe the best thing to do is explain the concept...and what we've
>> done to
>> meet our requirements.
>>
>> It is actually quite simple. We needed to be able to connect two
>> computers
>> together via TCP/IP, and have a publisher on one system, the
>> consumer on the
>> other. Granted we've got lot's of both on each - but, the premise
>> is that
>> link between is transparent.
>>
>> Currently, we are using a feature of ActiveMQ called "Network of
>> Brokers"
>> (NoB) to create a mapping of destinations/endpoints.
>>
>> Where it gets really complicated is when we only want to allow a
>> specific
>> MEPs to cross the NoB connection. In this example, bandwidth is
>> not a
>> commodity and must be tightly constrained. We were tolerant of
>> all the SEDA
>> flow handshaking, but I believe it would be nice if InOnly MEPS
>> really were
>> just a single transmission (turning off levels of reliability/
>> durability).
>> Also, in our environment multicast isn't possible, and the
>> networks are
>> fairly ad-hoc...meaning not stable. Plus, we need to know about
>> the state
>> of the link.
>>
>> Service registration happens also in different configurations.
>> For example,
>> one topology we support is a hierarchical flow (master-slaves).
>> Imagine a
>> simple sensor net. There would be a single point at the top,
>> where are data
>> were to be aggregated. So, in this example the NoBs need to support
>> "followers" only communicating with their "leader"...and the
>> "leader" only
>> communicating with its "leader". But, there might also be a need
>> to have
>> "shared" data that is available on all platforms in network
>> (health, state,
>> etc.). Ding lifecycle.
>>
>> I could keep going...but, am curious if anyone else looks at it
>> this way.
>> Obviously, the notion of simple performance scalability is one way
>> to look
>> at. There is a lot of capability in the NoB, but I think it falls
>> a bit
>> short. There are a few features that we'd like to see, that would
>> help us
>> federate better. BC/SE/SA-level authentication to the bus, as
>> well as
>> platform-to-platform, or NMR-to-NMR authentication would be very
>> helpful.
>> We've been looking at grid/cluster-like capabilities too - for
>> example, if
>> one platform is maxed out from a processing perspective, sending
>> the SA and
>> the message/task to another platform in network automatically.
>>
>> Thanks for taking the time to do this.
>>
>> On 8/23/07, Nodet Guillaume <gn...@gmail.com> wrote:
>>>
>>> Hi Kit,
>>>
>>> I'm quite sure you would have a very valuable input there, given
>>> your
>>> experience
>>> on ServiceMix. So I'm starting this new thread. Would you mind
>>> throwing a few
>>> ideas there ?
>>>
>>> Cheers,
>>> Guillaume Nodet
>>>
>>>
>>> On Aug 23, 2007, at 5:39 AM, Kit Plummer wrote:
>>>
>>>> On 8/22/07, Terry Cox <te...@meta-concepts.com> wrote:
>>>>>
>>>>> Interesting.
>>>>>
>>>>> We need to have a very serious chat about application
>>>>> lifecycles and
>>>>> governance...
>>>>>
>>>>> Terry
>>>>>
>>>>
>>>>
>>>> And Federating...distribution of the NMR across n-platforms!
>>>>
>>>> --
>>>> Kit Plummer
>>>> Nobody-in-Charge @ Black:Hole:Logic
>>>> http://www.blackholelogic.com
>>>
>>>
>
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by Nodet Guillaume <gn...@gmail.com>.
So if i understand you correctly, you are mostly concerned of enhancing
the JMS flow in the following areas:
* avoid ping/pong and lower bandwidth requirement
(avoid sending the whole exchange and only send the actual data)
* enhance security (authentication, encryption ?)
* enhance the endoint registry wrt to services or instances going
up and down
Did I catch you correcty ?
For the bandwidh requirement, I'm sure we can do that and reconstruct
a fake
exchange on the other side. We would loose informations wrt to the
input message
but I don't think this would be too much a problem. For the ping/
pong stuff, I'm sure
we can find a way to optimize it somehow. We may have troubles
handling the
InOptionalOut pattern though, as you don't really know if you will
receive an Out
message or nothing, but for the other simple patterns (InOnly and
InOut) it should
be quite easy to send back the exchange with a DONE status just after
having send
the jms message containing the In or Out.
On the security subject, I know there is lots to do, but this is an
area i'm not so familiar
with. My biggest concern is that we security can be applied at the
connection level
or at the message level. NMR-NMR security for the JMS flow could be
delegated
to ActiveMQ i guess (using AMQ security features).
On the registry side, I think one of the main problem is that there
is no way to tell the
difference between an endpoint that goes down because the server is
no more
accessibe (it will be up again at a later time) or because the
endpoint has been
undeployed. Imho, this is a key requirement to be able to make
routing decisions.
I don't know yet how to handle this problem: if a server has been
shutdown, it may
never go up again... So i'm still not sure how to handle the problem :-(
Cheers,
Guillaume Nodet
On Aug 23, 2007, at 3:35 PM, Kit Plummer wrote:
> Sure Guillaume.
>
> Maybe the best thing to do is explain the concept...and what we've
> done to
> meet our requirements.
>
> It is actually quite simple. We needed to be able to connect two
> computers
> together via TCP/IP, and have a publisher on one system, the
> consumer on the
> other. Granted we've got lot's of both on each - but, the premise
> is that
> link between is transparent.
>
> Currently, we are using a feature of ActiveMQ called "Network of
> Brokers"
> (NoB) to create a mapping of destinations/endpoints.
>
> Where it gets really complicated is when we only want to allow a
> specific
> MEPs to cross the NoB connection. In this example, bandwidth is not a
> commodity and must be tightly constrained. We were tolerant of all
> the SEDA
> flow handshaking, but I believe it would be nice if InOnly MEPS
> really were
> just a single transmission (turning off levels of reliability/
> durability).
> Also, in our environment multicast isn't possible, and the networks
> are
> fairly ad-hoc...meaning not stable. Plus, we need to know about
> the state
> of the link.
>
> Service registration happens also in different configurations. For
> example,
> one topology we support is a hierarchical flow (master-slaves).
> Imagine a
> simple sensor net. There would be a single point at the top, where
> are data
> were to be aggregated. So, in this example the NoBs need to support
> "followers" only communicating with their "leader"...and the
> "leader" only
> communicating with its "leader". But, there might also be a need
> to have
> "shared" data that is available on all platforms in network
> (health, state,
> etc.). Ding lifecycle.
>
> I could keep going...but, am curious if anyone else looks at it
> this way.
> Obviously, the notion of simple performance scalability is one way
> to look
> at. There is a lot of capability in the NoB, but I think it falls
> a bit
> short. There are a few features that we'd like to see, that would
> help us
> federate better. BC/SE/SA-level authentication to the bus, as well as
> platform-to-platform, or NMR-to-NMR authentication would be very
> helpful.
> We've been looking at grid/cluster-like capabilities too - for
> example, if
> one platform is maxed out from a processing perspective, sending
> the SA and
> the message/task to another platform in network automatically.
>
> Thanks for taking the time to do this.
>
> On 8/23/07, Nodet Guillaume <gn...@gmail.com> wrote:
>>
>> Hi Kit,
>>
>> I'm quite sure you would have a very valuable input there, given your
>> experience
>> on ServiceMix. So I'm starting this new thread. Would you mind
>> throwing a few
>> ideas there ?
>>
>> Cheers,
>> Guillaume Nodet
>>
>>
>> On Aug 23, 2007, at 5:39 AM, Kit Plummer wrote:
>>
>>> On 8/22/07, Terry Cox <te...@meta-concepts.com> wrote:
>>>>
>>>> Interesting.
>>>>
>>>> We need to have a very serious chat about application lifecycles
>>>> and
>>>> governance...
>>>>
>>>> Terry
>>>>
>>>
>>>
>>> And Federating...distribution of the NMR across n-platforms!
>>>
>>> --
>>> Kit Plummer
>>> Nobody-in-Charge @ Black:Hole:Logic
>>> http://www.blackholelogic.com
>>
>>
Re: ServiceMix 4.0 and federation (was Re: ServiceMix 4.0)
Posted by Kit Plummer <ki...@gmail.com>.
Sure Guillaume.
Maybe the best thing to do is explain the concept...and what we've done to
meet our requirements.
It is actually quite simple. We needed to be able to connect two computers
together via TCP/IP, and have a publisher on one system, the consumer on the
other. Granted we've got lot's of both on each - but, the premise is that
link between is transparent.
Currently, we are using a feature of ActiveMQ called "Network of Brokers"
(NoB) to create a mapping of destinations/endpoints.
Where it gets really complicated is when we only want to allow a specific
MEPs to cross the NoB connection. In this example, bandwidth is not a
commodity and must be tightly constrained. We were tolerant of all the SEDA
flow handshaking, but I believe it would be nice if InOnly MEPS really were
just a single transmission (turning off levels of reliability/durability).
Also, in our environment multicast isn't possible, and the networks are
fairly ad-hoc...meaning not stable. Plus, we need to know about the state
of the link.
Service registration happens also in different configurations. For example,
one topology we support is a hierarchical flow (master-slaves). Imagine a
simple sensor net. There would be a single point at the top, where are data
were to be aggregated. So, in this example the NoBs need to support
"followers" only communicating with their "leader"...and the "leader" only
communicating with its "leader". But, there might also be a need to have
"shared" data that is available on all platforms in network (health, state,
etc.). Ding lifecycle.
I could keep going...but, am curious if anyone else looks at it this way.
Obviously, the notion of simple performance scalability is one way to look
at. There is a lot of capability in the NoB, but I think it falls a bit
short. There are a few features that we'd like to see, that would help us
federate better. BC/SE/SA-level authentication to the bus, as well as
platform-to-platform, or NMR-to-NMR authentication would be very helpful.
We've been looking at grid/cluster-like capabilities too - for example, if
one platform is maxed out from a processing perspective, sending the SA and
the message/task to another platform in network automatically.
Thanks for taking the time to do this.
On 8/23/07, Nodet Guillaume <gn...@gmail.com> wrote:
>
> Hi Kit,
>
> I'm quite sure you would have a very valuable input there, given your
> experience
> on ServiceMix. So I'm starting this new thread. Would you mind
> throwing a few
> ideas there ?
>
> Cheers,
> Guillaume Nodet
>
>
> On Aug 23, 2007, at 5:39 AM, Kit Plummer wrote:
>
> > On 8/22/07, Terry Cox <te...@meta-concepts.com> wrote:
> >>
> >> Interesting.
> >>
> >> We need to have a very serious chat about application lifecycles and
> >> governance...
> >>
> >> Terry
> >>
> >
> >
> > And Federating...distribution of the NMR across n-platforms!
> >
> > --
> > Kit Plummer
> > Nobody-in-Charge @ Black:Hole:Logic
> > http://www.blackholelogic.com
>
>