Long post - results of a discussion with Sanjiva, Paul, Ant

[Paul Fremantle <pz...@gmail.com>]

(LONG POST WARNING)

Folks

Sanjiva, Ant and I sat down F2F and had a long discussion about the Synapse
Architecture. Sanjiva finally explained the phases clearly and I think I buy
it, especially if we make some enhancements. This post is *my* summary. Ant
and Sanjiva - and everyone else - please feel free to argue, contradict etc.
[Or even agree :-)]

I know that partial F2F meetings of groups of committers can be tricky. The
three of us came to a lot of conclusions that the rest of the team may or
may not agree to. However, it was also very productive, and a lot of where
we got to was incremental on the previous two F2F discussions that have been
held, so please bear with me as I try to explain this proposed model.

1. Sanjiva persuaded me that the idea of multiple phases was valuable to
help order the rules. Having written a few simple rules it began to be
obvious that it might be hard to correctly order multiple rules simply by
having a single ordered rule list. I persuaded Sanjiva that hardcoding just
3 phases is too specific. Especially if we allow there to be different kinds
of rulesets. So we agreed to have *multiple* phases.

2. <phase order="a b c"/>

3. We didn't quite agree on directionality. It should be either the phases
or the individual rules. Either way has issues. I think I would favour a
scheme that supports having the same in/out if you want. The in/out is not
from Synapse point of view but from the target service POV.

So:

<synapse:inphase order="a b c"/>
<synapse:outphase order="a d c"/>

4. We also discussed using the current Axis2 xml to define phases, but it
does seem a bit wordy/verbose.

5. The a b c refer to rulesets (see next).

6. The other issue with rules is that there doesn't seem to be a single
model of selecting rules that will meet everyones needs. For example, simple
regular expressions on parts of the message (wsa:To, wsa:Action) will be
very fast. Xpaths are general but hard to write. etc. etc.
So we agreed it was a good idea to allow different styles of rules.

7. The general xml tag for this might be:
<synapse:ruleset name="a" type="type-of-rules">

Inside a ruleset are a set of elements that are specific to the rule style.
For example, in XPaths,
<xpath:rule condition="//sq:getQuote">
or
<regex:rule regex="http://fremantle.org/*" field="wsa:To">

Another really simple ruleset would be "all" which doesn't do any checking
but always implements those rules.

8. Because you can only optimise one type of condition matching or rule, it
makes sense to make a particular phase exactly one kind of ruleset. Because
you cannot optimise across phases it makes sense to allow each phase to be
of a different style.
Each phase has a given style of ruleset (e.g. xpath, bpel, regex, etc). So
the <inphase/outphase> point to an ordered list of specific rulesets.

9. For example, there may be an initial phase where we do very fast matching
on wsa:To, and then a less efficient phase where we do specific
content-based routing using XPaths.

10. Optimisation of rules happens within a ruleset and phase.
Its up to the type of ruleset to define its own processing order, but in
general encourage the ordering within a phase to be decided by the rule
style and not by lexical ordering. In fact its up to the type of ruleset how
it defines the processing at all. The structure is really simple - the
Synapse engine calls on or more rulesets (in order specified). The rulesets
then call one or more mediators as they please.

11. We think are three levels of person.
a) Synapse developers i.e. people who develop the core synapse engine use
SPIs to extend Synapse with new rule types and mediator types (a few)
b) Mediator developers. Use APIs to add new mediators to Synapse (plenty)
c) Administrators. Configure rulesets and mediators to do useful stuff.
(lots - we hope)

12. We did want to have some fixed things. In particular, we want the a
consistent model for creating mediators. So the mediator interface defined
by Synapse must be supported by all ruleset styles.

13. The mediator is stateless and threadsafe (i.e. a single instance can be
used to handle multiple requests concurrently).

14. The Mediator interface is an API.
public interface Mediator {
boolean mediate(SynapseContext sc, SOAPMessageContext smc);
}

Boolean true indicates continue processing the ruleset and phases.
Boolean false indicates stop processing the ruleset and phases.

We added SynapseContext so as to be able to make the Mediator
stateless/threadsafe (as Glen has asked for). It also gives the mediator a
pointer back into the current synapse engine.

15. We discussed the int return type. We came up with some options for what
different return options could be....
0-continue in this ruleset
1-stop this ruleset/phase continue next one
2-stop period

16. and then a few more even odder ones.
3-restart this phase
4-restart beginning

17. But we thought we'd start with the boolean for the moment unless we
proved with use cases that we needed more. I personally prefer the boolean.
I think you can do everything you need with that.

18. Rulesets appear in code as implementations of RuleEngine.
Each Ruleset is initialised by passing the XML to the
RuleEngine.init(OMElement).

Once a ruleset is initialised, you can process multiple concurrent messages
thru it.
Synapse has a table from ruleset type to RuleEngineImpls.

RuleEngine is an *SPI*.

interface RuleEngine {
String getRulesetType();
void init(OMElement); // gets passed the <synapse:ruleset> element.
void process(SynapseContext sc, SOAPMessageContext smc); // stateless per
invocation
}

So if, for example, you want a highly optimised flow through Synapse, you
can have a ruleset called "all" that always runs every mediator. The
AllRuleEngine.process gets called. It has a single mediator, which is
stateless and that gets called, and the result is incredible performance. If
you want more dynamic options then that works too.

19. A SynapseContext is created as a message comes into Synapse. It has a
local storage to allow per-message state. It also has callbacks into the
current Synapse runtime (whatever is running this synapse.xml). This is how
you can add a new message into the runtime (suppose for example I want to do
a Tee and send a copy of a message on, but I want the copy to be mediated by
Synapse too, so I want to inject it into the current Synapse runtime.)

20. Somehow the SynapseContext is updated with correct config for the
current mediator (tho we weren't convinced this is beautiful).

21. *API*
public interface SynapseContext {
void injectMessage(MessageContext mc, "in|out|fault", phase?)
public Object getProperty(String propName);
public void setProperty(String propname, Object value);
getCurrentMediatorConfiguration();
}

22. We want the Synapse engine to be able to get messages not just from
Axis2. In general there are plenty of aspects in an Axis2 MessageContext
that aren't relevant to Synapse. But we don't want to replicate Axis2's MC.
So instead, what we would like is a SOAPMessageContext. Basically this is
the SOAPEnvelope plus the getTo/setTo, etc APIs but not things like
isPaused, AxisService, etc.

23. So we agreed to *ask* the Axis2 team to refactor AxisMessageContext to
subset a SOAPMessageContext only (= addressing + SOAPEnvelope). We think
there are actually some benefits to this. It will just be an interface and
Axis2 MC can extend it.

To start with we can just define the interface in Axis2 as a subset of the
Axis2 MC and say axis2.context.MessageContext implements SOAPMessageContext.
So this is a very small addition to Axis2 which makes Synapse much cleaner.

We also agreed that the messageContext properties override the envelope. So
for example, if you do smc.setTo(address), then that should be considered
the correct address. If we have an XSLT mediator that changes the
<SOAP:Header><wsa:To> element, then its up to the mediator will update the
setTo() property in the SOAPMessageContext.

24. We want the Mediator model to be configurable. So inside any rule there
is always exactly one element called mediator. All mediators are called as
Axis2 services (i.e. thru an AxisEngine and MessageReceiver).

The mediator has a given type. The default types will include:
* deployed axis2 services,
* spring bean assemblies,
* simple classes implementing Mediator,
* e4x.

We also discussed bpel (both a simple stateless kind and full stateful). We
discussed the idea of some builtin mediators like send, fault-back, log,
etc.

25. So the mediator element looks like:
<synapse:mediator type="service|spring|bpel|e4x|class|builtin|....">
<mediatorns:blah> .. type specific stuff goes here .. </mediatorns:blah>
</synapse:mediator>

26. Whatever XML sits inside the mediator is completely up to the mediator
type. So in e4x it could be a CDATA section containing the script, in spring
it could be a beans xml. It might be a BPEL document. For axis2 services, it
could parameterise them.

27. There is an SPI, MediatorConfigurator, that a mediator-type creator can
use to parse the XML.

*SPI*
interface MediatorConfigurator {
MediatorConfiguration parse(OMElement);
}

29. Then the mediator writer can get at the configuration inside the
mediation using the *API*.
interface MediatorConfiguration {
public OMElement getMediatorDefinition();
}

30. The idea is that a given type of MediatorConfiguration will also expose
pre-parsed information. For example, the

Axis2ServiceMediatorConfiguration implements MediatorConfiguration {
public OMElement getMediatorDefinition();
public Parameter getParameter(String name);
}

31. So when the rules are parsed, the engine calls the
Axis2ServiceMediatorConfigurator with the whole <mediator> element, which
parses the results into the Axis2ServiceMediatorConfiguration object. Then
at mediate time, the Mediator can get at the parsed parameters and use them.

32. Here is the XML structure we came up with:

<synapse:synapse xmlns:synapse="synapse">

<synapse:inphase order="a b c"/>
<synapse:outphase order=d e f"/>

<synapse:ruleset name="a" type="xpath">
<xpath:rule condition="xpath-expr" direction="*|in|out">
<synapse:apply qos="encrypt|sign|rmd|rms"/>*
<synapse:mediator type="service|spring|bpel|e4x|class|uri|builtin|....">
<mediatorns:stuff> .. type specific stuff goes here .. </mediatorns:stuff>
</synapse:mediator>
<!--exactly one mediator per rule -->
</xpath:rule>
<xpath:rule/> <!-- more rules -->
</synapse:ruleset>
<synapse:ruleset name="b" type="regex"> <!-- more rulesets -->
....
</synapse:ruleset>
</synapse:synapse>


33. That's all folks.

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Sanjiva Weerawarana <sa...@opensource.lk>]

On Mon, 2005-11-07 at 10:18 +0000, Paul Fremantle wrote:
> 18. Rulesets appear in code as implementations of RuleEngine. 
> Each Ruleset is initialised by passing the XML to the
> RuleEngine.init(OMElement). 
> Once a ruleset is initialised, you can process multiple concurrent
> messages thru it. 
> Synapse has a table from ruleset type to RuleEngineImpls.
> 
> RuleEngine is an *SPI*.    
> 
> interface RuleEngine {
>     String getRulesetType();
>     void init(OMElement); // gets passed the <synapse:ruleset>
> element. 
>     void process(SynapseContext sc, SOAPMessageContext smc); //
> stateless per invocation
> }

BTW .. since we want to allow a mediator to indicate whether processing
should continue within a ruleset, we clearly need to allow the
RuleEngine to propagate that info out of itself. So I'd say that process
needs to return boolean (or int; depending on how that pans out) so that
Synapse can decide whether to go to the next phase and continue running
rules or catch fire and die.

Sanjiva.



---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-dev-help@ws.apache.org

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Aleksander Slominski <as...@cs.indiana.edu>]

Paul Fremantle wrote:

> Alek
>
> I believe that the async model will work. We aim to make onward calls 
> using Axis2's non-blocking API. Although there are some reasons why it 
> won't quite work perfectly today, we hope to make it so that we put 
> enough information in the "reply-to" header that in general Synapse 
> can neither hold state nor threads open between requests and replies.
>
> It will require Axis2 to change the way it handles invokeNonBlocking. I
>
> deally, even in the case where there is a synchronous HTTP connection 
> under the covers, what would happen is this. As soon as the invoke 
> happens, control should return to the client code. A thread from a 
> (pluggable) thread pool should pick up the request and send it out. 
> That thread should give up control while the socket waits for the 
> response HTTP message. When the response is ready, a thread from the 
> pool should pick it up and handle it. Of course we could optimise 
> thread switches. For example the thread that sends the request could 
> potentially wait a short amount of time to see if the response is 
> available.
>
> The same sort of model would be good on the inbound side. 
> Unfortunately when Axis2 is deployed as a servlet we can't do these 
> tricks with threads. However, I believe we can get the model right and 
> then lay it onto the thread model of servlets even if it isn't 
> optimal. Once we have a good thread model and async model in Axis2 I 
> think we can make Synapse perform very effectively - statelessly and 
> without ever blocking threads on IO.
>
> Now, when it comes to doing stateful BPEL like models, I think we need 
> to *explicitly* offer mediations the ability to store state away. We 
> could extend SynapseContext to give mediations access to a store - 
> either an in-memory store or a persistent store. However, I think this 
> should be something that a mediation writer has to explicitly choose.
>
> In general it is MUCH better if we can make all the state required to 
> mediate the message be available in the message, either in the body, 
> or using RefParams to pass it back and forth. Otherwise we will end up 
> with a lot of state management headaches.
>
> What do you think?

I agree that state management is a headache :) I think that embracing an 
asynchronous model where messages are processed through queues may 
simplify internal architecture of Synapse - simply pass messages from nd 
to different queue and (pooled) threads run actual mediators. This will 
also help with high throughput, state management (mediation that waits 
for a response message can have its state persisted and swapped out of 
memory), allow very high number of mediation running concurrently (as 
they do not need to be bound to threads), and will allow make easier to 
run Synapse in clustering setup. However how important is scalability 
beyond one JVM and clustering support?

best,

alek

>
> On 11/8/05, *Aleksander Slominski* <aslom@cs.indiana.edu 
> <ma...@cs.indiana.edu>> wrote:
>
>     Sanjiva Weerawarana wrote:
>
>     >On Tue, 2005-11-08 at 10:51 +0530, Vikas wrote:
>     >
>     >
>     >
>     >>* Where is the SynapseContext placed in connection to AXIS's context
>     >>hierarchy?
>     >>
>     >>
>     >
>     >I'll reply to more stuff over time but the idea was that
>     SynapseContext
>     >has nothing to do with the Axis2 contexts. Basically this is a thing
>     >that Synapse uses to keep any state it wants for a given message
>     being
>     >mediated thru it. So it serves as the mechanism to share data across
>     >mediators, for example.
>     >
>     >A key point is that we seem to be clarifying is the architectural
>     >relationship between Synapse and Axis2. Synapse *architecturally* is
>     >independent from Axis2: It gets messages delivered to it (in the
>     form of
>     >a SOAPMessageContext) and then it creates a SynapseContext for it and
>     >runs thru lots and lots of mediators and then sends the resulting
>     >SOAPMessageContext out to the final resting place (aka the
>     service being
>     >mediated for).
>     >
>     >
>     hi,
>
>     sounds very good though i have one architectural concern about Synapse
>     <-> Axis2 interactions: what if there are mediators that need to send
>     message to some other places and it takes time - will synapse require
>     axis2 to keep the connection? what happens if it times out waiting?
>     should all of of Synapse work in async mode?
>
>     >The same goes for BPEL .. now someone can write a BPEL to be the
>     ruleset
>     >and plug it in very cleanly!
>     >
>     >
>     the same concern but even more: what if BPEL process run inside
>     mediator
>     actually sends and *waits* for some messages? what if it waits really
>     long - will mediation and BPEL mediator be "persisted"?
>
>     that clearly is for the next version(s) of Synapse but i wonder
>     what is
>     the direction you see it going?
>
>     thanks,
>
>     alek
>
>
>     >Sanjiva.
>     >
>     >
>     >
>     >---------------------------------------------------------------------
>
>     >To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
>     <ma...@ws.apache.org>
>     >For additional commands, e-mail: synapse-dev-help@ws.apache.org
>     <ma...@ws.apache.org>
>     >
>     >
>     >
>
>
>     --
>     The best way to predict the future is to invent it - Alan Kay
>
>
>     ---------------------------------------------------------------------
>     To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
>     <ma...@ws.apache.org>
>     For additional commands, e-mail: synapse-dev-help@ws.apache.org
>     <ma...@ws.apache.org>
>
>


-- 
The best way to predict the future is to invent it - Alan Kay


---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-dev-help@ws.apache.org

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Paul Fremantle <pz...@gmail.com>]

Alek

I believe that the async model will work. We aim to make onward calls using
Axis2's non-blocking API. Although there are some reasons why it won't quite
work perfectly today, we hope to make it so that we put enough information
in the "reply-to" header that in general Synapse can neither hold state nor
threads open between requests and replies.

It will require Axis2 to change the way it handles invokeNonBlocking. I

deally, even in the case where there is a synchronous HTTP connection under
the covers, what would happen is this. As soon as the invoke happens,
control should return to the client code. A thread from a (pluggable) thread
pool should pick up the request and send it out. That thread should give up
control while the socket waits for the response HTTP message. When the
response is ready, a thread from the pool should pick it up and handle it.
Of course we could optimise thread switches. For example the thread that
sends the request could potentially wait a short amount of time to see if
the response is available.

The same sort of model would be good on the inbound side. Unfortunately when
Axis2 is deployed as a servlet we can't do these tricks with threads.
However, I believe we can get the model right and then lay it onto the
thread model of servlets even if it isn't optimal. Once we have a good
thread model and async model in Axis2 I think we can make Synapse perform
very effectively - statelessly and without ever blocking threads on IO.

Now, when it comes to doing stateful BPEL like models, I think we need to
*explicitly* offer mediations the ability to store state away. We could
extend SynapseContext to give mediations access to a store - either an
in-memory store or a persistent store. However, I think this should be
something that a mediation writer has to explicitly choose.

In general it is MUCH better if we can make all the state required to
mediate the message be available in the message, either in the body, or
using RefParams to pass it back and forth. Otherwise we will end up with a
lot of state management headaches.

What do you think?

Paul

On 11/8/05, Aleksander Slominski <as...@cs.indiana.edu> wrote:
>
> Sanjiva Weerawarana wrote:
>
> >On Tue, 2005-11-08 at 10:51 +0530, Vikas wrote:
> >
> >
> >
> >>* Where is the SynapseContext placed in connection to AXIS's context
> >>hierarchy?
> >>
> >>
> >
> >I'll reply to more stuff over time but the idea was that SynapseContext
> >has nothing to do with the Axis2 contexts. Basically this is a thing
> >that Synapse uses to keep any state it wants for a given message being
> >mediated thru it. So it serves as the mechanism to share data across
> >mediators, for example.
> >
> >A key point is that we seem to be clarifying is the architectural
> >relationship between Synapse and Axis2. Synapse *architecturally* is
> >independent from Axis2: It gets messages delivered to it (in the form of
> >a SOAPMessageContext) and then it creates a SynapseContext for it and
> >runs thru lots and lots of mediators and then sends the resulting
> >SOAPMessageContext out to the final resting place (aka the service being
> >mediated for).
> >
> >
> hi,
>
> sounds very good though i have one architectural concern about Synapse
> <-> Axis2 interactions: what if there are mediators that need to send
> message to some other places and it takes time - will synapse require
> axis2 to keep the connection? what happens if it times out waiting?
> should all of of Synapse work in async mode?
>
> >The same goes for BPEL .. now someone can write a BPEL to be the ruleset
> >and plug it in very cleanly!
> >
> >
> the same concern but even more: what if BPEL process run inside mediator
> actually sends and *waits* for some messages? what if it waits really
> long - will mediation and BPEL mediator be "persisted"?
>
> that clearly is for the next version(s) of Synapse but i wonder what is
> the direction you see it going?
>
> thanks,
>
> alek
>
>
> >Sanjiva.
> >
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
> >For additional commands, e-mail: synapse-dev-help@ws.apache.org
> >
> >
> >
>
>
> --
> The best way to predict the future is to invent it - Alan Kay
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: synapse-dev-help@ws.apache.org
>
>

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Aleksander Slominski <as...@cs.indiana.edu>]

Sanjiva Weerawarana wrote:

>On Tue, 2005-11-08 at 10:51 +0530, Vikas wrote:
> 
>  
>
>>* Where is the SynapseContext placed in connection to AXIS's context
>>hierarchy?
>>    
>>
>
>I'll reply to more stuff over time but the idea was that SynapseContext
>has nothing to do with the Axis2 contexts. Basically this is a thing
>that Synapse uses to keep any state it wants for a given message being
>mediated thru it. So it serves as the mechanism to share data across
>mediators, for example.
>
>A key point is that we seem to be clarifying is the architectural
>relationship between Synapse and Axis2. Synapse *architecturally* is
>independent from Axis2: It gets messages delivered to it (in the form of
>a SOAPMessageContext) and then it creates a SynapseContext for it and
>runs thru lots and lots of mediators and then sends the resulting
>SOAPMessageContext out to the final resting place (aka the service being
>mediated for). 
>  
>
hi,

sounds very good though i have one architectural concern about Synapse 
<-> Axis2 interactions: what if there are mediators that need to send 
message to some other places and it takes time - will synapse require 
axis2 to keep the connection? what happens if it times out waiting? 
should all of of Synapse work in async mode?

>The same goes for BPEL .. now someone can write a BPEL to be the ruleset
>and plug it in very cleanly!
>  
>
the same concern but even more: what if BPEL process run inside mediator 
actually sends and *waits* for some messages? what if it waits really 
long - will mediation and BPEL mediator be "persisted"?

that clearly is for the next version(s) of Synapse but i wonder what is 
the direction you see it going?

thanks,

alek


>Sanjiva.
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
>For additional commands, e-mail: synapse-dev-help@ws.apache.org
>
>  
>


-- 
The best way to predict the future is to invent it - Alan Kay


---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-dev-help@ws.apache.org

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Sanjiva Weerawarana <sa...@opensource.lk>]

On Tue, 2005-11-08 at 10:51 +0530, Vikas wrote:
 
> * Where is the SynapseContext placed in connection to AXIS's context
> hierarchy?

I'll reply to more stuff over time but the idea was that SynapseContext
has nothing to do with the Axis2 contexts. Basically this is a thing
that Synapse uses to keep any state it wants for a given message being
mediated thru it. So it serves as the mechanism to share data across
mediators, for example.

A key point is that we seem to be clarifying is the architectural
relationship between Synapse and Axis2. Synapse *architecturally* is
independent from Axis2: It gets messages delivered to it (in the form of
a SOAPMessageContext) and then it creates a SynapseContext for it and
runs thru lots and lots of mediators and then sends the resulting
SOAPMessageContext out to the final resting place (aka the service being
mediated for). 

Now, to *implement* that architecture on Axis2, messages that come to
Synapse are Axis2 message contexts which come via Axis2 listeners etc.
(it simply makes no sense to replicate all that stuff), plus mediators
themselves are implemented as Axis2 services so that we get to turn on
QoS (RM, TX, Sec etc.) for mediators without any hassle. So in effect
its not that Synapse is independent of Axis2, its just that a Synapse
mediation programmers (of whom we expect lots as Paul wrote) and Synapse
rule writers (of whom we expect lots too), don't have to know a whole
lot more than what a SOAPMessageContext is (== SOAPMessage + addressing
stuff) as far as Axis2 knowledge goes. Clearly that info is fundamental:
if you want to mediate SOAP messages you better understand that stuff.

> * How about a few parts on Policy? [can wait for now]

Did you mean QoS (as discussed above) or did you mean WS-Policy
Attachment as a way to write the mediations? 

If its the latter, the beauty of the ruleset approach and the "type" of
ruleset concept is that we can write a ruleset implementation which
supports WS-Policy Attachment and life would be awesome. (And of course
everyone in the world, esp. Glen, will be using only that way to
configure Synapse ;-))

The same goes for BPEL .. now someone can write a BPEL to be the ruleset
and plug it in very cleanly!

Sanjiva.



---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-dev-help@ws.apache.org

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[ant elder <an...@gmail.com>]

What do people thing about having the Synapse and its config XML provide
lots of default behaviour. So for example:

1) if you don't need or want the overhead of working out if a message is a
request or response you can have both the inphase and outphase specify the
same ruleset order. Synapse then doesn't do the computation to work out if a
message is a request or response.

2) if you don't care about phases then you can completely leave out inPhase
and outPhase and Synapse will use a single implicit 'all' phase which
includes all the rulesets in the order they are defined in the XML.

3) if you don't care about phases or rulesets then you can leave out both
the phase and ruleSet definitions and Synapse will use an implicit 'all'
phase as in (2) and implicit rulesets grouping each list of rules of the
same type, in the order they are defined in the XML.

So for then you could have this simplified XML:

<synapse xmlns="synapse" xmlns:xpath="synapse.xpath">

<xpath:rule condition="*" mediator="sqTransform.js"/>
<xpath:rule condition="*" mediator="sender"/>

</synapse>


Which is equivilent to this fully specified XML:

<synapse xmlns="synapse" xmlns:xpath="synapse.xpath">

<inphases order="a b"/>
<outphases order="a b"/>

<ruleset phase="a" style="xpath">
<xpath:rule condition="*">
<mediator type="e4xFile">
<filename>sqTransform.js</filename>
</mediator>
</xpath:rule>
</ruleset>

<ruleset phase="b" style="xpath">
<xpath:rule condition="*">
<mediator type="axis2.service">
<servicename>sender</servicename>
</mediator>
</xpath:rule>
</ruleset>

</synapse>

...ant

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Anantharam S <an...@gmail.com>]

Paul,
I couldn't help noticing these lines in the mail...

> * The code in the rule engine decides wether the rules are revaluated on a
> return from the mediator or if the processing is sequential.
>

Not at first - we reserved that as a second option.
 Can you please explain what the first option is?
* Are there plans of passing the rule engine's 'knowledge' to someother
point where this decision can be made?
 Thanks
Ananth


 On 11/8/05, Paul Fremantle <pz...@gmail.com> wrote:
>
> Vikas
>
> Notes inline
>
> On 11/8/05, Vikas <vi...@infravio.com> wrote:
> >
> > * The ruleEngines would be initialized with the relevant ruleSet
> >
>
> Yes absolutely
>
>  * The code in the rule engine decides wether the rules are revaluated on
> > a return from the mediator or if the processing is sequential.
> >
>
> Not at first - we reserved that as a second option.
>
> > In boolean mediate(SynapseContext sc, SOAPMessageContext smc);
> > Boolean true indicates continue processing the ruleset and phases.
> > Boolean false indicates stop processing the ruleset and phases.
> >
>
> This was our preferred option
>
>  0-continue in this ruleset
> > 1-stop this ruleset/phase continue next one
> > 2-stop period
> >
>
> This was an alternative we agreed to use if the boolean approach above
> doesn't work. We are still on the "inject" model - if I want the rules
> re-evaluated I use the SynapseContext.injectMessage(message).
>
>  * Where is the SynapseContext placed in connection to AXIS's context
> > hierarchy?
> >
>
> Good question. I think we need to code some before we decide this.
>
>  * How about a few parts on Policy? [can wait for now]
> >
>
> Yes we have some markers in the XML for applying aspects of QoS. I didn't
> describe them, but they fit under the <rule> element. They specify which
> aspects to turn on/off. I agree that this can wait for now.
>
>  ~Good day.
> >
> >   ----- Original Message -----
> >
> > *From:* Paul Fremantle <pz...@gmail.com>
> > *To:* synapse-dev@ws.apache.org
> > *Sent:* Monday, November 07, 2005 3:48 PM
> > *Subject:* Long post - results of a discussion with Sanjiva, Paul, Ant
> >
> > (LONG POST WARNING)
> >
> > Folks
> >
> > Sanjiva, Ant and I sat down F2F and had a long discussion about the
> > Synapse Architecture. Sanjiva finally explained the phases clearly and I
> > think I buy it, especially if we make some enhancements. This post is *my*
> > summary. Ant and Sanjiva - and everyone else - please feel free to argue,
> > contradict etc. [Or even agree :-)]
> >
> > I know that partial F2F meetings of groups of committers can be tricky.
> > The three of us came to a lot of conclusions that the rest of the team may
> > or may not agree to. However, it was also very productive, and a lot of
> > where we got to was incremental on the previous two F2F discussions that
> > have been held, so please bear with me as I try to explain this proposed
> > model.
> >
> > 1. Sanjiva persuaded me that the idea of multiple phases was valuable to
> > help order the rules. Having written a few simple rules it began to be
> > obvious that it might be hard to correctly order multiple rules simply by
> > having a single ordered rule list. I persuaded Sanjiva that hardcoding just
> > 3 phases is too specific. Especially if we allow there to be different kinds
> > of rulesets. So we agreed to have *multiple* phases.
> >
> > 2. <phase order="a b c"/>
> >
> > 3. We didn't quite agree on directionality. It should be either the
> > phases or the individual rules. Either way has issues. I think I would
> > favour a scheme that supports having the same in/out if you want. The in/out
> > is not from Synapse point of view but from the target service POV.
> >
> > So:
> >
> > <synapse:inphase order="a b c"/>
> > <synapse:outphase order="a d c"/>
> >
> > 4. We also discussed using the current Axis2 xml to define phases, but
> > it does seem a bit wordy/verbose.
> >
> > 5. The a b c refer to rulesets (see next).
> >
> > 6. The other issue with rules is that there doesn't seem to be a single
> > model of selecting rules that will meet everyones needs. For example, simple
> > regular expressions on parts of the message (wsa:To, wsa:Action) will be
> > very fast. Xpaths are general but hard to write. etc. etc.
> > So we agreed it was a good idea to allow different styles of rules.
> >
> > 7. The general xml tag for this might be:
> > <synapse:ruleset name="a" type="type-of-rules">
> >
> > Inside a ruleset are a set of elements that are specific to the rule
> > style.
> > For example, in XPaths,
> > <xpath:rule condition="//sq:getQuote">
> > or
> > <regex:rule regex="http://fremantle.org/*" field="wsa:To">
> >
> > Another really simple ruleset would be "all" which doesn't do any
> > checking but always implements those rules.
> >
> > 8. Because you can only optimise one type of condition matching or rule,
> > it makes sense to make a particular phase exactly one kind of ruleset.
> > Because you cannot optimise across phases it makes sense to allow each phase
> > to be of a different style.
> > Each phase has a given style of ruleset (e.g. xpath, bpel, regex, etc).
> > So the <inphase/outphase> point to an ordered list of specific rulesets.
> >
> > 9. For example, there may be an initial phase where we do very fast
> > matching on wsa:To, and then a less efficient phase where we do specific
> > content-based routing using XPaths.
> >
> > 10. Optimisation of rules happens within a ruleset and phase.
> > Its up to the type of ruleset to define its own processing order, but in
> > general encourage the ordering within a phase to be decided by the rule
> > style and not by lexical ordering. In fact its up to the type of ruleset how
> > it defines the processing at all. The structure is really simple - the
> > Synapse engine calls on or more rulesets (in order specified). The rulesets
> > then call one or more mediators as they please.
> >
> > 11. We think are three levels of person.
> > a) Synapse developers i.e. people who develop the core synapse engine
> > use SPIs to extend Synapse with new rule types and mediator types (a few)
> > b) Mediator developers. Use APIs to add new mediators to Synapse
> > (plenty)
> > c) Administrators. Configure rulesets and mediators to do useful stuff.
> > (lots - we hope)
> >
> > 12. We did want to have some fixed things. In particular, we want the a
> > consistent model for creating mediators. So the mediator interface defined
> > by Synapse must be supported by all ruleset styles.
> >
> > 13. The mediator is stateless and threadsafe (i.e. a single instance can
> > be used to handle multiple requests concurrently).
> >
> > 14. The Mediator interface is an API.
> > public interface Mediator {
> > boolean mediate(SynapseContext sc, SOAPMessageContext smc);
> > }
> >
> > Boolean true indicates continue processing the ruleset and phases.
> > Boolean false indicates stop processing the ruleset and phases.
> >
> > We added SynapseContext so as to be able to make the Mediator
> > stateless/threadsafe (as Glen has asked for). It also gives the mediator a
> > pointer back into the current synapse engine.
> >
> > 15. We discussed the int return type. We came up with some options for
> > what different return options could be....
> > 0-continue in this ruleset
> > 1-stop this ruleset/phase continue next one
> > 2-stop period
> >
> > 16. and then a few more even odder ones.
> > 3-restart this phase
> > 4-restart beginning
> >
> > 17. But we thought we'd start with the boolean for the moment unless we
> > proved with use cases that we needed more. I personally prefer the boolean.
> > I think you can do everything you need with that.
> >
> > 18. Rulesets appear in code as implementations of RuleEngine.
> > Each Ruleset is initialised by passing the XML to the RuleEngine.init(OMElement).
> >
> > Once a ruleset is initialised, you can process multiple concurrent
> > messages thru it.
> > Synapse has a table from ruleset type to RuleEngineImpls.
> >
> > RuleEngine is an *SPI*.
> >
> > interface RuleEngine {
> > String getRulesetType();
> > void init(OMElement); // gets passed the <synapse:ruleset> element.
> > void process(SynapseContext sc, SOAPMessageContext smc); // stateless
> > per invocation
> > }
> >
> > So if, for example, you want a highly optimised flow through Synapse,
> > you can have a ruleset called "all" that always runs every mediator. The
> > AllRuleEngine.process gets called. It has a single mediator, which is
> > stateless and that gets called, and the result is incredible performance. If
> > you want more dynamic options then that works too.
> >
> > 19. A SynapseContext is created as a message comes into Synapse. It has
> > a local storage to allow per-message state. It also has callbacks into the
> > current Synapse runtime (whatever is running this synapse.xml). This is
> > how you can add a new message into the runtime (suppose for example I want
> > to do a Tee and send a copy of a message on, but I want the copy to be
> > mediated by Synapse too, so I want to inject it into the current Synapse
> > runtime.)
> >
> > 20. Somehow the SynapseContext is updated with correct config for the
> > current mediator (tho we weren't convinced this is beautiful).
> >
> > 21. *API*
> > public interface SynapseContext {
> > void injectMessage(MessageContext mc, "in|out|fault", phase?)
> > public Object getProperty(String propName);
> > public void setProperty(String propname, Object value);
> > getCurrentMediatorConfiguration();
> > }
> >
> > 22. We want the Synapse engine to be able to get messages not just from
> > Axis2. In general there are plenty of aspects in an Axis2 MessageContext
> > that aren't relevant to Synapse. But we don't want to replicate Axis2's MC.
> > So instead, what we would like is a SOAPMessageContext. Basically this is
> > the SOAPEnvelope plus the getTo/setTo, etc APIs but not things like
> > isPaused, AxisService, etc.
> >
> > 23. So we agreed to *ask* the Axis2 team to refactor AxisMessageContext
> > to subset a SOAPMessageContext only (= addressing + SOAPEnvelope). We think
> > there are actually some benefits to this. It will just be an interface and
> > Axis2 MC can extend it.
> >
> > To start with we can just define the interface in Axis2 as a subset of
> > the Axis2 MC and say axis2.context.MessageContext implements
> > SOAPMessageContext. So this is a very small addition to Axis2 which makes
> > Synapse much cleaner.
> >
> > We also agreed that the messageContext properties override the envelope.
> > So for example, if you do smc.setTo(address), then that should be
> > considered the correct address. If we have an XSLT mediator that changes the
> > <SOAP:Header><wsa:To> element, then its up to the mediator will update the
> > setTo() property in the SOAPMessageContext.
> >
> > 24. We want the Mediator model to be configurable. So inside any rule
> > there is always exactly one element called mediator. All mediators are
> > called as Axis2 services (i.e. thru an AxisEngine and MessageReceiver).
> >
> > The mediator has a given type. The default types will include:
> > * deployed axis2 services,
> > * spring bean assemblies,
> > * simple classes implementing Mediator,
> > * e4x.
> >
> > We also discussed bpel (both a simple stateless kind and full stateful).
> > We discussed the idea of some builtin mediators like send, fault-back, log,
> > etc.
> >
> > 25. So the mediator element looks like:
> > <synapse:mediator type="service|spring|bpel|e4x|class|builtin|....">
> > <mediatorns:blah> .. type specific stuff goes here .. </mediatorns:blah>
> >
> > </synapse:mediator>
> >
> > 26. Whatever XML sits inside the mediator is completely up to the
> > mediator type. So in e4x it could be a CDATA section containing the script,
> > in spring it could be a beans xml. It might be a BPEL document. For axis2
> > services, it could parameterise them.
> >
> > 27. There is an SPI, MediatorConfigurator, that a mediator-type creator
> > can use to parse the XML.
> >
> > *SPI*
> > interface MediatorConfigurator {
> > MediatorConfiguration parse(OMElement);
> > }
> >
> > 29. Then the mediator writer can get at the configuration inside the
> > mediation using the *API*.
> > interface MediatorConfiguration {
> > public OMElement getMediatorDefinition();
> > }
> >
> > 30. The idea is that a given type of MediatorConfiguration will also
> > expose pre-parsed information. For example, the
> >
> > Axis2ServiceMediatorConfiguration implements MediatorConfiguration {
> > public OMElement getMediatorDefinition();
> > public Parameter getParameter(String name);
> > }
> >
> > 31. So when the rules are parsed, the engine calls the
> > Axis2ServiceMediatorConfigurator with the whole <mediator> element, which
> > parses the results into the Axis2ServiceMediatorConfiguration object. Then
> > at mediate time, the Mediator can get at the parsed parameters and use them.
> >
> >
> > 32. Here is the XML structure we came up with:
> >
> > <synapse:synapse xmlns:synapse="synapse">
> >
> > <synapse:inphase order="a b c"/>
> > <synapse:outphase order=d e f"/>
> >
> > <synapse:ruleset name="a" type="xpath">
> > <xpath:rule condition="xpath-expr" direction="*|in|out">
> > <synapse:apply qos="encrypt|sign|rmd|rms"/>*
> > <synapse:mediator type="service|spring|bpel|e4x|class|uri|builtin|....">
> > <mediatorns:stuff> .. type specific stuff goes here ..
> > </mediatorns:stuff>
> > </synapse:mediator>
> > <!--exactly one mediator per rule -->
> > </xpath:rule>
> > <xpath:rule/> <!-- more rules -->
> > </synapse:ruleset>
> > <synapse:ruleset name="b" type="regex"> <!-- more rulesets -->
> > ....
> > </synapse:ruleset>
> > </synapse:synapse>
> >
> >
> > 33. That's all folks.
> >
> >
>

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Paul Fremantle <pz...@gmail.com>]

Vikas

Notes inline

On 11/8/05, Vikas <vi...@infravio.com> wrote:
>
> * The ruleEngines would be initialized with the relevant ruleSet
>

Yes absolutely

* The code in the rule engine decides wether the rules are revaluated on a
> return from the mediator or if the processing is sequential.
>

Not at first - we reserved that as a second option.

> In boolean mediate(SynapseContext sc, SOAPMessageContext smc);
> Boolean true indicates continue processing the ruleset and phases.
> Boolean false indicates stop processing the ruleset and phases.
>

This was our preferred option

0-continue in this ruleset
> 1-stop this ruleset/phase continue next one
> 2-stop period
>

This was an alternative we agreed to use if the boolean approach above
doesn't work. We are still on the "inject" model - if I want the rules
re-evaluated I use the SynapseContext.injectMessage(message).

* Where is the SynapseContext placed in connection to AXIS's context
> hierarchy?
>

Good question. I think we need to code some before we decide this.

* How about a few parts on Policy? [can wait for now]
>

Yes we have some markers in the XML for applying aspects of QoS. I didn't
describe them, but they fit under the <rule> element. They specify which
aspects to turn on/off. I agree that this can wait for now.

~Good day.
>
>  ----- Original Message -----
>
> *From:* Paul Fremantle <pz...@gmail.com>
> *To:* synapse-dev@ws.apache.org
> *Sent:* Monday, November 07, 2005 3:48 PM
> *Subject:* Long post - results of a discussion with Sanjiva, Paul, Ant
>
> (LONG POST WARNING)
>
> Folks
>
> Sanjiva, Ant and I sat down F2F and had a long discussion about the
> Synapse Architecture. Sanjiva finally explained the phases clearly and I
> think I buy it, especially if we make some enhancements. This post is *my*
> summary. Ant and Sanjiva - and everyone else - please feel free to argue,
> contradict etc. [Or even agree :-)]
>
> I know that partial F2F meetings of groups of committers can be tricky.
> The three of us came to a lot of conclusions that the rest of the team may
> or may not agree to. However, it was also very productive, and a lot of
> where we got to was incremental on the previous two F2F discussions that
> have been held, so please bear with me as I try to explain this proposed
> model.
>
> 1. Sanjiva persuaded me that the idea of multiple phases was valuable to
> help order the rules. Having written a few simple rules it began to be
> obvious that it might be hard to correctly order multiple rules simply by
> having a single ordered rule list. I persuaded Sanjiva that hardcoding just
> 3 phases is too specific. Especially if we allow there to be different kinds
> of rulesets. So we agreed to have *multiple* phases.
>
> 2. <phase order="a b c"/>
>
> 3. We didn't quite agree on directionality. It should be either the phases
> or the individual rules. Either way has issues. I think I would favour a
> scheme that supports having the same in/out if you want. The in/out is not
> from Synapse point of view but from the target service POV.
>
> So:
>
> <synapse:inphase order="a b c"/>
> <synapse:outphase order="a d c"/>
>
> 4. We also discussed using the current Axis2 xml to define phases, but it
> does seem a bit wordy/verbose.
>
> 5. The a b c refer to rulesets (see next).
>
> 6. The other issue with rules is that there doesn't seem to be a single
> model of selecting rules that will meet everyones needs. For example, simple
> regular expressions on parts of the message (wsa:To, wsa:Action) will be
> very fast. Xpaths are general but hard to write. etc. etc.
> So we agreed it was a good idea to allow different styles of rules.
>
> 7. The general xml tag for this might be:
> <synapse:ruleset name="a" type="type-of-rules">
>
> Inside a ruleset are a set of elements that are specific to the rule
> style.
> For example, in XPaths,
> <xpath:rule condition="//sq:getQuote">
> or
> <regex:rule regex="http://fremantle.org/*" field="wsa:To">
>
> Another really simple ruleset would be "all" which doesn't do any checking
> but always implements those rules.
>
> 8. Because you can only optimise one type of condition matching or rule,
> it makes sense to make a particular phase exactly one kind of ruleset.
> Because you cannot optimise across phases it makes sense to allow each phase
> to be of a different style.
> Each phase has a given style of ruleset (e.g. xpath, bpel, regex, etc). So
> the <inphase/outphase> point to an ordered list of specific rulesets.
>
> 9. For example, there may be an initial phase where we do very fast
> matching on wsa:To, and then a less efficient phase where we do specific
> content-based routing using XPaths.
>
> 10. Optimisation of rules happens within a ruleset and phase.
> Its up to the type of ruleset to define its own processing order, but in
> general encourage the ordering within a phase to be decided by the rule
> style and not by lexical ordering. In fact its up to the type of ruleset how
> it defines the processing at all. The structure is really simple - the
> Synapse engine calls on or more rulesets (in order specified). The rulesets
> then call one or more mediators as they please.
>
> 11. We think are three levels of person.
> a) Synapse developers i.e. people who develop the core synapse engine use
> SPIs to extend Synapse with new rule types and mediator types (a few)
> b) Mediator developers. Use APIs to add new mediators to Synapse (plenty)
> c) Administrators. Configure rulesets and mediators to do useful stuff.
> (lots - we hope)
>
> 12. We did want to have some fixed things. In particular, we want the a
> consistent model for creating mediators. So the mediator interface defined
> by Synapse must be supported by all ruleset styles.
>
> 13. The mediator is stateless and threadsafe (i.e. a single instance can
> be used to handle multiple requests concurrently).
>
> 14. The Mediator interface is an API.
> public interface Mediator {
> boolean mediate(SynapseContext sc, SOAPMessageContext smc);
> }
>
> Boolean true indicates continue processing the ruleset and phases.
> Boolean false indicates stop processing the ruleset and phases.
>
> We added SynapseContext so as to be able to make the Mediator
> stateless/threadsafe (as Glen has asked for). It also gives the mediator a
> pointer back into the current synapse engine.
>
> 15. We discussed the int return type. We came up with some options for
> what different return options could be....
> 0-continue in this ruleset
> 1-stop this ruleset/phase continue next one
> 2-stop period
>
> 16. and then a few more even odder ones.
> 3-restart this phase
> 4-restart beginning
>
> 17. But we thought we'd start with the boolean for the moment unless we
> proved with use cases that we needed more. I personally prefer the boolean.
> I think you can do everything you need with that.
>
> 18. Rulesets appear in code as implementations of RuleEngine.
> Each Ruleset is initialised by passing the XML to the RuleEngine.init(OMElement).
>
> Once a ruleset is initialised, you can process multiple concurrent
> messages thru it.
> Synapse has a table from ruleset type to RuleEngineImpls.
>
> RuleEngine is an *SPI*.
>
> interface RuleEngine {
> String getRulesetType();
> void init(OMElement); // gets passed the <synapse:ruleset> element.
> void process(SynapseContext sc, SOAPMessageContext smc); // stateless per
> invocation
> }
>
> So if, for example, you want a highly optimised flow through Synapse, you
> can have a ruleset called "all" that always runs every mediator. The
> AllRuleEngine.process gets called. It has a single mediator, which is
> stateless and that gets called, and the result is incredible performance. If
> you want more dynamic options then that works too.
>
> 19. A SynapseContext is created as a message comes into Synapse. It has a
> local storage to allow per-message state. It also has callbacks into the
> current Synapse runtime (whatever is running this synapse.xml). This is
> how you can add a new message into the runtime (suppose for example I want
> to do a Tee and send a copy of a message on, but I want the copy to be
> mediated by Synapse too, so I want to inject it into the current Synapse
> runtime.)
>
> 20. Somehow the SynapseContext is updated with correct config for the
> current mediator (tho we weren't convinced this is beautiful).
>
> 21. *API*
> public interface SynapseContext {
> void injectMessage(MessageContext mc, "in|out|fault", phase?)
> public Object getProperty(String propName);
> public void setProperty(String propname, Object value);
> getCurrentMediatorConfiguration();
> }
>
> 22. We want the Synapse engine to be able to get messages not just from
> Axis2. In general there are plenty of aspects in an Axis2 MessageContext
> that aren't relevant to Synapse. But we don't want to replicate Axis2's MC.
> So instead, what we would like is a SOAPMessageContext. Basically this is
> the SOAPEnvelope plus the getTo/setTo, etc APIs but not things like
> isPaused, AxisService, etc.
>
> 23. So we agreed to *ask* the Axis2 team to refactor AxisMessageContext to
> subset a SOAPMessageContext only (= addressing + SOAPEnvelope). We think
> there are actually some benefits to this. It will just be an interface and
> Axis2 MC can extend it.
>
> To start with we can just define the interface in Axis2 as a subset of the
> Axis2 MC and say axis2.context.MessageContext implements
> SOAPMessageContext. So this is a very small addition to Axis2 which makes
> Synapse much cleaner.
>
> We also agreed that the messageContext properties override the envelope.
> So for example, if you do smc.setTo(address), then that should be
> considered the correct address. If we have an XSLT mediator that changes the
> <SOAP:Header><wsa:To> element, then its up to the mediator will update the
> setTo() property in the SOAPMessageContext.
>
> 24. We want the Mediator model to be configurable. So inside any rule
> there is always exactly one element called mediator. All mediators are
> called as Axis2 services (i.e. thru an AxisEngine and MessageReceiver).
>
> The mediator has a given type. The default types will include:
> * deployed axis2 services,
> * spring bean assemblies,
> * simple classes implementing Mediator,
> * e4x.
>
> We also discussed bpel (both a simple stateless kind and full stateful).
> We discussed the idea of some builtin mediators like send, fault-back, log,
> etc.
>
> 25. So the mediator element looks like:
> <synapse:mediator type="service|spring|bpel|e4x|class|builtin|....">
> <mediatorns:blah> .. type specific stuff goes here .. </mediatorns:blah>
> </synapse:mediator>
>
> 26. Whatever XML sits inside the mediator is completely up to the mediator
> type. So in e4x it could be a CDATA section containing the script, in spring
> it could be a beans xml. It might be a BPEL document. For axis2 services, it
> could parameterise them.
>
> 27. There is an SPI, MediatorConfigurator, that a mediator-type creator
> can use to parse the XML.
>
> *SPI*
> interface MediatorConfigurator {
> MediatorConfiguration parse(OMElement);
> }
>
> 29. Then the mediator writer can get at the configuration inside the
> mediation using the *API*.
> interface MediatorConfiguration {
> public OMElement getMediatorDefinition();
> }
>
> 30. The idea is that a given type of MediatorConfiguration will also
> expose pre-parsed information. For example, the
>
> Axis2ServiceMediatorConfiguration implements MediatorConfiguration {
> public OMElement getMediatorDefinition();
> public Parameter getParameter(String name);
> }
>
> 31. So when the rules are parsed, the engine calls the
> Axis2ServiceMediatorConfigurator with the whole <mediator> element, which
> parses the results into the Axis2ServiceMediatorConfiguration object. Then
> at mediate time, the Mediator can get at the parsed parameters and use them.
>
>
> 32. Here is the XML structure we came up with:
>
> <synapse:synapse xmlns:synapse="synapse">
>
> <synapse:inphase order="a b c"/>
> <synapse:outphase order=d e f"/>
>
> <synapse:ruleset name="a" type="xpath">
> <xpath:rule condition="xpath-expr" direction="*|in|out">
> <synapse:apply qos="encrypt|sign|rmd|rms"/>*
> <synapse:mediator type="service|spring|bpel|e4x|class|uri|builtin|....">
> <mediatorns:stuff> .. type specific stuff goes here .. </mediatorns:stuff>
> </synapse:mediator>
> <!--exactly one mediator per rule -->
> </xpath:rule>
> <xpath:rule/> <!-- more rules -->
> </synapse:ruleset>
> <synapse:ruleset name="b" type="regex"> <!-- more rulesets -->
> ....
> </synapse:ruleset>
> </synapse:synapse>
>
>
> 33. That's all folks.
>
>

Re: Long post - results of a discussion with Sanjiva, Paul, Ant

[Vikas <vi...@infravio.com>]

Hi,

Pretty long mail...but really worthy.
Just checking if i got lost or am i on the right track!

So 

* The ruleEngines would be initialized with the relevant ruleSet

* The code in the rule engine decides wether the rules are revaluated on a return from the mediator or if the processing is sequential.

Extract from the mail:

In  boolean mediate(SynapseContext sc, SOAPMessageContext smc); 

Boolean true indicates continue processing the ruleset and phases.
Boolean false indicates stop processing the ruleset and phases. 

0-continue in this ruleset
1-stop this ruleset/phase continue next one
2-stop period

True maps to 0, what is the logical mapping between false and 1 or 2.

* Where is the SynapseContext placed in connection to AXIS's context hierarchy?

* How about a few parts on Policy? [can wait for now]

~Good day.



----- Original Message ----- 
  From: Paul Fremantle 
  To: synapse-dev@ws.apache.org 
  Sent: Monday, November 07, 2005 3:48 PM
  Subject: Long post - results of a discussion with Sanjiva, Paul, Ant


  (LONG POST WARNING)

  Folks

  Sanjiva, Ant and I sat down F2F and had a long discussion about the Synapse Architecture. Sanjiva finally explained the phases clearly and I think I buy it, especially if we make some enhancements. This post is *my* summary. Ant and Sanjiva - and everyone else - please feel free to argue, contradict etc. [Or even agree :-)] 

  I know that partial F2F meetings of groups of committers can be tricky. The three of us came to a lot of conclusions that the rest of the team may or may not agree to. However, it was also very productive, and a lot of where we got to was incremental on the previous two F2F discussions that have been held, so please bear with me as I try to explain this proposed model. 

  1. Sanjiva persuaded me that the idea of multiple phases was valuable to help order the rules. Having written a few simple rules it began to be obvious that it might be hard to correctly order multiple rules simply by having a single ordered rule list. I persuaded Sanjiva that hardcoding just 3 phases is too specific. Especially if we allow there to be different kinds of rulesets. So we agreed to have *multiple* phases. 

  2. <phase order="a b c"/>

  3. We didn't quite agree on directionality. It should be either the phases or the individual rules. Either way has issues. I think I would favour a scheme that supports having the same in/out if you want. The in/out is not from Synapse point of view but from the target service POV. 

  So:

  <synapse:inphase order="a b c"/>
  <synapse:outphase order="a d c"/>

  4. We also discussed using the current Axis2 xml to define phases, but it does seem a bit wordy/verbose. 

  5. The a b c refer to rulesets (see next).

  6. The other issue with rules is that there doesn't seem to be a single model of selecting rules that will meet everyones needs. For example, simple regular expressions on parts of the message (wsa:To, wsa:Action) will be very fast. Xpaths are general but hard to write. etc. etc. 
  So we agreed it was a good idea to allow different styles of rules. 

  7. The general xml tag for this might be:
  <synapse:ruleset name="a" type="type-of-rules">

  Inside a ruleset are a set of elements that are specific to the rule style. 
  For example, in XPaths, 
  <xpath:rule condition="//sq:getQuote">
  or
  <regex:rule regex="http://fremantle.org/*" field="wsa:To">

  Another really simple ruleset would be "all" which doesn't do any checking but always implements those rules.

  8. Because you can only optimise one type of condition matching or rule, it makes sense to make a particular phase exactly one kind of ruleset. Because you cannot optimise across phases it makes sense to allow each phase to be of a different style. 
  Each phase has a given style of ruleset (e.g. xpath, bpel, regex, etc). So the <inphase/outphase> point to an ordered list of specific rulesets. 

  9. For example, there may be an initial phase where we do very fast matching on wsa:To, and then a less efficient phase where we do specific content-based routing using XPaths. 

  10. Optimisation of rules happens within a ruleset and phase.
  Its up to the type of ruleset to define its own processing order, but in general encourage the ordering within a phase to be decided by the rule style and not by lexical ordering. In fact its up to the type of ruleset how it defines the processing at all. The structure is really simple - the Synapse engine calls on or more rulesets (in order specified). The rulesets then call one or more mediators as they please. 

  11. We think are three levels of person. 
  a) Synapse developers  i.e.  people who develop the core synapse engine use SPIs to extend Synapse with new rule types and mediator types (a few)
  b) Mediator developers. Use APIs to add new mediators to Synapse (plenty) 
  c) Administrators. Configure rulesets and mediators to do useful stuff. (lots - we hope)

  12. We did want to have some fixed things. In particular, we want the a consistent model for creating mediators. So the mediator interface defined by Synapse must be supported by all ruleset styles. 

  13. The mediator is stateless and threadsafe (i.e. a single instance can be used to handle multiple requests concurrently).

  14. The Mediator interface is an API. 
  public interface Mediator {
      boolean mediate(SynapseContext sc, SOAPMessageContext smc); 
  }

  Boolean true indicates continue processing the ruleset and phases.
  Boolean false indicates stop processing the ruleset and phases. 

  We added SynapseContext so as to be able to make the Mediator stateless/threadsafe (as Glen has asked for). It also gives the mediator a pointer back into the current synapse engine. 

  15. We discussed the int return type. We came up with some options for what different return options could be....
  0-continue in this ruleset
  1-stop this ruleset/phase continue next one
  2-stop period

  16. and then a few more even odder ones.
  3-restart this phase
  4-restart beginning

  17. But we thought we'd start with the boolean for the moment unless we proved with use cases that we needed more. I personally prefer the boolean. I think you can do everything you need with that. 

  18. Rulesets appear in code as implementations of RuleEngine. 
  Each Ruleset is initialised by passing the XML to the RuleEngine.init(OMElement). 
  Once a ruleset is initialised, you can process multiple concurrent messages thru it. 
  Synapse has a table from ruleset type to RuleEngineImpls.

  RuleEngine is an *SPI*.    

  interface RuleEngine {
      String getRulesetType();
      void init(OMElement); // gets passed the <synapse:ruleset> element. 
      void process(SynapseContext sc, SOAPMessageContext smc); // stateless per invocation
  }

  So if, for example, you want a highly optimised flow through Synapse, you can have a ruleset called "all" that always runs every mediator. The AllRuleEngine.process gets called. It has a single mediator, which is stateless and that gets called, and the result is incredible performance. If you want more dynamic options then that works too.

  19. A SynapseContext is created as a message comes into Synapse. It has a local storage to allow per-message state. It also has callbacks into the current Synapse runtime (whatever is running this synapse.xml). This is how you can add a new message into the runtime (suppose for example I want to do a Tee and send a copy of a message on, but I want the copy to be mediated by Synapse too, so I want to inject it into the current Synapse runtime.) 

  20. Somehow the SynapseContext is updated with correct config for the current mediator (tho we weren't convinced this is beautiful). 

  21. *API* 
  public interface SynapseContext {
      void injectMessage(MessageContext mc, "in|out|fault", phase?) 
      public Object getProperty(String propName);
      public void setProperty(String propname, Object value);
      getCurrentMediatorConfiguration();
  }

  22. We want the Synapse engine to be able to get messages not just from Axis2. In general there are plenty of aspects in an Axis2 MessageContext that aren't relevant to Synapse. But we don't want to replicate Axis2's MC. So instead, what we would like is a SOAPMessageContext. Basically this is the SOAPEnvelope plus the getTo/setTo, etc APIs but not things like isPaused, AxisService, etc. 

  23. So we agreed to *ask* the Axis2 team to refactor AxisMessageContext to subset a SOAPMessageContext only (= addressing + SOAPEnvelope).  We think there are actually some benefits to this. It will just be an interface and Axis2 MC can extend it. 

  To start with we can just define the interface in Axis2 as a subset of the Axis2 MC and say axis2.context.MessageContext implements SOAPMessageContext. So this is a very small addition to Axis2 which makes Synapse much cleaner. 

  We also agreed that the messageContext properties override the envelope. So for example, if you do smc.setTo(address), then that should be considered the correct address. If we have an XSLT mediator that changes the <SOAP:Header><wsa:To> element, then its up to the mediator will update the setTo() property in the SOAPMessageContext. 

  24. We want the Mediator model to be configurable. So inside any rule there is always exactly one element called mediator. All mediators are called as Axis2 services (i.e. thru an AxisEngine and MessageReceiver). 

  The mediator has a given type. The default types will include:
  * deployed axis2 services, 
  * spring bean assemblies, 
  * simple classes implementing Mediator, 
  * e4x.  

  We also discussed bpel (both a simple stateless kind and full stateful). We discussed the idea of some builtin mediators like send, fault-back, log, etc. 

  25. So the mediator element looks like:
      <synapse:mediator type="service|spring|bpel|e4x|class|builtin|....">
          <mediatorns:blah>        .. type specific stuff goes here .. </mediatorns:blah> 
      </synapse:mediator>
      
  26. Whatever XML sits inside the mediator is completely up to the mediator type. So in e4x it could be a CDATA section containing the script, in spring it could be a beans xml. It might be a BPEL document. For axis2 services, it could parameterise them. 

  27. There is an SPI, MediatorConfigurator, that a mediator-type creator can use to parse the XML. 

  *SPI*
   interface MediatorConfigurator {
      MediatorConfiguration parse(OMElement);
  }

  29. Then the mediator writer can get at the configuration inside the mediation using the *API*. 
  interface MediatorConfiguration {
      public OMElement getMediatorDefinition();
  }

  30. The idea is that a given type of MediatorConfiguration will also expose pre-parsed information. For example, the 

  Axis2ServiceMediatorConfiguration implements MediatorConfiguration {
      public OMElement getMediatorDefinition();
      public Parameter getParameter(String name);
  }

  31. So when the rules are parsed, the engine calls the Axis2ServiceMediatorConfigurator with the whole <mediator> element, which parses the results into the Axis2ServiceMediatorConfiguration object. Then at mediate time, the Mediator can get at the parsed parameters and use them. 

  32. Here is the XML structure we came up with:

  <synapse:synapse xmlns:synapse="synapse">
      
      <synapse:inphase order="a b c"/>
      <synapse:outphase order=d e f"/> 
      
      <synapse:ruleset name="a" type="xpath">
          <xpath:rule condition="xpath-expr" direction="*|in|out">
              <synapse:apply qos="encrypt|sign|rmd|rms"/>* 
              <synapse:mediator type="service|spring|bpel|e4x|class|uri|builtin|....">
                  <mediatorns:stuff> .. type specific stuff goes here .. </mediatorns:stuff>
              </synapse:mediator> 
              <!--exactly one mediator per rule -->
          </xpath:rule>
          <xpath:rule/> <!-- more rules -->
      </synapse:ruleset>
      <synapse:ruleset name="b" type="regex"> <!-- more rulesets --> 
      ....
      </synapse:ruleset>
  </synapse:synapse>


  33. That's all folks.