Re: question regarding sequence of header processing in synapse

[Th...@perficient.com]

Frederick, 
  It seems that it is not a matter of whether it can be done, but whether 
it should be done. As far as I can tell from what you have submitted, the 
intermediaries must be able to a) process the headers at all, ie we must 
have some kind of paradigm implemented where the headers are recognizable, 
and b) The intermediaries must contain logic necessary to be able to 
process the multiple headers, in varying order, according to a changing 
set of criteria. This we have established. My problem is that you are 
saying that the criteria is being established by a client application who 
is detailing to the service provider how a message is to be processed. 
This should not happen for many reasons. 1) It is a major security breach 
because you expose the entirety of you intermediary services to the 
outside world which is offering a buffet of information to potential 
intruders. 2) The client application thickens considerably in that now 
dynamic changes over time must be accounted for at this remote layer. 3) 
If it comes to pass that we must implement a custom processing scheme at 
the intermediary layer, it is best to isolate this programming and make it 
applicable to other clients, many of which may not yet exist. When you 
couple this client and this intermediary, you essentially bar expansion on 
both sides of the river.
 It comes down to who commands the processing sequence.
 Service providers and supporting intermediaries dictate  the sequence in 
which processing should occur and the current aim is to move away from any 
dependence on the client in this decision-making process because it 
defeats the concept of the Enterprise Service Bus altogether to house this 
understanding outside the service provider layer and its immediate 
supporting applications. This actually is a throwback to the whole 
interface concept in C++, CORBA, and later java, I want to expose an 
interface but telling the client the details regarding my application 
causes problems. If I begin telling clients how to route mesages, where 
does it end? How do I add useful capabilities to my service provider layer 
without breaking the coupling between client and server. In my humble 
opinion, I think that what you should be focusing on here is declarative 
coupling between service providers and dynamic routing of messages after 
they enter some type of intermediary gateway which is charged with the 
responsibility of message routing based on a policy. So we A) isolate the 
clients according to whatever WSDL has been published, but the client is 
blind to everything but the essential requirements. b) We implement a 
gateway intermediary that accepts the client invocation and applies a 
policy. even if this triggers a header rewrite, which is fine as long as 
it is stripped when going outside of a secure paradigm, C) implement 
supporting intermediaries that route according to the policy established. 
Thus, the intermediaries route based on the invocation, and the policy 
applies to the invocation alone. Consider, you said you may want to log, 
verify, and de-encrypt, or perform some other useful action, do we do this 
on the client, or on the invocation itself? If a client says do this, how 
do I implement the policy and be secure, and if I implement the policy to 
ensure verification or de-encryption for example, then I don't need the 
client to specify anything for me. All in all, it seems the same steps 
must be applied in either case so eliminating the client dependency is the 
correct move in terms of design. 
The usual response here is what about differentiation of clients, ie- one 
client requires said security while another does not. In this case, it 
makes no sense to deal with the header issue either because the coming 
standards for WSI interoperability and highly detailed WSDLs will 
eventually mandate seperate operation signatures and port urls, which will 
then use URL rewriting to realign on the back end when invoking the same 
service. So, we seem to be agreed here that you have to develop custom 
intermediaries. However, in this revised case, you move the responsibility 
from the client to the first intermediary and end up with a much more 
scaleable application. I think if you look at some of the more mainstream 
ESB based products you will see this multi-step protocols trying to 
establish this paradigm. Particular examples are SOA appiances such as 
datapower and reactivity, and the more scaleable ESB products such as BEA 
and websphere. If I am completely wrong here, please forgive me, everyone 
misfires. I just spend some time trying to get a grip on the client layer 
design issues and the scaleability problems of your description and have 
to wonder if perhaps there was a more eloquent way to solve this very 
important issue. 
-Thomas

Re: question regarding sequence of header processing in synapse

[Aleksander Slominski <as...@cs.indiana.edu>]

Frederik Juchart wrote:
> Hi Thomas,
>
> Thank you very much for your answer and detailed feedback.
> I agree that it is a very bad idea to have the client dictate the
> routing (as in now obsolete WS-Routing standard) as this introduces
> all of the problems and scalability issues described in your submission.
> That is why I plan to introduce a new routing header which is inserted
> by the client and only says something like "i want this message to be
> treated according to routing process A" (actually it only contains the
> URI of the process).
> Process A must already have been defined in an external BPEL-process -
> yes I would like to define routing information as bpel processes. So
> each intermediary may inspect the client inserted header which
> contains the URI and correlation id of a specific bpel process. That
> way each intermediary can query the bpel process on what is next to be
> done with the message. The process answers to these questions with the
> URI of an intermediary and the namespace of the header to be processed
> next. That makes it possible to route a message in a well defined way
> without the client knowing the details of the routing.
>
> As I am complete new to synapse: Is something like that possible?
> Could you give me some hints as to where to look further? That is if
> you don't think that I am absolutely crazy if I use BPEL as a simple
> routing table... ;)
hi Fred,

the only problem with it that i can see is that if the mechanism is
generic it would be possible to hijack clients and make them into zombie
that are controlled by BPEL overlord to direct WS-* (DeathStar?) based
DOS attacks ... which would be interesting ;-)
>
> Of course I am aware that introducing a central bpel process for
> routing purposes is critical in itself as there is no guarantee on
> this service being reachable from anywhere on the network.
it does not have to be central: if client uses WS-Addressing (or just
playing with DNS) the actual address of a BPEL router can be determined
by an intermediary (for looking on WS-Addressing headers for lower level
routing and/or resolving DNS address to multiple IP addresses and trying
them in round robin fashion etc ...)
> This would be something an administrator would have to assure prior to
> installing the extension ;) And that might be a security issue on its
> own *sigh*
it could be bad especially if the logic for routing is capable of
expressing enough for generating more than one message to send (your
aggregation service below?) without need to send a message to BPEL
controller (one message could trigger an avalanche of messages ...)
> It might very well be that defining routing as bpel process is like
> using a sledge-hammer to crack a nut - but it makes routing very
> flexible and might some day even make simultaneous processing of a
> message at multiple intermediaries possible (that is if one can come
> up with some sort of aggregation service). But that is still far far
> away...
>
> As to "who controls the decision process":
> I agree that these policies should not be controlled by the client but
> the service layer.
> Yet I think that even if you move the decision making away from the
> client it would still be necessary to have the client insert the
> required headers. So you can decide within the service layer when to
> invoke a specific service but not the parameters of that service. Or
> am I completely wrong here?
you could also create clients that have "cookies" that capture in an
opaque way bpel state (just a long number maybe with crypto and
timestamped to make sure client can not manipulate it) and it is passed
to routing controller for routing the message in a bigger conversation.

still i wonder what is exactly the use case for it? is it compelling
enough? how does it compare to other ways of doing the same?

best,

alek
>
> homas.Kelly@perficient.com wrote:
>>
>> Frederick,
>>   It seems that it is not a matter of whether it can be done, but
>> whether it should be done. As far as I can tell from what you have
>> submitted, the intermediaries must be able to a) process the headers
>> at all, ie we must have some kind of paradigm implemented where the
>> headers are recognizable, and b) The intermediaries must contain
>> logic necessary to be able to process the multiple headers, in
>> varying order, according to a changing set of criteria. This we have
>> established. My problem is that you are saying that the criteria is
>> being established by a client application who is detailing to the
>> service provider how a message is to be processed. This should not
>> happen for many reasons. 1) It is a major security breach because you
>> expose the entirety of you intermediary services to the outside world
>> which is offering a buffet of information to potential intruders. 2)
>> The client application thickens considerably in that now dynamic
>> changes over time must be accounted for at this remote layer. 3) If
>> it comes to pass that we must implement a custom processing scheme at
>> the intermediary layer, it is best to isolate this programming and
>> make it applicable to other clients, many of which may not yet exist.
>> When you couple this client and this intermediary, you essentially
>> bar expansion on both sides of the river.
>>  It comes down to who commands the processing sequence.
>>  Service providers and supporting intermediaries dictate  the
>> sequence in which processing should occur and the current aim is to
>> move away from any dependence on the client in this decision-making
>> process because it defeats the concept of the Enterprise Service Bus
>> altogether to house this understanding outside the service provider
>> layer and its immediate supporting applications. This actually is a
>> throwback to the whole interface concept in C++, CORBA, and later
>> java, I want to expose an interface but telling the client the
>> details regarding my application causes problems. If I begin telling
>> clients how to route mesages, where does it end? How do I add useful
>> capabilities to my service provider layer without breaking the
>> coupling between client and server. In my humble opinion, I think
>> that what you should be focusing on here is declarative coupling
>> between service providers and dynamic routing of messages after they
>> enter some type of intermediary gateway which is charged with the
>> responsibility of message routing based on a policy. So we A) isolate
>> the clients according to whatever WSDL has been published, but the
>> client is blind to everything but the essential requirements. b) We
>> implement a gateway intermediary that accepts the client invocation
>> and applies a policy. even if this triggers a header rewrite, which
>> is fine as long as it is stripped when going outside of a secure
>> paradigm, C) implement supporting intermediaries that route according
>> to the policy established.
>> Thus, the intermediaries route based on the invocation, and the
>> policy applies to the invocation alone. Consider, you said you may
>> want to log, verify, and de-encrypt, or perform some other useful
>> action, do we do this on the client, or on the invocation itself? If
>> a client says do this, how do I implement the policy and be secure,
>> and if I implement the policy to ensure verification or de-encryption
>> for example, then I don't need the client to specify anything for me.
>> All in all, it seems the same steps must be applied in either case so
>> eliminating the client dependency is the correct move in terms of
>> design.
>> The usual response here is what about differentiation of clients, ie-
>> one client requires said security while another does not. In this
>> case, it makes no sense to deal with the header issue either because
>> the coming standards for WSI interoperability and highly detailed
>> WSDLs will eventually mandate seperate operation signatures and port
>> urls, which will then use URL rewriting to realign on the back end
>> when invoking the same service. So, we seem to be agreed here that
>> you have to develop custom intermediaries. However, in this revised
>> case, you move the responsibility from the client to the first
>> intermediary and end up with a much more scaleable application. I
>> think if you look at some of the more mainstream ESB based products
>> you will see this multi-step protocols trying to establish this
>> paradigm. Particular examples are SOA appiances such as datapower and
>> reactivity, and the more scaleable ESB products such as BEA and
>> websphere. If I am completely wrong here, please forgive me, everyone
>> misfires. I just spend some time trying to get a grip on the client
>> layer design issues and the scaleability problems of your description
>> and have to wonder if perhaps there was a more eloquent way to solve
>> this very important issue.
>> -Thomas
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: synapse-dev-help@ws.apache.org
>


-- 
The best way to predict the future is to invent it - Alan Kay


---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-dev-help@ws.apache.org

Re: question regarding sequence of header processing in synapse

[Frederik Juchart <fr...@juchart.de>]

Hi Thomas,

Thank you very much for your answer and detailed feedback.
I agree that it is a very bad idea to have the client dictate the 
routing (as in now obsolete WS-Routing standard) as this introduces all 
of the problems and scalability issues described in your submission.
That is why I plan to introduce a new routing header which is inserted 
by the client and only says something like "i want this message to be 
treated according to routing process A" (actually it only contains the 
URI of the process).
Process A must already have been defined in an external BPEL-process - 
yes I would like to define routing information as bpel processes. So 
each intermediary may inspect the client inserted header which contains 
the URI and correlation id of a specific bpel process. That way each 
intermediary can query the bpel process on what is next to be done with 
the message. The process answers to these questions with the URI of an 
intermediary and the namespace of the header to be processed next. That 
makes it possible to route a message in a well defined way without the 
client knowing the details of the routing.

As I am complete new to synapse: Is something like that possible? Could 
you give me some hints as to where to look further? That is if you don't 
think that I am absolutely crazy if I use BPEL as a simple routing 
table... ;)

Of course I am aware that introducing a central bpel process for routing 
purposes is critical in itself as there is no guarantee on this service 
being reachable from anywhere on the network. This would be something an 
administrator would have to assure prior to installing the extension ;) 
And that might be a security issue on its own *sigh*
It might very well be that defining routing as bpel process is like 
using a sledge-hammer to crack a nut - but it makes routing very 
flexible and might some day even make simultaneous processing of a 
message at multiple intermediaries possible (that is if one can come up 
with some sort of aggregation service). But that is still far far away...

As to "who controls the decision process":
I agree that these policies should not be controlled by the client but 
the service layer.
Yet I think that even if you move the decision making away from the 
client it would still be necessary to have the client insert the 
required headers. So you can decide within the service layer when to 
invoke a specific service but not the parameters of that service. Or am 
I completely wrong here?


-Fred

Thomas.Kelly@perficient.com wrote:
>
> Frederick,
>   It seems that it is not a matter of whether it can be done, but 
> whether it should be done. As far as I can tell from what you have 
> submitted, the intermediaries must be able to a) process the headers 
> at all, ie we must have some kind of paradigm implemented where the 
> headers are recognizable, and b) The intermediaries must contain logic 
> necessary to be able to process the multiple headers, in varying 
> order, according to a changing set of criteria. This we have 
> established. My problem is that you are saying that the criteria is 
> being established by a client application who is detailing to the 
> service provider how a message is to be processed. This should not 
> happen for many reasons. 1) It is a major security breach because you 
> expose the entirety of you intermediary services to the outside world 
> which is offering a buffet of information to potential intruders. 2) 
> The client application thickens considerably in that now dynamic 
> changes over time must be accounted for at this remote layer. 3) If it 
> comes to pass that we must implement a custom processing scheme at the 
> intermediary layer, it is best to isolate this programming and make it 
> applicable to other clients, many of which may not yet exist. When you 
> couple this client and this intermediary, you essentially bar 
> expansion on both sides of the river.
>  It comes down to who commands the processing sequence.
>  Service providers and supporting intermediaries dictate  the sequence 
> in which processing should occur and the current aim is to move away 
> from any dependence on the client in this decision-making process 
> because it defeats the concept of the Enterprise Service Bus 
> altogether to house this understanding outside the service provider 
> layer and its immediate supporting applications. This actually is a 
> throwback to the whole interface concept in C++, CORBA, and later 
> java, I want to expose an interface but telling the client the details 
> regarding my application causes problems. If I begin telling clients 
> how to route mesages, where does it end? How do I add useful 
> capabilities to my service provider layer without breaking the 
> coupling between client and server. In my humble opinion, I think that 
> what you should be focusing on here is declarative coupling between 
> service providers and dynamic routing of messages after they enter 
> some type of intermediary gateway which is charged with the 
> responsibility of message routing based on a policy. So we A) isolate 
> the clients according to whatever WSDL has been published, but the 
> client is blind to everything but the essential requirements. b) We 
> implement a gateway intermediary that accepts the client invocation 
> and applies a policy. even if this triggers a header rewrite, which is 
> fine as long as it is stripped when going outside of a secure 
> paradigm, C) implement supporting intermediaries that route according 
> to the policy established.
> Thus, the intermediaries route based on the invocation, and the policy 
> applies to the invocation alone. Consider, you said you may want to 
> log, verify, and de-encrypt, or perform some other useful action, do 
> we do this on the client, or on the invocation itself? If a client 
> says do this, how do I implement the policy and be secure, and if I 
> implement the policy to ensure verification or de-encryption for 
> example, then I don't need the client to specify anything for me. All 
> in all, it seems the same steps must be applied in either case so 
> eliminating the client dependency is the correct move in terms of design.
> The usual response here is what about differentiation of clients, ie- 
> one client requires said security while another does not. In this 
> case, it makes no sense to deal with the header issue either because 
> the coming standards for WSI interoperability and highly detailed 
> WSDLs will eventually mandate seperate operation signatures and port 
> urls, which will then use URL rewriting to realign on the back end 
> when invoking the same service. So, we seem to be agreed here that you 
> have to develop custom intermediaries. However, in this revised case, 
> you move the responsibility from the client to the first intermediary 
> and end up with a much more scaleable application. I think if you look 
> at some of the more mainstream ESB based products you will see this 
> multi-step protocols trying to establish this paradigm. Particular 
> examples are SOA appiances such as datapower and reactivity, and the 
> more scaleable ESB products such as BEA and websphere. If I am 
> completely wrong here, please forgive me, everyone misfires. I just 
> spend some time trying to get a grip on the client layer design issues 
> and the scaleability problems of your description and have to wonder 
> if perhaps there was a more eloquent way to solve this very important 
> issue.
> -Thomas
>


---------------------------------------------------------------------
To unsubscribe, e-mail: synapse-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: synapse-dev-help@ws.apache.org