You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cxf.apache.org by Andreas Veithen <an...@gmail.com> on 2010/10/08 13:23:55 UTC
Re: Security token processing without SAAJ dependency
On Tue, Sep 28, 2010 at 22:58, Daniel Kulp <dk...@apache.org> wrote:
> On Tuesday 28 September 2010 3:00:27 pm Oliver Wulff wrote:
>> Hi all
>>
>> CXF delegates all the incoming security token processing down to WSS4J
>> which requires the SAAJ interceptor due to the requirement of a dom tree.
>>
>> If you don't use a SAML token as a signing or encryption token
>> (holder-of-key) you can validate the soap header and its signature without
>> creating a dom tree or only for the saml token itself.
>>
>> If you use a username token you don't have to pass it down to WSS4J.
>> Further, the STS client could be used to validate the UsernameToken
>> against an STS.
>>
>> If you use a binary security token which is not used as a signing or
>> encryption token (x509) then you can process this in a steaming manner.
>>
>> What are your thoughts and ideas on that?
>
> Colm and I have tossed some ideas around this a bit in regards to WSS4J, just
> never had time to really dig in.
>
> First, to clarify, CXF always parses the headers into a DOM. Thus, any stuff
> that does processing on the headers can feel free to use the DOM. Passing
> the UsernameToken into WSS4J is really a non-issue for this. We do have
> optimizations in place for the UsernameToken ws-secpol stuff that does bypass
> the SAAJ stuff for UsernameToken, but it processes it by calling the WSS4J
> handler directly. We definitely could do this for other cases as well.
>
> HOWEVER, the better option, IMO is to get a better callback and element lookup
> mechanism in place in WSS4J. We can pass the "header only" DOM that we have
> to WSS4J and if it NEEDS the body, it can call back to us where we can build
> the rest of the SAAJ/DOM model. (actually, longer term, going some sort of
> stax route could be cool as well) That would simplify much of the "special
> cases that don't need to load the body" to basically being automatic. We
> wouldn't need to write code around any of the special cases. This would take
> care of the INCOMING message.
>
> Thus, the work around the special cases and such is for the outbound chain,
> but that's not nearly as much work. An interceptor can add custom things to
> the security header via our normal List<Header> stuff and the security things
> can augment it later if they get triggered.
>
> --
> Daniel Kulp
> dkulp@apache.org
> http://dankulp.com/blog
>
The other alternative is to leave WSS4J unchanged and to let the SAAJ
implementation materialize the different parts of the object model on
demand. I have a working PoC that does this. It currently implements
an alternative SAAJInInterceptor that provides the following features:
* It always puts a SOAPMessage into the Message object, but the
SOAPPart (including the SOAP headers) is only instantiated on demand
when the SOAPMessage is accessed.
* The content of the SOAPBody is only materialized when is it actually
accessed. In addition, deferred parsing and building of the nodes is
supported for this part of the message. This means that WSS4J can
start processing the body of the message before it has been received
entirely.
* If the body has not been accessed, the StAX events are passed
through unchanged (e.g. to the interceptor that does the databinding
stuff). If the body has been accessed through the SAAJ API, then a
W3CDOMStreamReader is used to regenerate the StAX events.
All this is completely transparent for the interceptors further down the chain.
Of course, this is impossible to achieve using a standard SAAJ
implementation. Therefore the interceptor uses an alternative SAAJ
implementation which I have written as part of a larger project called
DDOM [1]. The implementation is still incomplete, but to give you an
idea about its current status:
* The underlying DOM implementation successfully passes the W3C DOM1
and DOM2 test suites, except for features not relevant to SOAP
processing (such as DTD information, entity references, etc.). It also
successfully passes the most important parts of the DOM3 test suite.
* When configured as SAAJ implementation in the CXF build (without the
custom SAAJInInterceptor), it passes all unit tests up to the systests
(It seems that this is primarily due to the fact that some of the
systests use SAAJ to compose test messages).
* When configured as DOM implementation in the WSS4J build, it passes
all unit tests except some of the tests related to SAML (because
OpenSAML expects to get schema support from the DOM implementation
available though JAXP).
* I tested the implementation successfully with the scenarios
developed by Dennis Sosnoski [2] (both by simply configuring DDOM as
SAAJ implementation as well as using my alternative
SAAJInInterceptor).
The code for the custom SAAJInInterceptor is available here:
http://ddom.googlecode.com/svn/trunk/modules/ddom-cxf/
Note that besides the limitations mentioned above, there are currently
some features missing in this SAAJInInterceptor implementation that
needs to be mentioned, namely processing of SOAP faults and the
ability to access attachments through the SAAJ API.
Some preconfigured CXF (2.3.0) clients and services (containing code
from Dennis' article) are available here:
http://ddom.googlecode.com/svn/trunk/perftests/
Note that for the moment, they still use Sun's SAAJ implementation for
outgoing messages. Therefore the interesting things happen in the
response processing on the client side.
I did some initial performance tests with the scenarios developed by
Dennis and the results suggest that my SAAJ implementation performs
slightly better than the one from Sun, resulting in a few percent
points improvement in the "sign" and "signencr" scenarios. Note that
the features of the custom SAAJInInterceptor are not relevant here
because the body is accessed anyway in these scenarios. I think that
the improvement primarily results from a reduced memory footprint.
Maybe Dennis could volunteer as an "independent" performance tester to
confirm this?
BTW, while doing some profiling, I noticed that in the "signencr"
scenario, CXF suffers from a flaw in Santuario's code, namely
org.apache.xml.security.encryption.XMLCipher.Serializer#deserialize,
which does some highly inefficient stuff. I think that by doing that
part of the processing in a smarter way, it may be possible to perform
better than Metro in the small-response case and to increase the gap
with respect to Metro in the large-response case.
Andreas
PS: As noted between the lines by Daniel, optimizing the security
processing for outgoing messages is much more difficult, at least in
CXF. In principle, the Axis2 architecture would allow to achieve this
more easily, but before this can be leveraged, there are some other
performance issues in Axis2/Rampart that need to be fixed...
[1] http://code.google.com/p/ddom/
[2] http://www.ibm.com/developerworks/java/library/j-jws14/index.html
Re: Security token processing without SAAJ dependency
Posted by Andreas Veithen <an...@gmail.com>.
On Mon, Oct 11, 2010 at 20:03, Daniel Kulp <dk...@apache.org> wrote:
> On Saturday 09 October 2010 9:13:27 am Andreas Veithen wrote:
>> On Fri, Oct 8, 2010 at 18:13, Daniel Kulp <dk...@apache.org> wrote:
>> > And I think this is where the issues may start popping up, but definitely
>> > resolvable. If you look at the CXF survey, one of the MAJOR "areas of
>> > improvement" for CXF was too many external dependencies. I'm kind of
>> > reluctant to add more "required" deps. Thus, the security stuff would
>> > need to work with the default SAAJ impl (in particular, the one built
>> > into the JDK), but I could definitely see making something like this as
>> > an optional "performance enhancment" type thing. Ideally, the existing
>> > SAAJ interceptor would be smart enough to checkt he SAAJ impl being used
>> > and do the smart thing depending on what it sees.
>>
>> What about the following solution:
>> * Introduce a new SAAJManager interface that encapsulates the SAAJ
>> handling stuff.
>> * Create a default implementation based on the code currently
>> contained in the standard SAAJ interceptors.
>> * Refactor SAAJInInterceptor and SAAJOutInterceptor to look up the
>> SAAJManager using Bus#getExtension. If none is found, fall back to the
>> default implementation.
>> * Maybe also refactor the WSS4J interceptors to use SAAJManager
>> instead of instantiating SAAJ(In|Out)Interceptor.
>> * My ddom-cxf stuff would then provide an alternative SAAJManager
>> implementation and automatically register that as a bus extension.
>
> Sure. That works. :-)
Now we only need somebody who does the SAAJ and WSS4J interceptor
refactoring to create the extension point. How are we going to do
this?
>> > If the DDOM stuff we need can be squashed/shaded
>> > into a single jar that is just a drop in replacement of saaj-impl, that
>> > concern could also be aleviated quite a bit as well.
>>
>> Yes, I was also thinking the same. There are a couple of other reasons
>> why this is a good idea:
>> * The DDOM project is split into a multitude of smaller modules, so
>> that for every use case, you only get the stuff that is really needed
>> (e.g. some modules depend on the Axiom APIs, but you don't want that
>> as a transitive dependency for the CXF stuff ;-). This works well when
>> Maven is used, but of course it's a pain for users who use other
>> tools.
>> * The code in the CXF stuff is (currently) tightly coupled to some
>> internal DDOM APIs that are likely to change over time. Thus,
>> packaging all the required stuff into a single JAR and shade it makes
>> sense because it avoids conflicts caused by incompatible versions of
>> DDOM modules.
>
> That all makes sense. Keeping the dep list small is definitely a welcome
> thing.
>
>
>> > That said, the callback method added to WSS4J has an additional use case
>> > that Sergey and I have been noodling on outside of SOAP. One thing
>> > we're trying to figure out how to do is leverage some of the WS-SecPol
>> > support to provide message level security for XML REST payloads. With
>> > the callback, we MAY be able to have separate XML sources for the
>> > security header and the actual payload (think separate MIME containers)
>> > where we feed the security header into WSS4J, and it calls back out to
>> > us to load the other parts as/if needed. Very early in the "noodling in
>> > our heads" stages. :-)
>>
>> The DDOM solution here would simply be to compose the equivalent SOAP
>> message and pass that to WSS4J. This would mean creating a skeleton
>> SOAP message and adding the source objects for the header and body
>> parts to that skeleton. It is definitely possible to defer the
>> expansion of these parts (and even the lookup of the corresponding
>> MIME parts from the message) until the very last moment. This is
>> conceptually similar to your solution except that the callbacks are
>> invoked by the DOM/SAAJ implementation instead of WSS4J. This could
>> have some advantages:
>>
>> * The scope of WSS4J would remain strictly limited to WS-Security and
>> no API changes are needed.
>> * If I understand your idea correctly, then conceptually you are
>> defining a mapping between REST style requests/responses and SOAP
>> messages conforming to WS-Security. With DDOM, the code that
>> implements this would simply be a direct transcription of these
>> mapping rules (plus of course some glue code).
>
> This is beginning to sound too much like Synapse.
>
> Seriously, this isn't ideal for certain policy elements. For example, an
> XPATH expression for the encrypted elements should, ideally, not have any soap
> things in it for a REST invokation. I'd need to think about this some more
> though. The main issue here would be the fact that the Message object that is
> passed from Interceptor to Interceptor when using JAX-RS is not a SoapMessage.
> It's probably just a MessageImpl. If you try to even configure any of the
> WSS4J or SAAJ interceptors onto the JAX-RS chains, it will throw and exception
> before the handleMessage call is even called. The Java runtime won't even
> allow it. The WSS4J interceptors would need to be updated to adjust for that
> anyway.
You are correct here. If one specifies REST security by defining
equivalencies between REST request/responses and SOAP messages, then
there is indeed a problem for XPath expressions.
>> >> It also
>> >> successfully passes the most important parts of the DOM3 test suite.
>> >> * When configured as SAAJ implementation in the CXF build (without the
>> >> custom SAAJInInterceptor), it passes all unit tests up to the systests
>> >> (It seems that this is primarily due to the fact that some of the
>> >> systests use SAAJ to compose test messages).
>> >
>> > Likely yes. Do you have a feeling as to whether those are issues in
>> > the CXF tests or issues with DDOM?
>>
>> These are issues in DDOM. They are simply due to the fact that many
>> SAAJ methods in DDOM are still unimplemented ;-).
>
> It's almost a shame you are doing this at google instead of in Apache
> someplace. Apache has access to the standalone SAAJ TCK for testing that,
> but we're only really allowed to test it on Apache projects. Maybe convince
> Geronimo to try using DDOM as an optional SAAJ impl and see how the SAAJ J2EE
> tests progress. Oh well.
Yes and no. The scope, goals and architecture of the project have
changed quite a lot over time. It grew out of some code that was
initially built on top of Axiom and then recycled into a pure DOM
implementation supporting deferred parsing and building (hence the
name DDOM). Only later I noticed that by using bytecode manipulation
(first with AspectJ and now using a custom ASM based weaver) it is
possible to support multiple APIs (DOM, SAAJ, Axiom) and even to mix
them (DOM+Axiom). At this point, the primary goal became to provide a
decent replacement for Axiom DOOM in order to solve some of the
performance problems with WS-Security processing in Axis2. Even now,
the scope of the project is still changing. E.g. the idea to use it as
a SAAJ implementation that enables tighter integration into an
existing SOAP stack is new and actually grew out of this discussion.
I think that doing the first phases of the project outside of Apache
was necessary to have the required freedom and flexibility for this
kind of changes in scope and goals and to explore new use cases and
solutions. The question is not so much whether this should become or
not an Apache project, but where, how and when this should happen.
>> What I wanted to say
>> here is that there is enough stuff in DDOM's SAAJ implementation to
>> play nicely with CXF's own code. Of course, once the SOAPMessage is
>> exposed to user code through a JAX-WS handler or provider endpoint,
>> this is no longer enough. As I said, for the moment it's a PoC.
>
> OK. Makes sense. Gotta start someplace. :-)
>
>
>> > I see in the code that it doesn't support attachments either. That
>> > would be required for the JAX-WS TCK. (and likely for some of the
>> > systests as well) That said, those should be fairly easy to handle since
>> > you have the SoapMessage. The DDOM version could be MUCH more
>> > efficient by not buffering the attachments unless they really are
>> > accessed. One of my "issues" with the JAX-WS handlers is they require
>> > us to stop streaming EVERYTHING and buffer it, even if the only thing
>> > the handler does is: System.out.println("Hello World!");
>> > this could be a huge help with that.
>>
>> The SAAJ stuff in DDOM is actually split into two parts:
>> * The part that implements the SAAJ object model. This contains the
>> code that extends DOM and also some partial implementations of
>> SOAPMessage and SOAPPart. On the other hand, it doesn't contain any
>> MIME parsing or attachment management logic.
>
> And it shouldn't need to contain any MIME parsing stuff for the CXF version.
> CXF has already taken care of that.
>
>> * The part that provides the SAAJ factories and thus the complete SAAJ
>> implementation. Of course this part depends on the first part and will
>> eventually contain the MIME parsing logic to satisfy the requirements
>> of the SAAJ spec.
>
> But we don't use any of that in CXF. So irrelevant for a first pass CXF
> integration. Once we get into using it to support Dispatch clients and
> Provider based services, I suppose it would be required though.
I think you got the idea ;-)
>
>> When running the CXF test suites against DDOM I use both parts as a
>> drop-in replacement for Sun's saaj-impl. On the other hand, the custom
>> SAAJInInterceptor only uses the first part. This means that the
>> SOAPMessage implementation is not the same as the one that would be
>> created by MessageFactory. Instead it uses an implementation that is
>> basically a wrapper around a org.apache.cxf.binding.soap.SoapMessage
>> object. This can be seen here:
>>
>> http://ddom.googlecode.com/svn/trunk/modules/ddom-cxf/src/main/java/com/goo
>> glecode/ddom/cxf/SOAPMessageImpl.java
>
> Right. I was looking at that.
>
>
>> The idea is that the methods that are used to manage attachments (and
>> that are currently unimplemented) would all delegate to the
>> corresponding CXF APIs. This means that all optimizations done by CXF
>> would also work when accessing the attachments through SAAJ. Since
>> SAAJ exposes attachments via iterators, the only case were all
>> attachments are buffered would be a call to countAttachments...
>
> OK. I didn't read far enough again. :-) It's pretty easy. The CXF
> SoapMessage (or just Message) has an Attachment list that can be retrieved
> via:
>
> Collection<Attachment> attachments =
> CastUtils.cast((Collection<?>)mc.get(Message.ATTACHMENTS));
>
> That can easily be wrapped or similar to implement the required SAAJ API's.
>
>
>
> Anyway, this is definitely all very interesting. Thanks for pursuing this.
> Kind of can't wait till Colm frees up a bit to have his brain noodling on
> things as well. :-)
>
>
> --
> Daniel Kulp
> dkulp@apache.org
> http://dankulp.com/blog
>
Re: Security token processing without SAAJ dependency
Posted by Daniel Kulp <dk...@apache.org>.
On Saturday 09 October 2010 9:13:27 am Andreas Veithen wrote:
> On Fri, Oct 8, 2010 at 18:13, Daniel Kulp <dk...@apache.org> wrote:
> > And I think this is where the issues may start popping up, but definitely
> > resolvable. If you look at the CXF survey, one of the MAJOR "areas of
> > improvement" for CXF was too many external dependencies. I'm kind of
> > reluctant to add more "required" deps. Thus, the security stuff would
> > need to work with the default SAAJ impl (in particular, the one built
> > into the JDK), but I could definitely see making something like this as
> > an optional "performance enhancment" type thing. Ideally, the existing
> > SAAJ interceptor would be smart enough to checkt he SAAJ impl being used
> > and do the smart thing depending on what it sees.
>
> What about the following solution:
> * Introduce a new SAAJManager interface that encapsulates the SAAJ
> handling stuff.
> * Create a default implementation based on the code currently
> contained in the standard SAAJ interceptors.
> * Refactor SAAJInInterceptor and SAAJOutInterceptor to look up the
> SAAJManager using Bus#getExtension. If none is found, fall back to the
> default implementation.
> * Maybe also refactor the WSS4J interceptors to use SAAJManager
> instead of instantiating SAAJ(In|Out)Interceptor.
> * My ddom-cxf stuff would then provide an alternative SAAJManager
> implementation and automatically register that as a bus extension.
Sure. That works. :-)
> > If the DDOM stuff we need can be squashed/shaded
> > into a single jar that is just a drop in replacement of saaj-impl, that
> > concern could also be aleviated quite a bit as well.
>
> Yes, I was also thinking the same. There are a couple of other reasons
> why this is a good idea:
> * The DDOM project is split into a multitude of smaller modules, so
> that for every use case, you only get the stuff that is really needed
> (e.g. some modules depend on the Axiom APIs, but you don't want that
> as a transitive dependency for the CXF stuff ;-). This works well when
> Maven is used, but of course it's a pain for users who use other
> tools.
> * The code in the CXF stuff is (currently) tightly coupled to some
> internal DDOM APIs that are likely to change over time. Thus,
> packaging all the required stuff into a single JAR and shade it makes
> sense because it avoids conflicts caused by incompatible versions of
> DDOM modules.
That all makes sense. Keeping the dep list small is definitely a welcome
thing.
> > That said, the callback method added to WSS4J has an additional use case
> > that Sergey and I have been noodling on outside of SOAP. One thing
> > we're trying to figure out how to do is leverage some of the WS-SecPol
> > support to provide message level security for XML REST payloads. With
> > the callback, we MAY be able to have separate XML sources for the
> > security header and the actual payload (think separate MIME containers)
> > where we feed the security header into WSS4J, and it calls back out to
> > us to load the other parts as/if needed. Very early in the "noodling in
> > our heads" stages. :-)
>
> The DDOM solution here would simply be to compose the equivalent SOAP
> message and pass that to WSS4J. This would mean creating a skeleton
> SOAP message and adding the source objects for the header and body
> parts to that skeleton. It is definitely possible to defer the
> expansion of these parts (and even the lookup of the corresponding
> MIME parts from the message) until the very last moment. This is
> conceptually similar to your solution except that the callbacks are
> invoked by the DOM/SAAJ implementation instead of WSS4J. This could
> have some advantages:
>
> * The scope of WSS4J would remain strictly limited to WS-Security and
> no API changes are needed.
> * If I understand your idea correctly, then conceptually you are
> defining a mapping between REST style requests/responses and SOAP
> messages conforming to WS-Security. With DDOM, the code that
> implements this would simply be a direct transcription of these
> mapping rules (plus of course some glue code).
This is beginning to sound too much like Synapse.
Seriously, this isn't ideal for certain policy elements. For example, an
XPATH expression for the encrypted elements should, ideally, not have any soap
things in it for a REST invokation. I'd need to think about this some more
though. The main issue here would be the fact that the Message object that is
passed from Interceptor to Interceptor when using JAX-RS is not a SoapMessage.
It's probably just a MessageImpl. If you try to even configure any of the
WSS4J or SAAJ interceptors onto the JAX-RS chains, it will throw and exception
before the handleMessage call is even called. The Java runtime won't even
allow it. The WSS4J interceptors would need to be updated to adjust for that
anyway.
> >> It also
> >> successfully passes the most important parts of the DOM3 test suite.
> >> * When configured as SAAJ implementation in the CXF build (without the
> >> custom SAAJInInterceptor), it passes all unit tests up to the systests
> >> (It seems that this is primarily due to the fact that some of the
> >> systests use SAAJ to compose test messages).
> >
> > Likely yes. Do you have a feeling as to whether those are issues in
> > the CXF tests or issues with DDOM?
>
> These are issues in DDOM. They are simply due to the fact that many
> SAAJ methods in DDOM are still unimplemented ;-).
It's almost a shame you are doing this at google instead of in Apache
someplace. Apache has access to the standalone SAAJ TCK for testing that,
but we're only really allowed to test it on Apache projects. Maybe convince
Geronimo to try using DDOM as an optional SAAJ impl and see how the SAAJ J2EE
tests progress. Oh well.
> What I wanted to say
> here is that there is enough stuff in DDOM's SAAJ implementation to
> play nicely with CXF's own code. Of course, once the SOAPMessage is
> exposed to user code through a JAX-WS handler or provider endpoint,
> this is no longer enough. As I said, for the moment it's a PoC.
OK. Makes sense. Gotta start someplace. :-)
> > I see in the code that it doesn't support attachments either. That
> > would be required for the JAX-WS TCK. (and likely for some of the
> > systests as well) That said, those should be fairly easy to handle since
> > you have the SoapMessage. The DDOM version could be MUCH more
> > efficient by not buffering the attachments unless they really are
> > accessed. One of my "issues" with the JAX-WS handlers is they require
> > us to stop streaming EVERYTHING and buffer it, even if the only thing
> > the handler does is: System.out.println("Hello World!");
> > this could be a huge help with that.
>
> The SAAJ stuff in DDOM is actually split into two parts:
> * The part that implements the SAAJ object model. This contains the
> code that extends DOM and also some partial implementations of
> SOAPMessage and SOAPPart. On the other hand, it doesn't contain any
> MIME parsing or attachment management logic.
And it shouldn't need to contain any MIME parsing stuff for the CXF version.
CXF has already taken care of that.
> * The part that provides the SAAJ factories and thus the complete SAAJ
> implementation. Of course this part depends on the first part and will
> eventually contain the MIME parsing logic to satisfy the requirements
> of the SAAJ spec.
But we don't use any of that in CXF. So irrelevant for a first pass CXF
integration. Once we get into using it to support Dispatch clients and
Provider based services, I suppose it would be required though.
> When running the CXF test suites against DDOM I use both parts as a
> drop-in replacement for Sun's saaj-impl. On the other hand, the custom
> SAAJInInterceptor only uses the first part. This means that the
> SOAPMessage implementation is not the same as the one that would be
> created by MessageFactory. Instead it uses an implementation that is
> basically a wrapper around a org.apache.cxf.binding.soap.SoapMessage
> object. This can be seen here:
>
> http://ddom.googlecode.com/svn/trunk/modules/ddom-cxf/src/main/java/com/goo
> glecode/ddom/cxf/SOAPMessageImpl.java
Right. I was looking at that.
> The idea is that the methods that are used to manage attachments (and
> that are currently unimplemented) would all delegate to the
> corresponding CXF APIs. This means that all optimizations done by CXF
> would also work when accessing the attachments through SAAJ. Since
> SAAJ exposes attachments via iterators, the only case were all
> attachments are buffered would be a call to countAttachments...
OK. I didn't read far enough again. :-) It's pretty easy. The CXF
SoapMessage (or just Message) has an Attachment list that can be retrieved
via:
Collection<Attachment> attachments =
CastUtils.cast((Collection<?>)mc.get(Message.ATTACHMENTS));
That can easily be wrapped or similar to implement the required SAAJ API's.
Anyway, this is definitely all very interesting. Thanks for pursuing this.
Kind of can't wait till Colm frees up a bit to have his brain noodling on
things as well. :-)
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog
Re: Security token processing without SAAJ dependency
Posted by Andreas Veithen <an...@gmail.com>.
On Fri, Oct 8, 2010 at 18:13, Daniel Kulp <dk...@apache.org> wrote:
> On Friday 08 October 2010 7:23:55 am Andreas Veithen wrote:
>> The other alternative is to leave WSS4J unchanged and to let the SAAJ
>> implementation materialize the different parts of the object model on
>> demand. I have a working PoC that does this. It currently implements
>> an alternative SAAJInInterceptor that provides the following features:
>>
>> * It always puts a SOAPMessage into the Message object, but the
>> SOAPPart (including the SOAP headers) is only instantiated on demand
>> when the SOAPMessage is accessed.
>> * The content of the SOAPBody is only materialized when is it actually
>> accessed. In addition, deferred parsing and building of the nodes is
>> supported for this part of the message. This means that WSS4J can
>> start processing the body of the message before it has been received
>> entirely.
>> * If the body has not been accessed, the StAX events are passed
>> through unchanged (e.g. to the interceptor that does the databinding
>> stuff). If the body has been accessed through the SAAJ API, then a
>> W3CDOMStreamReader is used to regenerate the StAX events.
>>
>> All this is completely transparent for the interceptors further down the
>> chain.
>
> Hmm.. that's kind of cool. More or less what Axiom should have been. ;-)
>
>
>> Of course, this is impossible to achieve using a standard SAAJ
>> implementation. Therefore the interceptor uses an alternative SAAJ
>> implementation which I have written as part of a larger project called
>> DDOM [1].
>
> And I think this is where the issues may start popping up, but definitely
> resolvable. If you look at the CXF survey, one of the MAJOR "areas of
> improvement" for CXF was too many external dependencies. I'm kind of
> reluctant to add more "required" deps. Thus, the security stuff would need
> to work with the default SAAJ impl (in particular, the one built into the
> JDK), but I could definitely see making something like this as an optional
> "performance enhancment" type thing. Ideally, the existing SAAJ interceptor
> would be smart enough to checkt he SAAJ impl being used and do the smart thing
> depending on what it sees.
What about the following solution:
* Introduce a new SAAJManager interface that encapsulates the SAAJ
handling stuff.
* Create a default implementation based on the code currently
contained in the standard SAAJ interceptors.
* Refactor SAAJInInterceptor and SAAJOutInterceptor to look up the
SAAJManager using Bus#getExtension. If none is found, fall back to the
default implementation.
* Maybe also refactor the WSS4J interceptors to use SAAJManager
instead of instantiating SAAJ(In|Out)Interceptor.
* My ddom-cxf stuff would then provide an alternative SAAJManager
implementation and automatically register that as a bus extension.
> If the DDOM stuff we need can be squashed/shaded
> into a single jar that is just a drop in replacement of saaj-impl, that
> concern could also be aleviated quite a bit as well.
Yes, I was also thinking the same. There are a couple of other reasons
why this is a good idea:
* The DDOM project is split into a multitude of smaller modules, so
that for every use case, you only get the stuff that is really needed
(e.g. some modules depend on the Axiom APIs, but you don't want that
as a transitive dependency for the CXF stuff ;-). This works well when
Maven is used, but of course it's a pain for users who use other
tools.
* The code in the CXF stuff is (currently) tightly coupled to some
internal DDOM APIs that are likely to change over time. Thus,
packaging all the required stuff into a single JAR and shade it makes
sense because it avoids conflicts caused by incompatible versions of
DDOM modules.
> That said, the callback method added to WSS4J has an additional use case that
> Sergey and I have been noodling on outside of SOAP. One thing we're trying
> to figure out how to do is leverage some of the WS-SecPol support to provide
> message level security for XML REST payloads. With the callback, we MAY be
> able to have separate XML sources for the security header and the actual
> payload (think separate MIME containers) where we feed the security header
> into WSS4J, and it calls back out to us to load the other parts as/if needed.
> Very early in the "noodling in our heads" stages. :-)
The DDOM solution here would simply be to compose the equivalent SOAP
message and pass that to WSS4J. This would mean creating a skeleton
SOAP message and adding the source objects for the header and body
parts to that skeleton. It is definitely possible to defer the
expansion of these parts (and even the lookup of the corresponding
MIME parts from the message) until the very last moment. This is
conceptually similar to your solution except that the callbacks are
invoked by the DOM/SAAJ implementation instead of WSS4J. This could
have some advantages:
* The scope of WSS4J would remain strictly limited to WS-Security and
no API changes are needed.
* If I understand your idea correctly, then conceptually you are
defining a mapping between REST style requests/responses and SOAP
messages conforming to WS-Security. With DDOM, the code that
implements this would simply be a direct transcription of these
mapping rules (plus of course some glue code).
>> The implementation is still incomplete, but to give you an
>> idea about its current status:
>>
>> * The underlying DOM implementation successfully passes the W3C DOM1
>> and DOM2 test suites, except for features not relevant to SOAP
>> processing (such as DTD information, entity references, etc.).
>
> That's good. DTD handling with Stax really sucks. Good luck with that.
> ;-)
It's definitely not possible to implement all JAXP/DOM features by
relying on a standard StAX implementation as the underlying parser. If
there is a requirement for full JAXP/DOM support, then probably the
better option would be to use Xerces XNI (which also has some kind of
pull parsing support).
>> It also
>> successfully passes the most important parts of the DOM3 test suite.
>> * When configured as SAAJ implementation in the CXF build (without the
>> custom SAAJInInterceptor), it passes all unit tests up to the systests
>> (It seems that this is primarily due to the fact that some of the
>> systests use SAAJ to compose test messages).
>
> Likely yes. Do you have a feeling as to whether those are issues in the CXF
> tests or issues with DDOM?
These are issues in DDOM. They are simply due to the fact that many
SAAJ methods in DDOM are still unimplemented ;-). What I wanted to say
here is that there is enough stuff in DDOM's SAAJ implementation to
play nicely with CXF's own code. Of course, once the SOAPMessage is
exposed to user code through a JAX-WS handler or provider endpoint,
this is no longer enough. As I said, for the moment it's a PoC.
> I see in the code that it doesn't support attachments either. That would be
> required for the JAX-WS TCK. (and likely for some of the systests as well)
> That said, those should be fairly easy to handle since you have the
> SoapMessage. The DDOM version could be MUCH more efficient by not
> buffering the attachments unless they really are accessed. One of my
> "issues" with the JAX-WS handlers is they require us to stop streaming
> EVERYTHING and buffer it, even if the only thing the handler does is:
> System.out.println("Hello World!");
> this could be a huge help with that.
The SAAJ stuff in DDOM is actually split into two parts:
* The part that implements the SAAJ object model. This contains the
code that extends DOM and also some partial implementations of
SOAPMessage and SOAPPart. On the other hand, it doesn't contain any
MIME parsing or attachment management logic.
* The part that provides the SAAJ factories and thus the complete SAAJ
implementation. Of course this part depends on the first part and will
eventually contain the MIME parsing logic to satisfy the requirements
of the SAAJ spec.
When running the CXF test suites against DDOM I use both parts as a
drop-in replacement for Sun's saaj-impl. On the other hand, the custom
SAAJInInterceptor only uses the first part. This means that the
SOAPMessage implementation is not the same as the one that would be
created by MessageFactory. Instead it uses an implementation that is
basically a wrapper around a org.apache.cxf.binding.soap.SoapMessage
object. This can be seen here:
http://ddom.googlecode.com/svn/trunk/modules/ddom-cxf/src/main/java/com/googlecode/ddom/cxf/SOAPMessageImpl.java
The idea is that the methods that are used to manage attachments (and
that are currently unimplemented) would all delegate to the
corresponding CXF APIs. This means that all optimizations done by CXF
would also work when accessing the attachments through SAAJ. Since
SAAJ exposes attachments via iterators, the only case were all
attachments are buffered would be a call to countAttachments...
BTW, I'm interested in references to test cases, issue reports or
mailing list discussions that describe use cases that could be
optimized using this custom SAAJ implementation.
>> * When configured as DOM implementation in the WSS4J build, it passes
>> all unit tests except some of the tests related to SAML (because
>> OpenSAML expects to get schema support from the DOM implementation
>> available though JAXP).
>> * I tested the implementation successfully with the scenarios
>> developed by Dennis Sosnoski [2] (both by simply configuring DDOM as
>> SAAJ implementation as well as using my alternative
>> SAAJInInterceptor).
>>
>> The code for the custom SAAJInInterceptor is available here:
>>
>> http://ddom.googlecode.com/svn/trunk/modules/ddom-cxf/
>>
>> Note that besides the limitations mentioned above, there are currently
>> some features missing in this SAAJInInterceptor implementation that
>> needs to be mentioned, namely processing of SOAP faults and the
>> ability to access attachments through the SAAJ API.
>
> Ah. Yep. Didn't read far enough. :-)
>
>
>
>> Some preconfigured CXF (2.3.0) clients and services (containing code
>> from Dennis' article) are available here:
>>
>> http://ddom.googlecode.com/svn/trunk/perftests/
>>
>> Note that for the moment, they still use Sun's SAAJ implementation for
>> outgoing messages. Therefore the interesting things happen in the
>> response processing on the client side.
>>
>> I did some initial performance tests with the scenarios developed by
>> Dennis and the results suggest that my SAAJ implementation performs
>> slightly better than the one from Sun, resulting in a few percent
>> points improvement in the "sign" and "signencr" scenarios. Note that
>> the features of the custom SAAJInInterceptor are not relevant here
>> because the body is accessed anyway in these scenarios. I think that
>> the improvement primarily results from a reduced memory footprint.
>> Maybe Dennis could volunteer as an "independent" performance tester to
>> confirm this?
>>
>> BTW, while doing some profiling, I noticed that in the "signencr"
>> scenario, CXF suffers from a flaw in Santuario's code, namely
>> org.apache.xml.security.encryption.XMLCipher.Serializer#deserialize,
>> which does some highly inefficient stuff. I think that by doing that
>> part of the processing in a smarter way, it may be possible to perform
>> better than Metro in the small-response case and to increase the gap
>> with respect to Metro in the large-response case.
>
> Yea. Colm is aware of a few places in both WSS4J and Santuario that could use
> some work. He hasn't had time to work on it yet, but, if all goes well,
> he'll have some time soon. ;-)
>
>
> Dan
>
>> Andreas
>>
>> PS: As noted between the lines by Daniel, optimizing the security
>> processing for outgoing messages is much more difficult, at least in
>> CXF. In principle, the Axis2 architecture would allow to achieve this
>> more easily, but before this can be leveraged, there are some other
>> performance issues in Axis2/Rampart that need to be fixed...
>>
>> [1] http://code.google.com/p/ddom/
>> [2] http://www.ibm.com/developerworks/java/library/j-jws14/index.html
>
> --
> Daniel Kulp
> dkulp@apache.org
> http://dankulp.com/blog
>
Re: Security token processing without SAAJ dependency
Posted by Daniel Kulp <dk...@apache.org>.
On Friday 08 October 2010 7:23:55 am Andreas Veithen wrote:
> The other alternative is to leave WSS4J unchanged and to let the SAAJ
> implementation materialize the different parts of the object model on
> demand. I have a working PoC that does this. It currently implements
> an alternative SAAJInInterceptor that provides the following features:
>
> * It always puts a SOAPMessage into the Message object, but the
> SOAPPart (including the SOAP headers) is only instantiated on demand
> when the SOAPMessage is accessed.
> * The content of the SOAPBody is only materialized when is it actually
> accessed. In addition, deferred parsing and building of the nodes is
> supported for this part of the message. This means that WSS4J can
> start processing the body of the message before it has been received
> entirely.
> * If the body has not been accessed, the StAX events are passed
> through unchanged (e.g. to the interceptor that does the databinding
> stuff). If the body has been accessed through the SAAJ API, then a
> W3CDOMStreamReader is used to regenerate the StAX events.
>
> All this is completely transparent for the interceptors further down the
> chain.
Hmm.. that's kind of cool. More or less what Axiom should have been. ;-)
> Of course, this is impossible to achieve using a standard SAAJ
> implementation. Therefore the interceptor uses an alternative SAAJ
> implementation which I have written as part of a larger project called
> DDOM [1].
And I think this is where the issues may start popping up, but definitely
resolvable. If you look at the CXF survey, one of the MAJOR "areas of
improvement" for CXF was too many external dependencies. I'm kind of
reluctant to add more "required" deps. Thus, the security stuff would need
to work with the default SAAJ impl (in particular, the one built into the
JDK), but I could definitely see making something like this as an optional
"performance enhancment" type thing. Ideally, the existing SAAJ interceptor
would be smart enough to checkt he SAAJ impl being used and do the smart thing
depending on what it sees. If the DDOM stuff we need can be squashed/shaded
into a single jar that is just a drop in replacement of saaj-impl, that
concern could also be aleviated quite a bit as well.
That said, the callback method added to WSS4J has an additional use case that
Sergey and I have been noodling on outside of SOAP. One thing we're trying
to figure out how to do is leverage some of the WS-SecPol support to provide
message level security for XML REST payloads. With the callback, we MAY be
able to have separate XML sources for the security header and the actual
payload (think separate MIME containers) where we feed the security header
into WSS4J, and it calls back out to us to load the other parts as/if needed.
Very early in the "noodling in our heads" stages. :-)
> The implementation is still incomplete, but to give you an
> idea about its current status:
>
> * The underlying DOM implementation successfully passes the W3C DOM1
> and DOM2 test suites, except for features not relevant to SOAP
> processing (such as DTD information, entity references, etc.).
That's good. DTD handling with Stax really sucks. Good luck with that.
;-)
> It also
> successfully passes the most important parts of the DOM3 test suite.
> * When configured as SAAJ implementation in the CXF build (without the
> custom SAAJInInterceptor), it passes all unit tests up to the systests
> (It seems that this is primarily due to the fact that some of the
> systests use SAAJ to compose test messages).
Likely yes. Do you have a feeling as to whether those are issues in the CXF
tests or issues with DDOM?
I see in the code that it doesn't support attachments either. That would be
required for the JAX-WS TCK. (and likely for some of the systests as well)
That said, those should be fairly easy to handle since you have the
SoapMessage. The DDOM version could be MUCH more efficient by not
buffering the attachments unless they really are accessed. One of my
"issues" with the JAX-WS handlers is they require us to stop streaming
EVERYTHING and buffer it, even if the only thing the handler does is:
System.out.println("Hello World!");
this could be a huge help with that.
> * When configured as DOM implementation in the WSS4J build, it passes
> all unit tests except some of the tests related to SAML (because
> OpenSAML expects to get schema support from the DOM implementation
> available though JAXP).
> * I tested the implementation successfully with the scenarios
> developed by Dennis Sosnoski [2] (both by simply configuring DDOM as
> SAAJ implementation as well as using my alternative
> SAAJInInterceptor).
>
> The code for the custom SAAJInInterceptor is available here:
>
> http://ddom.googlecode.com/svn/trunk/modules/ddom-cxf/
>
> Note that besides the limitations mentioned above, there are currently
> some features missing in this SAAJInInterceptor implementation that
> needs to be mentioned, namely processing of SOAP faults and the
> ability to access attachments through the SAAJ API.
Ah. Yep. Didn't read far enough. :-)
> Some preconfigured CXF (2.3.0) clients and services (containing code
> from Dennis' article) are available here:
>
> http://ddom.googlecode.com/svn/trunk/perftests/
>
> Note that for the moment, they still use Sun's SAAJ implementation for
> outgoing messages. Therefore the interesting things happen in the
> response processing on the client side.
>
> I did some initial performance tests with the scenarios developed by
> Dennis and the results suggest that my SAAJ implementation performs
> slightly better than the one from Sun, resulting in a few percent
> points improvement in the "sign" and "signencr" scenarios. Note that
> the features of the custom SAAJInInterceptor are not relevant here
> because the body is accessed anyway in these scenarios. I think that
> the improvement primarily results from a reduced memory footprint.
> Maybe Dennis could volunteer as an "independent" performance tester to
> confirm this?
>
> BTW, while doing some profiling, I noticed that in the "signencr"
> scenario, CXF suffers from a flaw in Santuario's code, namely
> org.apache.xml.security.encryption.XMLCipher.Serializer#deserialize,
> which does some highly inefficient stuff. I think that by doing that
> part of the processing in a smarter way, it may be possible to perform
> better than Metro in the small-response case and to increase the gap
> with respect to Metro in the large-response case.
Yea. Colm is aware of a few places in both WSS4J and Santuario that could use
some work. He hasn't had time to work on it yet, but, if all goes well,
he'll have some time soon. ;-)
Dan
> Andreas
>
> PS: As noted between the lines by Daniel, optimizing the security
> processing for outgoing messages is much more difficult, at least in
> CXF. In principle, the Axis2 architecture would allow to achieve this
> more easily, but before this can be leveraged, there are some other
> performance issues in Axis2/Rampart that need to be fixed...
>
> [1] http://code.google.com/p/ddom/
> [2] http://www.ibm.com/developerworks/java/library/j-jws14/index.html
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog