[jira] [Comment Edited] (WSS-430) Support for secured SOAP attachments

["Marc Giger (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/WSS-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629431#comment-13629431 ] 

Marc Giger edited comment on WSS-430 at 4/11/13 9:40 PM:
---------------------------------------------------------

Hi Nathan,

Please, excuse the delay...

>With regards to buffering the attachments, I see from JIRA that you guys have done some awesome work to support   
>streaming in WSS4J. This will help us a lot in the future because the messages we're looking at could be quite large and not 
>having to have a DOM representation would be great. It would also be great if there was a way to stream the attachment 
>content for signature/encryption so that it didn't have to be buffered in memory.

Just to be clear here and to prevent misunderstandings: On the outbound side (signature-creation and encryption) we can't really stream anything. Neither the primary SOAP Document nor the attachments. The StAX implementation will not change that. Simple reason: The the references and digest values are always stored in the Security-Header. That means we have to process and buffer the whole document to calculate the digest first and then build the soap-header.
In contrary, on the inbound-side real streaming (with the StAX impl.) is possible.
If attachments are fully streamable (from network) depends to some degree also how the SOAP-Stack handles it.

>I've had a go at implementing the canonicalisation of attachments. I ended up doing it in my custom URIDereferencer when 
>the attachment content type is text/xml, application/xml or application/soap+xml. The dereferencer parses the XML into a 
>DOM (since I'm working with 1.6.9) and returns an ApacheNodeSetData instance. The addReferencesToSign() adds the 
>exclusive canonicalisation transform to the transforms list.

As far as I understood the specification the c14n transformation is not explicit added to the reference-transforms but is done implicit when the attachment is of type xml.
And the c14n is your next "problem" regarding memory. Because c14n has to be done for XML attachments you have to
read the attachment fully into memory, do c14n and calculate the digest. So there is no memory saving. The outcome is
more or less the same as when the xml-attachment would be in the SOAP-Body. Do you agree?
A solution to this could be if you transport the xml attachment as application/octet-stream. Then no c14n and such should/will be done.

>I am aware of the attachment complete transform. The Axiom API doesn't seem to provide me with enough MIME header 
>values to implement this, so I'm hoping I can get away with just using the attachment content transform :) If that doesn't turn 
>out to be the case, I'll look at patching my local copy of Axiom to get the extra header values.

Hmm, Axiom? Yet I didn't look how axis handles attachments but for my understanding axiom has nothing to do with the attachments. Did you mean the Axis2-API?

Btw, how large are the XML documents in Megabytes? How many start elements? Do they have large text-nodes?
Is the XML structure highly repetitive? I have experience with 250MB signed and encrypted SOAP documents which needs
about 8 gigs of memory (DOM). Perhaps this information is useful for you...

Thanks,

Marc
                
      was (Author: giger):
    Hi Nathan,

Please, excuse the delay...

>With regards to buffering the attachments, I see from JIRA that you guys have done some awesome work to support   >streaming in WSS4J. This will help us a lot in the future because the messages we're looking at could be quite large and not >having to have a DOM representation would be great. It would also be great if there was a way to stream the attachment >content for signature/encryption so that it didn't have to be buffered in memory.

Just to be clear here and to prevent misunderstandings: On the outbound side (signature-creation and encryption) we can't really stream anything. Neither the primary SOAP Document nor the attachments. The StAX implementation will not change that. Simple reason: The the references and digest values are always stored in the Security-Header. That means we have to process and buffer the whole document to calculate the digest first and then build the soap-header.
In contrary, on the inbound-side real streaming (with the StAX impl.) is possible.
If attachments are fully streamable (from network) depends to some degree also how the SOAP-Stack handles it.

>I've had a go at implementing the canonicalisation of attachments. I ended up doing it in my custom URIDereferencer when >the attachment content type is text/xml, application/xml or application/soap+xml. The dereferencer parses the XML into a >DOM (since I'm working with 1.6.9) and returns an ApacheNodeSetData instance. The addReferencesToSign() adds the >exclusive canonicalisation transform to the transforms list.

As far as I understood the specification the c14n transformation is not explicit added to the reference-transforms but is done implicit when the attachment is of type xml.
And the c14n is your next "problem" regarding memory. Because c14n has to be done for XML attachments you have to
read the attachment fully into memory, do c14n and calculate the digest. So there is no memory saving. The outcome is
more or less the same as when the xml-attachment would be in the SOAP-Body. Do you agree?
A solution to this could be if you transport the xml attachment as application/octet-stream. Then no c14n and such should/will be done.

>I am aware of the attachment complete transform. The Axiom API doesn't seem to provide me with enough MIME header >values to implement this, so I'm hoping I can get away with just using the attachment content transform :) If that doesn't turn >out to be the case, I'll look at patching my local copy of Axiom to get the extra header values.

Hmm, Axiom? Yet I didn't look how axis handles attachments but for my understanding axiom has nothing to do with the attachments. Did you mean the Axis2-API?

Btw, how large are the XML documents in Megabytes? How many start elements? Do they have large text-nodes?
Is the XML structure highly repetitive? I have experience with 250MB signed and encrypted SOAP documents which needs
about 8 gigs of memory (DOM). Perhaps this information is useful for you...

Thanks,

Marc
                  
> Support for secured SOAP attachments
> ------------------------------------
>
>                 Key: WSS-430
>                 URL: https://issues.apache.org/jira/browse/WSS-430
>             Project: WSS4J
>          Issue Type: New Feature
>          Components: WSS4J Core
>            Reporter: Marc Giger
>            Assignee: Marc Giger
>            Priority: Minor
>         Attachments: santuario-swa.diff, wss4j-swa.diff
>
>
> The attached patches should serve as a basis for discussions how 
> the support for SwA in WSS4j and the integration in
> a SOAP-Stack should look like.
> Some notes to the patch:
> - Applies to the current trunk of santuario and wss4j.
> - The client side demonstrates the DOM approach whereas the server side uses the StAX implementation.
> - I've implemented the very basic just to have a working proof-of-concept.
> - Attachments are requested via callback from the soap-stack because 
>       - of decoupling from soap-stack
>       - to support full streaming from network to SIB as far as possible
> - CXF dependencies are just a leftover from V1 and because the SecurityInInterceptor is not ported to V2
> - Encryption / Decryption of an attachment is streaming oriented, no buffering is done in WSS4J.
> - Signature creation and verification is at the moment buffered in WSS4j but the signature-verification can, under some conditions, be streamed as well.
> - To prevent patching of CXF for the prototype the WSS4J Interceptors and some dependencies are copied and modified and also included in the patch. The santuatio changes are necessary for the StAX impl.
>   For DOM all necessary santuario changes are done via reflection or other hacks for now.
> Feedback is very welcome and also necessary!
> Thanks,
> Marc

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ws.apache.org
For additional commands, e-mail: dev-help@ws.apache.org