You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by kongar <ak...@5amsolutions.com> on 2012/10/24 20:53:02 UTC
WSS4J Timestamp expiring before large file transfer finishes
Hullo,
I am trying to stream large files over a secured connection, and I am
finding that if the transfer takes a certain amount of time (due to the file
size or slow connection), I get the following exception:
[#|2012-10-24T10:38:06.265-0400|WARNING|glassfish3.1.2|org.apache.cxf.ws.security.wss4j.WSS4JInInterceptor|_ThreadID=111;_ThreadName=Thread-2;|org.apache.ws.security.WSSecurityException:
The message has expired (WSSecurityEngine: Invalid timestamp The security
semantics of the message have expired)
at
org.apache.ws.security.validate.TimestampValidator.validate(TimestampValidator.java:62)
at
org.apache.ws.security.processor.TimestampProcessor.handleToken(TimestampProcessor.java:62)
at
org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:397)
This occurs AFTER the file has been streamed successfully into the
CachedOutputStream.OutputDirectory. Is this the correct behavior?
Shouldn't timestamp validation occur before the file is streamed as the
transfer could take a while?
I am planning to alleviate this problem by increasing the <Expires> date in
the timestamp, but I'm wondering if there's a better way of solving this
problem...
In addition, I noticed that often when the connection fails during a large
file transfer (such as the one above), the temp files were never deleted and
trying to delete it manually fails due to it being locked.
Thanks for any help you can give.
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348.html
Sent from the cxf-user mailing list archive at Nabble.com.
RE: WSS4J Timestamp expiring before large file transfer finishes
Posted by kongar <ak...@5amsolutions.com>.
Hi Karthik,
invocationTime is the time when the message is received by the server (i.e.
the time according to the server/receiver)
credential.getTimestamp() is the timestamp that is found on the message
itself (i.e. the time according to the client/sender).
Those are the timestamps that are being compared to ensure that the client's
timestamp has not expired.
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5724028.html
Sent from the cxf-user mailing list archive at Nabble.com.
RE: WSS4J Timestamp expiring before large file transfer finishes
Posted by ucrkarthik <uc...@hotmail.com>.
Hi kongar,
Thanks for your help. It was very helpful.
I just have quick question...
In TimestampValidator.java lines 91 and 92.. whats the difference between
the credential.getTimestamp().getCreated() vs the invocationTime timestamp?
Thanks
Karthik
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5723947.html
Sent from the cxf-user mailing list archive at Nabble.com.
RE: WSS4J Timestamp expiring before large file transfer finishes
Posted by kongar <ak...@5amsolutions.com>.
Hi Karthik,
My workaround was the same as I posted above. In short, it was:
1. Create an interceptor that will save the time when the message is first
received.
2. Override the default timestamp validator with a custom one. This custom
one will use the time saved for comparison with the message timestamp rather
than using the current time.
3. Register both the interceptor and the timestamp in cxf-servlet.xml
Here is the code that I wrote for the interceptor:
https://github.com/CONNECT-Solution/CONNECT/blob/4.0.0/Product/Production/Common/CONNECTCoreLib/src/main/java/gov/hhs/fha/nhinc/callback/cxf/largefile/TimestampInterceptor.java
For the timestamp validator:
https://github.com/CONNECT-Solution/CONNECT/blob/4.0.0/Product/Production/Common/CONNECTCoreLib/src/main/java/gov/hhs/fha/nhinc/callback/cxf/largefile/TimestampValidator.java
And registering them in cxf:
https://github.com/CONNECT-Solution/CONNECT/blob/CONNECT_integration/Product/Production/Gateway/DocumentSubmission_20/src/main/resources/docsubmission/_20/webservices.xml
Hopefully, that will get you going.
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5723814.html
Sent from the cxf-user mailing list archive at Nabble.com.
RE: WSS4J Timestamp expiring before large file transfer finishes
Posted by ucrkarthik <uc...@hotmail.com>.
Hi Kongar,
Would you be able to share your TimeStampValidator workaround? I am running
into the same issue.
Thanks,
Karthik
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5723426.html
Sent from the cxf-user mailing list archive at Nabble.com.
RE: WSS4J Timestamp expiring before large file transfer finishes
Posted by kongar <ak...@5amsolutions.com>.
Thanks for validating my approach and for the issue number. I think I'm able
to work around that issue now.
One more thing that I do want to point out though. It does look like CXF is
not properly closing the input stream when a soap fault occurs during a
secured streaming transfer of a large file. The timestamp validation errors
that I've been getting was leaving locked tmp files in the
CachedOutputStream's tmp directory. I had to create a FaultOutInterceptor
with a SETUP_ENDING Phase that goes through all the attachment data source
of the message and release/close the input stream. I believe this problem
occurs if you get any sort of exceptions thrown after the file is streamed
in inside the handleMessage() method of WSS4JInInterceptor. This is with
2.6.0 so I'm not sure if this is fixed in the later releases.
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717495.html
Sent from the cxf-user mailing list archive at Nabble.com.
RE: WSS4J Timestamp expiring before large file transfer finishes
Posted by Andrei Shakirin <as...@talend.com>.
>So will the stream based WSS4J solve this issue? If so, is there a ticket for it that I can track?
Yes, stream based WSS4J will not load the whole message if it is not needed.
You can track https://issues.apache.org/jira/browse/WSS-359 in WSS4J. It is planned for WSS4J 2.0.
>As a workaround, I'm trying to implement my own TimestampValidator which will compare the timestamp with the time that the service is first invoked (i.e.
>i'll create my own interceptor which will be at the top of the chain and stash the current time on the MessageContext for the modified TimestampValidator to
>use). Is this a good approach? Is there any other way of obtaining the time that a cxf web service is first called?
For me it is a valid workaround. Perhaps Colm can suggest the best approach.
Andrei.
-----Original Message-----
From: kongar [mailto:akong@5amsolutions.com]
Sent: Donnerstag, 25. Oktober 2012 22:31
To: users@cxf.apache.org
Subject: Re: WSS4J Timestamp expiring before large file transfer finishes
Thanks Andrei and Dan for the replies. Definitely clears things up for me.
So will the stream based WSS4J solve this issue? If so, is there a ticket for it that I can track? As it currently stands, it seems like CXF secured streaming will only work if the download time is less than what is configured by ws-security.timestamp.timeToLive (which is defaulted to 5 minutes).
As a workaround, I'm trying to implement my own TimestampValidator which will compare the timestamp with the time that the service is first invoked (i.e. i'll create my own interceptor which will be at the top of the chain and stash the current time on the MessageContext for the modified TimestampValidator to use). Is this a good approach? Is there any other way of obtaining the time that a cxf web service is first called?
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717461.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WSS4J Timestamp expiring before large file transfer finishes
Posted by kongar <ak...@5amsolutions.com>.
Thanks Andrei and Dan for the replies. Definitely clears things up for me.
So will the stream based WSS4J solve this issue? If so, is there a ticket
for it that I can track? As it currently stands, it seems like CXF secured
streaming will only work if the download time is less than what is
configured by ws-security.timestamp.timeToLive (which is defaulted to 5
minutes).
As a workaround, I'm trying to implement my own TimestampValidator which
will compare the timestamp with the time that the service is first invoked
(i.e. i'll create my own interceptor which will be at the top of the chain
and stash the current time on the MessageContext for the modified
TimestampValidator to use). Is this a good approach? Is there any other
way of obtaining the time that a cxf web service is first called?
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717461.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WSS4J Timestamp expiring before large file transfer finishes
Posted by Daniel Kulp <dk...@apache.org>.
On Oct 25, 2012, at 5:10 AM, Andrei Shakirin <as...@talend.com> wrote:
> Actually WSS4J works only with DOM based SOAPMessages. Therefore CXF WSS security interceptors always invoke SAAJ interceptors before process the message. That means the interceptors break streaming.
Yep. Building up the SAAJ model requires all the attachments to be available and cached. Thus, we have to pull them all in. Otherwise the SOAPHandlers and the Provider<SOAPMessage> type models would not work properly per spec.
We could likely add a property that the SAAJInInterceptor could check to turn that off. In the case of WSS4J, it ignores all the attachments so we likely could optionally skip that. Even the SOAPHandlers and Providers that don't touch any of the attachments could likely skip it.
Dan
>
> AFAIK guys already started work on stream based WSS4J. Colm knows more about it.
>
> I am not sure if any essential optimization is possible with current WSS4J.
>
> Regards,
> Andrei.
>
> -----Original Message-----
> From: kongar [mailto:akong@5amsolutions.com]
> Sent: Donnerstag, 25. Oktober 2012 02:48
> To: users@cxf.apache.org
> Subject: Re: WSS4J Timestamp expiring before large file transfer finishes
>
> Ok, sorry for spamming this list as I keep replying to myself, but removing the LoggingInInterceptor didn't help as I had thought. This looks like it may be an actual problem in the CXF implementation of WSS4JInterceptor. On the handleMessage() method, this line:
>
> SOAPMessage doc = getSOAPMessage(msg);
>
> Downloads the entire attachments to disk. And only afterwards are the security headers processed with these lines:
>
> Element elem = WSSecurityUtil.getSecurityHeader(doc.getSOAPPart(), actor); List<WSSecurityEngineResult> wsResult = engine.processSecurityHeader(elem,
> reqData);
>
> Ideally, shouldn't the headers be validated before the attachment is downloaded?
>
>
>
>
> --
> View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717373.html
> Sent from the cxf-user mailing list archive at Nabble.com.
--
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com
RE: WSS4J Timestamp expiring before large file transfer finishes
Posted by Andrei Shakirin <as...@talend.com>.
Actually WSS4J works only with DOM based SOAPMessages. Therefore CXF WSS security interceptors always invoke SAAJ interceptors before process the message. That means the interceptors break streaming.
AFAIK guys already started work on stream based WSS4J. Colm knows more about it.
I am not sure if any essential optimization is possible with current WSS4J.
Regards,
Andrei.
-----Original Message-----
From: kongar [mailto:akong@5amsolutions.com]
Sent: Donnerstag, 25. Oktober 2012 02:48
To: users@cxf.apache.org
Subject: Re: WSS4J Timestamp expiring before large file transfer finishes
Ok, sorry for spamming this list as I keep replying to myself, but removing the LoggingInInterceptor didn't help as I had thought. This looks like it may be an actual problem in the CXF implementation of WSS4JInterceptor. On the handleMessage() method, this line:
SOAPMessage doc = getSOAPMessage(msg);
Downloads the entire attachments to disk. And only afterwards are the security headers processed with these lines:
Element elem = WSSecurityUtil.getSecurityHeader(doc.getSOAPPart(), actor); List<WSSecurityEngineResult> wsResult = engine.processSecurityHeader(elem,
reqData);
Ideally, shouldn't the headers be validated before the attachment is downloaded?
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717373.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WSS4J Timestamp expiring before large file transfer finishes
Posted by kongar <ak...@5amsolutions.com>.
Ok, sorry for spamming this list as I keep replying to myself, but removing
the LoggingInInterceptor didn't help as I had thought. This looks like it
may be an actual problem in the CXF implementation of WSS4JInterceptor. On
the handleMessage() method, this line:
SOAPMessage doc = getSOAPMessage(msg);
Downloads the entire attachments to disk. And only afterwards are the
security headers processed with these lines:
Element elem = WSSecurityUtil.getSecurityHeader(doc.getSOAPPart(), actor);
List<WSSecurityEngineResult> wsResult = engine.processSecurityHeader(elem,
reqData);
Ideally, shouldn't the headers be validated before the attachment is
downloaded?
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717373.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WSS4J Timestamp expiring before large file transfer finishes
Posted by kongar <ak...@5amsolutions.com>.
So I figured out what the problem was. I had http dump option enabled on my
application server which was causing the LoggingInInterceptor to be invoked
before the WSS4JInInterceptor. The logging interceptor was streaming in the
file right at the beginning of the chain, so by the time it was done, the
timestamp was expired. Glad that was just a configuration problem...
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717367.html
Sent from the cxf-user mailing list archive at Nabble.com.
Re: WSS4J Timestamp expiring before large file transfer finishes
Posted by kongar <ak...@5amsolutions.com>.
One correction I want to make. It might not be the <Expire> element but more
the <Created> element in the timestamp that is causing me problems. I
understand that the jax-ws properties ws-security.timestamp.timeToLive or
ws-security.timestamp.futureTimeToLive can help, but I would need to set
those to a very high number which I would prefer not to. If somehow I can
make the timestamp validation occur before the data is saved to the file
system, then I think I will be OK.
--
View this message in context: http://cxf.547215.n5.nabble.com/WSS4J-Timestamp-expiring-before-large-file-transfer-finishes-tp5717348p5717350.html
Sent from the cxf-user mailing list archive at Nabble.com.