I/O Changes [was svn commit: r546656 ...]

[Dan Diephouse <da...@envoisolutions.com>]

Hi all,

Just wanted to give an explicit "heads up" on this commit as it changes the
IO layer. I was doing some performance optimizations with JAXB so it would
write to the OutputStream instead of the XMLStreamWriter when possible
(which is a lot faster). Along the way I noticed that much of the code was
dependent on flush() being called just once and having a CachedOutputStream
around [1] - which are both bad assumptions. See my commit message for more
info.

I fixed most of this, however there seem to be two outstanding issues:
1. FailoverTest.testDefaultSequentialStrategy seems to be failing on Linux
(or randomly - CruiseControl is telling me its failing, but it doesn't on my
machine)
2. HTTPConduitTest.testHttpsRedirect401Response is failing - to be honest I
haven't explored this one as in depth as I could have, but I was wondering
if someone with more expertise on what is supposed to happen in the HTTP
layer could take a look.

I will spend more time on these tomorrow, but I felt it was very important
to make these changes and let people review them for a day or two as we're
quite close to a release.

- Dan

1. Transports might not supply a cachedOutputStream. Also, there could be
interceptors which do I/O transformation like GZip ones, which would break
all the code which was there before.


---------- Forwarded message ----------
From: dandiep@apache.org <da...@apache.org>
Date: Jun 12, 2007 6:49 PM
Subject: svn commit: r546656 [1/2] - in /incubator/cxf/trunk:
api/src/main/java/org/apache/cxf/io/
api/src/main/java/org/apache/cxf/message/
api/src/main/java/org/apache/cxf/phase/
common/common/src/main/java/org/apache/cxf/helpers/
rt/bindings/http/src/main/ja...
To: cxf-commits@incubator.apache.org

Author: dandiep
Date: Tue Jun 12 15:49:08 2007
New Revision: 546656

URL: http://svn.apache.org/viewvc?view=rev&rev=546656
Log:

Refactoring of the low level IO layer inside CXF.

Remove the dependence on flush() being called ONCE and only once
inside CXF. There was the assumption in all the code that things would
be cached until we actually finished databinding, which is a bad assumption.
There are a couple problems with this:
1. The Soap envelope writer was writing the envelope twice if a fault was
thrown during databinding and the stream was already flushed. (i.e. during
validation)
2. The code didn't differentiate between the type of caching that was going
on. i.e. are we a) writing to the underlying stream AND caching the message
at the same time. Or b) are we caching the message and then writing on
close()

Ensure that the RM and Logging layers correctly do their work on close()
instead of flush().

Transports now pass in the underlying stream. The RM layer and logging
interceptors correct detect when a cached stream is not being used
and act accordingly. This is much more robust as I/O transformations
are a very valid use case an many people will want to switch the
underlying streams. We also shouldn't force transport writers to
use CachedOutputStream.

Rename AbstractCachedOutputStream to CachedOutputStream and make
it a non-abstract class. This cuts down on the number of CachedOutputStream
classes laying around! There are now two extended implementations of
COS as well:
1. WriteOnCloseOutputStream. This caches the message until the outputstream
is closed. This is needed for the RM scenarios where we are caching the
message
until the CreateSequence is done. We don't want to open the connection until
all of this is done. (I think we could probably reset this output stream
as soon as the createsequence is finished though)
2. CacheAndWriteOutputStream. This caches the message while it is written
to the underlying stream at the same time. Useful for logging.

Update the DataBinding code to have JAXB write directly to the OutputStream
whenever possible. This gives a great performance improvement for some
scenarios.

Refactor the JAXB code so we don't have so many data readers/writers.

TODO: Make RM and Logging work when the message is larger than memory or the
CachedOutputStream writes to disk.

TODO: Fix HTTPS 401 Redirect test


-- 
Dan Diephouse
Envoi Solutions
http://envoisolutions.com | http://netzooid.com/blog

Re: I/O Changes [was svn commit: r546656 ...]

[Dan Diephouse <da...@envoisolutions.com>]

I forgot to ask: can someone on linux run the FailoverTest on their machine
and see what happens? I'm wondering if its random threading problems or a
bonafide windows/linux issue. A server log (turned up to FINE) would be
greatly appreciated as well if you have time.

(Or if you're particularly motivated you could fix it as well ;-) )

Thanks,
- Dan

On 6/12/07, Dan Diephouse <da...@envoisolutions.com> wrote:
>
> Hi all,
>
> Just wanted to give an explicit "heads up" on this commit as it changes
> the IO layer. I was doing some performance optimizations with JAXB so it
> would write to the OutputStream instead of the XMLStreamWriter when possible
> (which is a lot faster). Along the way I noticed that much of the code was
> dependent on flush() being called just once and having a CachedOutputStream
> around [1] - which are both bad assumptions. See my commit message for more
> info.
>
> I fixed most of this, however there seem to be two outstanding issues:
> 1. FailoverTest.testDefaultSequentialStrategy seems to be failing on Linux
> (or randomly - CruiseControl is telling me its failing, but it doesn't on my
> machine)
> 2. HTTPConduitTest.testHttpsRedirect401Response is failing - to be honest
> I haven't explored this one as in depth as I could have, but I was wondering
> if someone with more expertise on what is supposed to happen in the HTTP
> layer could take a look.
>
> I will spend more time on these tomorrow, but I felt it was very important
> to make these changes and let people review them for a day or two as we're
> quite close to a release.
>
> - Dan
>
> 1. Transports might not supply a cachedOutputStream. Also, there could be
> interceptors which do I/O transformation like GZip ones, which would break
> all the code which was there before.
>
>
> ---------- Forwarded message ----------
> From: dandiep@apache.org <dandiep@apache.org >
> Date: Jun 12, 2007 6:49 PM
> Subject: svn commit: r546656 [1/2] - in /incubator/cxf/trunk:
> api/src/main/java/org/apache/cxf/io/
> api/src/main/java/org/apache/cxf/message/
> api/src/main/java/org/apache/cxf/phase/
> common/common/src/main/java/org/apache/cxf/helpers/
> rt/bindings/http/src/main/ja...
> To: cxf-commits@incubator.apache.org
>
> Author: dandiep
> Date: Tue Jun 12 15:49:08 2007
> New Revision: 546656
>
> URL: http://svn.apache.org/viewvc?view=rev&rev=546656
> Log:
>
> Refactoring of the low level IO layer inside CXF.
>
> Remove the dependence on flush() being called ONCE and only once
> inside CXF. There was the assumption in all the code that things would
> be cached until we actually finished databinding, which is a bad
> assumption.
> There are a couple problems with this:
> 1. The Soap envelope writer was writing the envelope twice if a fault was
> thrown during databinding and the stream was already flushed. ( i.e.
> during
> validation)
> 2. The code didn't differentiate between the type of caching that was
> going
> on. i.e. are we a) writing to the underlying stream AND caching the
> message
> at the same time. Or b) are we caching the message and then writing on
> close()
>
> Ensure that the RM and Logging layers correctly do their work on close()
> instead of flush().
>
> Transports now pass in the underlying stream. The RM layer and logging
> interceptors correct detect when a cached stream is not being used
> and act accordingly. This is much more robust as I/O transformations
> are a very valid use case an many people will want to switch the
> underlying streams. We also shouldn't force transport writers to
> use CachedOutputStream.
>
> Rename AbstractCachedOutputStream to CachedOutputStream and make
> it a non-abstract class. This cuts down on the number of
> CachedOutputStream
> classes laying around! There are now two extended implementations of
> COS as well:
> 1. WriteOnCloseOutputStream. This caches the message until the
> outputstream
> is closed. This is needed for the RM scenarios where we are caching the
> message
> until the CreateSequence is done. We don't want to open the connection
> until
> all of this is done. (I think we could probably reset this output stream
> as soon as the createsequence is finished though)
> 2. CacheAndWriteOutputStream. This caches the message while it is written
> to the underlying stream at the same time. Useful for logging.
>
> Update the DataBinding code to have JAXB write directly to the
> OutputStream
> whenever possible. This gives a great performance improvement for some
> scenarios.
>
> Refactor the JAXB code so we don't have so many data readers/writers.
>
> TODO: Make RM and Logging work when the message is larger than memory or
> the
> CachedOutputStream writes to disk.
>
> TODO: Fix HTTPS 401 Redirect test
>
>
> --
> Dan Diephouse
> Envoi Solutions
> http://envoisolutions.com | http://netzooid.com/blog




-- 
Dan Diephouse
Envoi Solutions
http://envoisolutions.com | http://netzooid.com/blog