You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by ext2 <xu...@tongtech.com> on 2011/04/27 05:40:06 UTC
Re: could cxf save larget attachemnt in temporary file?(A new doubt)
Thanks Daniel Kulp for your explain.
Because CXF maintain the attachments stream underlying, now I feel doubt
about how could I cleaning the temporary file.
There are two use case, as following:
1) at server side, receiving request with large attachment.
Could CXF engine clean the temporary files automatically, just after the
service send response?
2) at client side, receiving response with large attachment.
Now the CXF Engine cannot automatically clean temporary files, only the user
know the time when to clean.
Because the CXF may buffer the last attachment's stream to disk
automatically, the user may unaware of this conversion.
So does CXF providing some method(API),which I could use to clean the
temporary attachment file safely?
Thanks any suggestion;
-----Original -----
Sender: Daniel Kulp [mailto:dkulp@apache.org]
Date: 2011/4/26 10:35
Sender: users@cxf.apache.or
Subject: Re: could cxf save larget attachemnt in temporary file?
What Freeman describes below isn't the FULL story. Normally, even with a
single large attachment, nothing would be written to disk. Whenever
possible, CXF tries to stream things directly. In the case of a single
attachment, or the last attachment when using multiple attachments, CXF can
stream directly and thus it wouldn't normally create the files on disk.
However, when any of the cases that force the attachment to be fully
realized
occurs and breaks the streaming, it would automatically buffer them into the
files on disk like Freeman describes.
Dan
On Sunday 24 April 2011 11:36:01 PM Freeman Fang wrote:
> Hi,
>
> My comment inline
>
> On 2011-4-25, at 上午11:16, ext2 wrote:
> > Hi:
> > Sometimes, web service (both service and client) need to receive
> > very large attachment.
> >
> > 1) does cxf support save large attachment in temporary file for
> >
> > service to receive request , and client to receive response?
>
> Yeah,
> When cxf message exceed a threshold(64 k by default), it will save the
> memory message to temp file on hard disk(by default it save to the
> folder specified by a pre-set well-known java system properties
> java.io.tempdir + cxf-temp-random ).
>
> > 2)how could I configure the temporary directory and threshold?
>
> You can specify the threshold with property
> org.apache.cxf.io.CachedOutputStream.Threshold (64k by default) t
> and specify the cxf temp dir for big size message with property
> org.apache.cxf.io.CachedOutputStream.OutputDirectory
> to a folder you have full acesss permission.
> Freeman
>
>
>
>
>
>
> ---------------------------------------------
> Freeman Fang
>
> FuseSource
> Email:ffang@fusesource.com
> Web: fusesource.com
> Twitter: freemanfang
> Blog: http://freemanfang.blogspot.com
> Connect at CamelOne May 24-26
> The Open Source Integration Conference
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog
Talend - http://www.talend.com
Re: could cxf save larget attachemnt in temporary file?(A new doubt)
Posted by ext2 <xu...@tongtech.com>.
It's great.
Thanks Daniel Kulp. Thank you very much;
> -----original -----
>Sender: Daniel Kulp [mailto:dkulp@apache.org]
>Date: 2010/4/28 2:25
> Receiver: users@cxf.apache.org
> Subject: Re: could cxf save larget attachemnt in temporary file?(A new
doubt)
>
> On Tuesday, April 26, 2011 11:40:06 PM ext2 wrote:
> > Thanks Daniel Kulp for your explain.
> >
> > Because CXF maintain the attachments stream underlying, now I feel
doubt
> > about how could I cleaning the temporary file.
> >
> > There are two use case, as following:
> >
> > 1) at server side, receiving request with large attachment.
> > Could CXF engine clean the temporary files automatically, just after the
> > service send response?
>
> Well, yes and no. When STREAMING attachments into the service, when the
> stream is closed, the attachment file gets deleted. Thus, if you are
seeing
> a bunch of stuff left on disk, it's likely that your code isn't properly
> closing the input streams.
>
>
> > 2) at client side, receiving response with large attachment.
> > Now the CXF Engine cannot automatically clean temporary files, only the
> > user know the time when to clean.
>
> Same as above. The inputstreams we provide to the datahandler/datasource
> things override the close() method to track things and will remove the
temp
> files automatically.
>
> > Because the CXF may buffer the last attachment's stream to disk
> > automatically, the user may unaware of this conversion.
> > So does CXF providing some method(API),which I could use to clean the
> > temporary attachment file safely?
>
> Just close the inputstreams. Note: this is ALSO important for the non-
> buffered attachments (like the last one) as the call to close will let us
know
> to close the underlying HTTP input stream and the URL connect can be
returned
> to the keep-alive pool and such.
>
> Dan
>
>
>
> >
> > Thanks any suggestion;
> >
> >
> > -----Original -----
> > Sender: Daniel Kulp [mailto:dkulp@apache.org]
> > Date: 2011/4/26 10:35
> > Sender: users@cxf.apache.or
> > Subject: Re: could cxf save larget attachemnt in temporary file?
> >
> >
> > What Freeman describes below isn't the FULL story. Normally, even with
a
> > single large attachment, nothing would be written to disk. Whenever
> > possible, CXF tries to stream things directly. In the case of a single
> > attachment, or the last attachment when using multiple attachments, CXF
can
> > stream directly and thus it wouldn't normally create the files on disk.
> >
> > However, when any of the cases that force the attachment to be fully
> > realized
> > occurs and breaks the streaming, it would automatically buffer them into
> > the
> >
> > files on disk like Freeman describes.
> >
> > Dan
> >
> > On Sunday 24 April 2011 11:36:01 PM Freeman Fang wrote:
> > > Hi,
> > >
> > > My comment inline
> > >
> > > On 2011-4-25, at 上午11:16, ext2 wrote:
> > > > Hi:
> > > > Sometimes, web service (both service and client) need to
receive
> > > >
> > > > very large attachment.
> > > >
> > > > 1) does cxf support save large attachment in temporary file
for
> > > >
> > > > service to receive request , and client to receive response?
> > >
> > > Yeah,
> > > When cxf message exceed a threshold(64 k by default), it will save the
> > > memory message to temp file on hard disk(by default it save to the
> > > folder specified by a pre-set well-known java system properties
> > > java.io.tempdir + cxf-temp-random ).
> > >
> > > > 2)how could I configure the temporary directory and
threshold?
> > >
> > > You can specify the threshold with property
> > > org.apache.cxf.io.CachedOutputStream.Threshold (64k by default) t
> > > and specify the cxf temp dir for big size message with property
> > > org.apache.cxf.io.CachedOutputStream.OutputDirectory
> > > to a folder you have full acesss permission.
> > > Freeman
> > >
> > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------
> > > Freeman Fang
> > >
> > > FuseSource
> > > Email:ffang@fusesource.com
> > > Web: fusesource.com
> > > Twitter: freemanfang
> > > Blog: http://freemanfang.blogspot.com
> > > Connect at CamelOne May 24-26
> > > The Open Source Integration Conference
>
> --
> Daniel Kulp
> dkulp@apache.org
> http://dankulp.com/blog
> Talend - http://www.talend.com
Re: could cxf save larget attachemnt in temporary file?(A new doubt)
Posted by Daniel Kulp <dk...@apache.org>.
On Tuesday, April 26, 2011 11:40:06 PM ext2 wrote:
> Thanks Daniel Kulp for your explain.
>
> Because CXF maintain the attachments stream underlying, now I feel doubt
> about how could I cleaning the temporary file.
>
> There are two use case, as following:
>
> 1) at server side, receiving request with large attachment.
> Could CXF engine clean the temporary files automatically, just after the
> service send response?
Well, yes and no. When STREAMING attachments into the service, when the
stream is closed, the attachment file gets deleted. Thus, if you are seeing
a bunch of stuff left on disk, it's likely that your code isn't properly
closing the input streams.
> 2) at client side, receiving response with large attachment.
> Now the CXF Engine cannot automatically clean temporary files, only the
> user know the time when to clean.
Same as above. The inputstreams we provide to the datahandler/datasource
things override the close() method to track things and will remove the temp
files automatically.
> Because the CXF may buffer the last attachment's stream to disk
> automatically, the user may unaware of this conversion.
> So does CXF providing some method(API),which I could use to clean the
> temporary attachment file safely?
Just close the inputstreams. Note: this is ALSO important for the non-
buffered attachments (like the last one) as the call to close will let us know
to close the underlying HTTP input stream and the URL connect can be returned
to the keep-alive pool and such.
Dan
>
> Thanks any suggestion;
>
>
> -----Original -----
> Sender: Daniel Kulp [mailto:dkulp@apache.org]
> Date: 2011/4/26 10:35
> Sender: users@cxf.apache.or
> Subject: Re: could cxf save larget attachemnt in temporary file?
>
>
> What Freeman describes below isn't the FULL story. Normally, even with a
> single large attachment, nothing would be written to disk. Whenever
> possible, CXF tries to stream things directly. In the case of a single
> attachment, or the last attachment when using multiple attachments, CXF can
> stream directly and thus it wouldn't normally create the files on disk.
>
> However, when any of the cases that force the attachment to be fully
> realized
> occurs and breaks the streaming, it would automatically buffer them into
> the
>
> files on disk like Freeman describes.
>
> Dan
>
> On Sunday 24 April 2011 11:36:01 PM Freeman Fang wrote:
> > Hi,
> >
> > My comment inline
> >
> > On 2011-4-25, at 上午11:16, ext2 wrote:
> > > Hi:
> > > Sometimes, web service (both service and client) need to receive
> > >
> > > very large attachment.
> > >
> > > 1) does cxf support save large attachment in temporary file for
> > >
> > > service to receive request , and client to receive response?
> >
> > Yeah,
> > When cxf message exceed a threshold(64 k by default), it will save the
> > memory message to temp file on hard disk(by default it save to the
> > folder specified by a pre-set well-known java system properties
> > java.io.tempdir + cxf-temp-random ).
> >
> > > 2)how could I configure the temporary directory and threshold?
> >
> > You can specify the threshold with property
> > org.apache.cxf.io.CachedOutputStream.Threshold (64k by default) t
> > and specify the cxf temp dir for big size message with property
> > org.apache.cxf.io.CachedOutputStream.OutputDirectory
> > to a folder you have full acesss permission.
> > Freeman
> >
> >
> >
> >
> >
> >
> > ---------------------------------------------
> > Freeman Fang
> >
> > FuseSource
> > Email:ffang@fusesource.com
> > Web: fusesource.com
> > Twitter: freemanfang
> > Blog: http://freemanfang.blogspot.com
> > Connect at CamelOne May 24-26
> > The Open Source Integration Conference
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog
Talend - http://www.talend.com