You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cxf.apache.org by Daniel Lipofsky <Da...@bricsnet.com> on 2008/12/02 19:14:40 UTC
Closing temp files after streaming is complete
I have a web-service that is returning a stream like this:
<xsd:element name="objects" type="xsd:base64Binary"
xmime:expectedContentTypes="application/octet-stream"/>
My implementation method works by creating a
temporary file and the returning like
return new DataHandler(new FileDataSource(tempFile));
My understanding is that this is asynchronous.
So how can I trigger a deletion of this file
when the streaming is done?
I am already calling tempFile.deleteOnExit() but it
could be months or years before the JVM actually exits,
and I want to delete it sooner.
Thanks,
Dan
RE: Closing temp files after streaming is complete
Posted by Daniel Lipofsky <Da...@bricsnet.com>.
solution:
after a few emails with Daniel Kulp he determined that
writeTo() is only being called when MTOM is enabled.
Which is what I really wanted, I just didn't have it
configured right. Without MTOM it is handled by JAXB
and therefore out of the control of the CXF developers.
However, the code below will work either with or without MTOM:
public DataHandler getObjectsAsAttachment(...) ... {
try {
final File tempFile = ...;
FileDataSource ds = new FileDataSource(tempFile) {
@Override
public InputStream getInputStream() throws IOException {
return new FileInputStream(tempFile) {
@Override
public void close() throws IOException {
super.close();
tempFile.delete();
}
};
}
};
return new DataHandler(ds);
} catch (Exception e) {
...
}
}
Note: close is also called when FileInputStream is
finalized by the garbage collector, and I also called
tempFile.deleteOnExit(), so I am not bothering to try
to cleanup the file in the catch block.
Thanks for your help.
- Dan
On Friday 05 December 2008 3:22 pm Daniel Kulp wrote:
>
> On Thursday 04 December 2008 7:21:36 pm Daniel Lipofsky wrote:
> > Are those changes in 2.1.3 or are they unreleased?
> > I upgraded to 2.1.3, and using the code below I can
> > see that it is not calling either writeTo() or
> > getInputStream() on the DataHandler. It is calling
> > getInputStream() on the DataSource. I see output from
> > the first 2 log calls but not any of the others.
>
> Interesting. Anychance you could put a Thread.dumpStack() in there
> and let me know where that's being called?
>
> The AttachmentOut stuff calls writeTo, but it's possible one of the
> other interceptors is remapping things someplace.
>
> > Also, do you think this code below is acceptable.
> > Notice I am NOT calling super on getInputStream().
> > This is because I wanted to create my own InputStream
> > rather than wrap the one that was returned by super.
> > Do you see any problems? (it works)
>
> Yea, that's fine and probably preferable since if we do take the
> DataSource out of the DataHandler, it would still work.
>
> Dan
Re: Closing temp files after streaming is complete
Posted by Daniel Kulp <dk...@apache.org>.
On Thursday 04 December 2008 7:21:36 pm Daniel Lipofsky wrote:
> Are those changes in 2.1.3 or are they unreleased?
> I upgraded to 2.1.3, and using the code below I can
> see that it is not calling either writeTo() or
> getInputStream() on the DataHandler. It is calling
> getInputStream() on the DataSource. I see output from
> the first 2 log calls but not any of the others.
Interesting. Anychance you could put a Thread.dumpStack() in there and let
me know where that's being called?
The AttachmentOut stuff calls writeTo, but it's possible one of the other
interceptors is remapping things someplace.
> Also, do you think this code below is acceptable.
> Notice I am NOT calling super on getInputStream().
> This is because I wanted to create my own InputStream
> rather than wrap the one that was returned by super.
> Do you see any problems? (it works)
Yea, that's fine and probably preferable since if we do take the DataSource
out of the DataHandler, it would still work.
Dan
>
> FileDataSource ds = new FileDataSource(tempFile) {
> @Override
> public InputStream getInputStream() throws IOException {
> log.warn("I ***** " + tempFile);
> return new FileInputStream(tempFile) {
> public void close() throws IOException {
> log.warn("C ***** " + tempFile);
> super.close();
> tempFile.delete();
> }
> };
> }tuff at:
> http://mssoapinterop.org/ilab/
>
I can see UX, UXD, AD...., MS Interop Plugfest scenarios. I too have
worked on preparing them with Jiandong Guo (
> };
> return new DataHandler(ds);
>
> Thanks,
> Dan
>
> > -----Original Message-----
> > From: Daniel Kulp [mailto:dkulp@apache.org]
> > Sent: Wednesday, December 03, 2008 12:18 PM
> > To: users@cxf.apache.org
> > Cc: Daniel Lipofsky
> > Subject: Re: Closing temp files after streaming is complete
> >
> >
> > I was looking at the latest 2.1.x code which I know has had a bunch of
> > changes
> > around the attachment support to make sure streams are closed and
>
> such.
>
> > It's
> > quite possible that 2.1.0 is different.
> >
> > That said, the OTHER thing you can try is to override the
> > getInputStream()
> > call on the FileDataSource to return a FilterOutputStream that
> > overrides
> > the "close" method to call super.close() and then delete the file.
>
> For
>
> > the
> > most part, I THINK we either have to call DataHandler.writeTo(..) or
> > call
> > DataHandler.getInputStream() and copy the bytes out of there and into
> > the
> > target stream.
> >
> > Dan
> >
> > On Wednesday 03 December 2008 2:08:14 pm Daniel Lipofsky wrote:
> > > Unfortunately that doesn't seem to work. I see no evidence
> > > that my code is actually being called, i.e. there is no log
> > > output and the file is not deleted, but everything works
> > > without errors like it did before I added this code.
> > > Does CXF really call this method? It seems possible
> > > that it could be getting the streams directly from the
> > > DataHandler. I'm using CXF 2.1.0, JBoss 4.2.3, JDK 1.5.
> > >
> > > This is what I am implemented:
> > >
> > > public DataHandler getObjectsAsAttachment(...) throws WSException {
> > > try {
> > > final File tempFile = ...;
> > > ...
> > > return new DataHandler(new FileDataSource(tempFile)) {
> > > @Override
> > > public void writeTo(OutputStream out) throws IOException
> >
> > {
> >
> > > super.writeTo(out);
> > > log.info("########### " + tempFile + "
>
> ###########");
>
> > > tempFile.delete();
> > > }
> > > };
> > > } catch (Exception e) {
> > > log.error("Exception thrown in getObjectsAsAttachment", e);
> > > throw new WSException(e);
> > > }
> > > }
> > >
> > > > -----Original Message-----
> > > > From: Daniel Kulp
> > > > Sent: Tuesday, December 02, 2008 2:38 PM
> > > >
> > > > In general, in this type of case, I recommend doing something
>
> like:
> > > > (pseudo
> > > > code)
> > > >
> > > > return new DataHandler(new FileDataSource(tempFile)) {
> > > > public void writeTo(OutputStream out) {
> > > > super.writeTo(out);
> > > > tempFile.delete();
> > > > }
> > > > };
> > > >
> > > > or similar. Basically, when the datahandler is done being
>
> written
>
> > > > out,
> > > > delete the file.
> > > >
> > > > Dan
> > > >
> > > > On Tuesday 02 December 2008 1:14:40 pm Daniel Lipofsky wrote:
> > > > > I have a web-service that is returning a stream like this:
> > > > >
> > > > > <xsd:element name="objects" type="xsd:base64Binary"
> > > > > xmime:expectedContentTypes="application/octet-stream"/>
> > > > >
> > > > > My implementation method works by creating a
> > > > > temporary file and the returning like
> > > > >
> > > > > return new DataHandler(new FileDataSource(tempFile));
> > > > >
> > > > > My understanding is that this is asynchronous.
> > > > > So how can I trigger a deletion of this file
> > > > > when the streaming is done?
> > > > >
> > > > > I am already calling tempFile.deleteOnExit() but it
> > > > > could be months or years before the JVM actually exits,
> > > > > and I want to delete it sooner.
> > > > >
> > > > > Thanks,
> > > > > Dan
> >
> > --
> > Daniel Kulp
> > dkulp@apache.org
> > http://dankulp.com/blog
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog
RE: Closing temp files after streaming is complete
Posted by Daniel Lipofsky <Da...@bricsnet.com>.
Are those changes in 2.1.3 or are they unreleased?
I upgraded to 2.1.3, and using the code below I can
see that it is not calling either writeTo() or
getInputStream() on the DataHandler. It is calling
getInputStream() on the DataSource. I see output from
the first 2 log calls but not any of the others.
public DataHandler getObjectsAsAttachment(...) throws WSException {
final File tempFile = ...;
log.warn("!!!!! " + tempFile);
FileDataSource ds = new FileDataSource(tempFile) {
@Override
public InputStream getInputStream() throws IOException {
log.warn("I ***** " + tempFile);
return super.getInputStream();
}
@Override
public OutputStream getOutputStream() throws IOException {
log.warn("O ***** " + tempFile);
return super.getOutputStream();
}
};
return new DataHandler(ds) {
@Override
public void writeTo(OutputStream out) throws IOException {
log.warn("W ##### " + tempFile);
super.writeTo(out);
}
@Override
public InputStream getInputStream() throws IOException {
log.warn("I ##### " + tempFile);
return super.getInputStream();
}
@Override
public OutputStream getOutputStream() throws IOException {
log.warn("O ##### " + tempFile);
return super.getOutputStream();
}
};
}
Also, do you think this code below is acceptable.
Notice I am NOT calling super on getInputStream().
This is because I wanted to create my own InputStream
rather than wrap the one that was returned by super.
Do you see any problems? (it works)
FileDataSource ds = new FileDataSource(tempFile) {
@Override
public InputStream getInputStream() throws IOException {
log.warn("I ***** " + tempFile);
return new FileInputStream(tempFile) {
public void close() throws IOException {
log.warn("C ***** " + tempFile);
super.close();
tempFile.delete();
}
};
}
};
return new DataHandler(ds);
Thanks,
Dan
> -----Original Message-----
> From: Daniel Kulp [mailto:dkulp@apache.org]
> Sent: Wednesday, December 03, 2008 12:18 PM
> To: users@cxf.apache.org
> Cc: Daniel Lipofsky
> Subject: Re: Closing temp files after streaming is complete
>
>
> I was looking at the latest 2.1.x code which I know has had a bunch of
> changes
> around the attachment support to make sure streams are closed and
such.
> It's
> quite possible that 2.1.0 is different.
>
> That said, the OTHER thing you can try is to override the
> getInputStream()
> call on the FileDataSource to return a FilterOutputStream that
> overrides
> the "close" method to call super.close() and then delete the file.
For
> the
> most part, I THINK we either have to call DataHandler.writeTo(..) or
> call
> DataHandler.getInputStream() and copy the bytes out of there and into
> the
> target stream.
>
> Dan
>
>
> On Wednesday 03 December 2008 2:08:14 pm Daniel Lipofsky wrote:
> > Unfortunately that doesn't seem to work. I see no evidence
> > that my code is actually being called, i.e. there is no log
> > output and the file is not deleted, but everything works
> > without errors like it did before I added this code.
> > Does CXF really call this method? It seems possible
> > that it could be getting the streams directly from the
> > DataHandler. I'm using CXF 2.1.0, JBoss 4.2.3, JDK 1.5.
> >
> > This is what I am implemented:
> >
> > public DataHandler getObjectsAsAttachment(...) throws WSException {
> > try {
> > final File tempFile = ...;
> > ...
> > return new DataHandler(new FileDataSource(tempFile)) {
> > @Override
> > public void writeTo(OutputStream out) throws IOException
> {
> > super.writeTo(out);
> > log.info("########### " + tempFile + "
###########");
> > tempFile.delete();
> > }
> > };
> > } catch (Exception e) {
> > log.error("Exception thrown in getObjectsAsAttachment", e);
> > throw new WSException(e);
> > }
> > }
> >
> > > -----Original Message-----
> > > From: Daniel Kulp
> > > Sent: Tuesday, December 02, 2008 2:38 PM
> > >
> > > In general, in this type of case, I recommend doing something
like:
> > > (pseudo
> > > code)
> > >
> > > return new DataHandler(new FileDataSource(tempFile)) {
> > > public void writeTo(OutputStream out) {
> > > super.writeTo(out);
> > > tempFile.delete();
> > > }
> > > };
> > >
> > > or similar. Basically, when the datahandler is done being
written
> > > out,
> > > delete the file.
> > >
> > > Dan
> > >
> > > On Tuesday 02 December 2008 1:14:40 pm Daniel Lipofsky wrote:
> > > > I have a web-service that is returning a stream like this:
> > > >
> > > > <xsd:element name="objects" type="xsd:base64Binary"
> > > > xmime:expectedContentTypes="application/octet-stream"/>
> > > >
> > > > My implementation method works by creating a
> > > > temporary file and the returning like
> > > >
> > > > return new DataHandler(new FileDataSource(tempFile));
> > > >
> > > > My understanding is that this is asynchronous.
> > > > So how can I trigger a deletion of this file
> > > > when the streaming is done?
> > > >
> > > > I am already calling tempFile.deleteOnExit() but it
> > > > could be months or years before the JVM actually exits,
> > > > and I want to delete it sooner.
> > > >
> > > > Thanks,
> > > > Dan
>
>
>
> --
> Daniel Kulp
> dkulp@apache.org
> http://dankulp.com/blog
Re: Closing temp files after streaming is complete
Posted by Daniel Kulp <dk...@apache.org>.
I was looking at the latest 2.1.x code which I know has had a bunch of changes
around the attachment support to make sure streams are closed and such. It's
quite possible that 2.1.0 is different.
That said, the OTHER thing you can try is to override the getInputStream()
call on the FileDataSource to return a FilterOutputStream that overrides
the "close" method to call super.close() and then delete the file. For the
most part, I THINK we either have to call DataHandler.writeTo(..) or call
DataHandler.getInputStream() and copy the bytes out of there and into the
target stream.
Dan
On Wednesday 03 December 2008 2:08:14 pm Daniel Lipofsky wrote:
> Unfortunately that doesn't seem to work. I see no evidence
> that my code is actually being called, i.e. there is no log
> output and the file is not deleted, but everything works
> without errors like it did before I added this code.
> Does CXF really call this method? It seems possible
> that it could be getting the streams directly from the
> DataHandler. I'm using CXF 2.1.0, JBoss 4.2.3, JDK 1.5.
>
> This is what I am implemented:
>
> public DataHandler getObjectsAsAttachment(...) throws WSException {
> try {
> final File tempFile = ...;
> ...
> return new DataHandler(new FileDataSource(tempFile)) {
> @Override
> public void writeTo(OutputStream out) throws IOException {
> super.writeTo(out);
> log.info("########### " + tempFile + " ###########");
> tempFile.delete();
> }
> };
> } catch (Exception e) {
> log.error("Exception thrown in getObjectsAsAttachment", e);
> throw new WSException(e);
> }
> }
>
> > -----Original Message-----
> > From: Daniel Kulp
> > Sent: Tuesday, December 02, 2008 2:38 PM
> >
> > In general, in this type of case, I recommend doing something like:
> > (pseudo
> > code)
> >
> > return new DataHandler(new FileDataSource(tempFile)) {
> > public void writeTo(OutputStream out) {
> > super.writeTo(out);
> > tempFile.delete();
> > }
> > };
> >
> > or similar. Basically, when the datahandler is done being written
> > out,
> > delete the file.
> >
> > Dan
> >
> > On Tuesday 02 December 2008 1:14:40 pm Daniel Lipofsky wrote:
> > > I have a web-service that is returning a stream like this:
> > >
> > > <xsd:element name="objects" type="xsd:base64Binary"
> > > xmime:expectedContentTypes="application/octet-stream"/>
> > >
> > > My implementation method works by creating a
> > > temporary file and the returning like
> > >
> > > return new DataHandler(new FileDataSource(tempFile));
> > >
> > > My understanding is that this is asynchronous.
> > > So how can I trigger a deletion of this file
> > > when the streaming is done?
> > >
> > > I am already calling tempFile.deleteOnExit() but it
> > > could be months or years before the JVM actually exits,
> > > and I want to delete it sooner.
> > >
> > > Thanks,
> > > Dan
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog
RE: Closing temp files after streaming is complete
Posted by Daniel Lipofsky <Da...@bricsnet.com>.
Unfortunately that doesn't seem to work. I see no evidence
that my code is actually being called, i.e. there is no log
output and the file is not deleted, but everything works
without errors like it did before I added this code.
Does CXF really call this method? It seems possible
that it could be getting the streams directly from the
DataHandler. I'm using CXF 2.1.0, JBoss 4.2.3, JDK 1.5.
This is what I am implemented:
public DataHandler getObjectsAsAttachment(...) throws WSException {
try {
final File tempFile = ...;
...
return new DataHandler(new FileDataSource(tempFile)) {
@Override
public void writeTo(OutputStream out) throws IOException {
super.writeTo(out);
log.info("########### " + tempFile + " ###########");
tempFile.delete();
}
};
} catch (Exception e) {
log.error("Exception thrown in getObjectsAsAttachment", e);
throw new WSException(e);
}
}
> -----Original Message-----
> From: Daniel Kulp
> Sent: Tuesday, December 02, 2008 2:38 PM
>
> In general, in this type of case, I recommend doing something like:
> (pseudo
> code)
>
> return new DataHandler(new FileDataSource(tempFile)) {
> public void writeTo(OutputStream out) {
> super.writeTo(out);
> tempFile.delete();
> }
> };
>
> or similar. Basically, when the datahandler is done being written
> out,
> delete the file.
>
> Dan
>
>
> On Tuesday 02 December 2008 1:14:40 pm Daniel Lipofsky wrote:
> > I have a web-service that is returning a stream like this:
> >
> > <xsd:element name="objects" type="xsd:base64Binary"
> > xmime:expectedContentTypes="application/octet-stream"/>
> >
> > My implementation method works by creating a
> > temporary file and the returning like
> >
> > return new DataHandler(new FileDataSource(tempFile));
> >
> > My understanding is that this is asynchronous.
> > So how can I trigger a deletion of this file
> > when the streaming is done?
> >
> > I am already calling tempFile.deleteOnExit() but it
> > could be months or years before the JVM actually exits,
> > and I want to delete it sooner.
> >
> > Thanks,
> > Dan
Re: Closing temp files after streaming is complete
Posted by Daniel Kulp <dk...@apache.org>.
In general, in this type of case, I recommend doing something like: (pseudo
code)
return new DataHandler(new FileDataSource(tempFile)) {
public void writeTo(OutputStream out) {
super.writeTo(out);
tempFile.delete();
}
};
or similar. Basically, when the datahandler is done being written out,
delete the file.
Dan
On Tuesday 02 December 2008 1:14:40 pm Daniel Lipofsky wrote:
> I have a web-service that is returning a stream like this:
>
> <xsd:element name="objects" type="xsd:base64Binary"
> xmime:expectedContentTypes="application/octet-stream"/>
>
> My implementation method works by creating a
> temporary file and the returning like
>
> return new DataHandler(new FileDataSource(tempFile));
>
> My understanding is that this is asynchronous.
> So how can I trigger a deletion of this file
> when the streaming is done?
>
> I am already calling tempFile.deleteOnExit() but it
> could be months or years before the JVM actually exits,
> and I want to delete it sooner.
>
> Thanks,
> Dan
--
Daniel Kulp
dkulp@apache.org
http://dankulp.com/blog