You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Thusitha Thilina Dayaratne <th...@gmail.com> on 2016/05/30 05:43:10 UTC
How to Access File Streams Later
Hi All,
I'm trying to use apache commons-fileupload to support file streaming with
my own rest framework. I get a multipart/form-data request from the client
side and want to the file as as stream to the backend.
FileItemIterator iter = upload.getItemIterator(request);
> while (iter.hasNext()) {
> FileItemStream item = iter.next();
> // Add objects to a map (I don't want to process the stream here.
> Just pass)
> ..... // If I access the stream here no errors and everything ok
> }
But in the backend side when I'm trying to access the stream I'm getting
ItemSkippedException
AFAIU it seems that hasNext() method closes the stream. Therefore I have to
consume the stream before calling next hasNext().
But I just want to access as a proxy and consume the stream later. Is there
a possible way for me to consume the stream later ?
--
Thanks
Thusitha
Re: How to Access File Streams Later
Posted by Thusitha Thilina Dayaratne <th...@gmail.com>.
Hi Li Ying,
Thanks for the detailed explanation. I will look into that.
Btw I directly pass the FileItemIterator to service (without processing) so
the developer can consume the streams at the service level. Do you find any
issues with that approach?
Best Regards
Thusitha
2016-05-31 7:34 GMT+05:30 Li Ying <li...@gmail.com>:
> Hi, Thusitha
>
> I beleive you have to use temporary files, for the following reasons:
>
> (1)If I remember correct, HTTP Request is an one-time input stream.
> That means, you have to read out (and consume) the whole HTTP Request
> content to retrieve the uploaded files and the other request parameters,
> so, there is not a way you can hold the input stream and consume it later
>
> (2)If the file size is large, it will cost too many memory,
> so you must use temporary files, to prevent an out of memory error
>
>
> I am not sure, but I think that:
> The DiskFileItemFactory class can help you to manange the temporary files
> thing.
> it will dump the file content(if it is large) to temporary files for you,
> and give you back an instance of DiskFileItem, which wrap a temporary file
>
> *read these pages for detail info:
> https://commons.apache.org/proper/commons-fileupload/using.html
>
> https://commons.apache.org/proper/commons-fileupload/apidocs/org/apache/commons/fileupload/disk/DiskFileItemFactory.html
>
> https://commons.apache.org/proper/commons-fileupload/apidocs/org/apache/commons/fileupload/disk/DiskFileItem.html
>
>
> 2016-05-31 10:31 GMT+09:00 Thusitha Thilina Dayaratne <
> thusithathilina@gmail.com>:
>
> > Hi Li Ying,
> >
> > Thanks for the response. Yeah I agree with you I can save a temp file and
> > then create new set of stream. But when it comes to large files it will
> > take much time since first I've to save the file and then send it to the
> > actual service.
> > Therefore I was thinking of not to process the stream but just passing
> that
> > to the service.
> >
> > Thanks
> >
> > 2016-05-31 5:54 GMT+05:30 Li Ying <li...@gmail.com>:
> >
> > > Hi Thusitha
> > >
> > > Maybe you need save the upload-file stream into a temporary file on
> your
> > > server. And then, you can access the file content from the temporary
> file
> > >
> > > 2016-05-30 14:43 GMT+09:00 Thusitha Thilina Dayaratne <
> > > thusithathilina@gmail.com>:
> > >
> > > > Hi All,
> > > >
> > > > I'm trying to use apache commons-fileupload to support file streaming
> > > with
> > > > my own rest framework. I get a multipart/form-data request from the
> > > client
> > > > side and want to the file as as stream to the backend.
> > > >
> > > > FileItemIterator iter = upload.getItemIterator(request);
> > > > > while (iter.hasNext()) {
> > > > > FileItemStream item = iter.next();
> > > > > // Add objects to a map (I don't want to process the stream
> > here.
> > > > > Just pass)
> > > > > ..... // If I access the stream here no errors and everything ok
> > > > > }
> > > >
> > > >
> > > > But in the backend side when I'm trying to access the stream I'm
> > getting
> > > > ItemSkippedException
> > > > AFAIU it seems that hasNext() method closes the stream. Therefore I
> > have
> > > to
> > > > consume the stream before calling next hasNext().
> > > > But I just want to access as a proxy and consume the stream later. Is
> > > there
> > > > a possible way for me to consume the stream later ?
> > > >
> > > > --
> > > > Thanks
> > > > Thusitha
> > > >
> > >
> >
> >
> >
> > --
> >
>
--
Re: How to Access File Streams Later
Posted by Li Ying <li...@gmail.com>.
Hi, Thusitha
I beleive you have to use temporary files, for the following reasons:
(1)If I remember correct, HTTP Request is an one-time input stream.
That means, you have to read out (and consume) the whole HTTP Request
content to retrieve the uploaded files and the other request parameters,
so, there is not a way you can hold the input stream and consume it later
(2)If the file size is large, it will cost too many memory,
so you must use temporary files, to prevent an out of memory error
I am not sure, but I think that:
The DiskFileItemFactory class can help you to manange the temporary files
thing.
it will dump the file content(if it is large) to temporary files for you,
and give you back an instance of DiskFileItem, which wrap a temporary file
*read these pages for detail info:
https://commons.apache.org/proper/commons-fileupload/using.html
https://commons.apache.org/proper/commons-fileupload/apidocs/org/apache/commons/fileupload/disk/DiskFileItemFactory.html
https://commons.apache.org/proper/commons-fileupload/apidocs/org/apache/commons/fileupload/disk/DiskFileItem.html
2016-05-31 10:31 GMT+09:00 Thusitha Thilina Dayaratne <
thusithathilina@gmail.com>:
> Hi Li Ying,
>
> Thanks for the response. Yeah I agree with you I can save a temp file and
> then create new set of stream. But when it comes to large files it will
> take much time since first I've to save the file and then send it to the
> actual service.
> Therefore I was thinking of not to process the stream but just passing that
> to the service.
>
> Thanks
>
> 2016-05-31 5:54 GMT+05:30 Li Ying <li...@gmail.com>:
>
> > Hi Thusitha
> >
> > Maybe you need save the upload-file stream into a temporary file on your
> > server. And then, you can access the file content from the temporary file
> >
> > 2016-05-30 14:43 GMT+09:00 Thusitha Thilina Dayaratne <
> > thusithathilina@gmail.com>:
> >
> > > Hi All,
> > >
> > > I'm trying to use apache commons-fileupload to support file streaming
> > with
> > > my own rest framework. I get a multipart/form-data request from the
> > client
> > > side and want to the file as as stream to the backend.
> > >
> > > FileItemIterator iter = upload.getItemIterator(request);
> > > > while (iter.hasNext()) {
> > > > FileItemStream item = iter.next();
> > > > // Add objects to a map (I don't want to process the stream
> here.
> > > > Just pass)
> > > > ..... // If I access the stream here no errors and everything ok
> > > > }
> > >
> > >
> > > But in the backend side when I'm trying to access the stream I'm
> getting
> > > ItemSkippedException
> > > AFAIU it seems that hasNext() method closes the stream. Therefore I
> have
> > to
> > > consume the stream before calling next hasNext().
> > > But I just want to access as a proxy and consume the stream later. Is
> > there
> > > a possible way for me to consume the stream later ?
> > >
> > > --
> > > Thanks
> > > Thusitha
> > >
> >
>
>
>
> --
>
Re: How to Access File Streams Later
Posted by Thusitha Thilina Dayaratne <th...@gmail.com>.
Hi Li Ying,
Thanks for the response. Yeah I agree with you I can save a temp file and
then create new set of stream. But when it comes to large files it will
take much time since first I've to save the file and then send it to the
actual service.
Therefore I was thinking of not to process the stream but just passing that
to the service.
Thanks
2016-05-31 5:54 GMT+05:30 Li Ying <li...@gmail.com>:
> Hi Thusitha
>
> Maybe you need save the upload-file stream into a temporary file on your
> server. And then, you can access the file content from the temporary file
>
> 2016-05-30 14:43 GMT+09:00 Thusitha Thilina Dayaratne <
> thusithathilina@gmail.com>:
>
> > Hi All,
> >
> > I'm trying to use apache commons-fileupload to support file streaming
> with
> > my own rest framework. I get a multipart/form-data request from the
> client
> > side and want to the file as as stream to the backend.
> >
> > FileItemIterator iter = upload.getItemIterator(request);
> > > while (iter.hasNext()) {
> > > FileItemStream item = iter.next();
> > > // Add objects to a map (I don't want to process the stream here.
> > > Just pass)
> > > ..... // If I access the stream here no errors and everything ok
> > > }
> >
> >
> > But in the backend side when I'm trying to access the stream I'm getting
> > ItemSkippedException
> > AFAIU it seems that hasNext() method closes the stream. Therefore I have
> to
> > consume the stream before calling next hasNext().
> > But I just want to access as a proxy and consume the stream later. Is
> there
> > a possible way for me to consume the stream later ?
> >
> > --
> > Thanks
> > Thusitha
> >
>
--
Re: How to Access File Streams Later
Posted by Li Ying <li...@gmail.com>.
Hi Thusitha
Maybe you need save the upload-file stream into a temporary file on your
server. And then, you can access the file content from the temporary file
2016-05-30 14:43 GMT+09:00 Thusitha Thilina Dayaratne <
thusithathilina@gmail.com>:
> Hi All,
>
> I'm trying to use apache commons-fileupload to support file streaming with
> my own rest framework. I get a multipart/form-data request from the client
> side and want to the file as as stream to the backend.
>
> FileItemIterator iter = upload.getItemIterator(request);
> > while (iter.hasNext()) {
> > FileItemStream item = iter.next();
> > // Add objects to a map (I don't want to process the stream here.
> > Just pass)
> > ..... // If I access the stream here no errors and everything ok
> > }
>
>
> But in the backend side when I'm trying to access the stream I'm getting
> ItemSkippedException
> AFAIU it seems that hasNext() method closes the stream. Therefore I have to
> consume the stream before calling next hasNext().
> But I just want to access as a proxy and consume the stream later. Is there
> a possible way for me to consume the stream later ?
>
> --
> Thanks
> Thusitha
>