[fileupload] memory consumption

[Josh Foure <us...@yahoo.com>]

Hi All,

I have noticed a concerning memory issue when I use
the File Upload 1.1 with the following code:

int maxFileSize = 100000;
DiskFileItemFactory factory = new
DiskFileItemFactory();
factory.setSizeThreshold(maxFileSize);
ServletFileUpload upload = new
ServletFileUpload(factory);
upload.setSizeMax(maxFileSize);
List items = upload.parseRequest(request);

I set the max file size and the threshold size to be
the same (100k) so that files are never writen into
the temp directory.  The problem I noticed is that
when I submitted a form containing 50 input fields, a
temporary byte array of 100k was created for each of
the 50 fields.  So in my case, 50 fields * 100k per
field = 5MB of memory was used for a single form being
submitted.  Is this the normal behavior?  Also, the
documentation states:

# Uploaded items should be retained in memory as long
as they are reasonably small.
# Larger items should be written to a temporary file
on disk.
# Very large upload requests should not be permitted.

Although I would imagine that "reasonably small" and
"larger items" and "very large" would depend on the
application, the memory available and the amount of
concurrent users, are there any guidelines?

Thanks,
Josh

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org

Re: [fileupload] memory consumption

[Martin Cooper <ma...@apache.org>]

On 12/12/05, Josh Foure <us...@yahoo.com> wrote:
>
> Hi All,
>
> I have noticed a concerning memory issue when I use
> the File Upload 1.1


FileUpload 1.1 has not yet been release. Could you be a little more specific
as to what code you are using?

with the following code:
>
> int maxFileSize = 100000;
> DiskFileItemFactory factory = new
> DiskFileItemFactory();
> factory.setSizeThreshold(maxFileSize);
> ServletFileUpload upload = new
> ServletFileUpload(factory);
> upload.setSizeMax(maxFileSize);
> List items = upload.parseRequest(request);
>
> I set the max file size and the threshold size to be
> the same (100k) so that files are never writen into
> the temp directory.  The problem I noticed is that
> when I submitted a form containing 50 input fields, a
> temporary byte array of 100k was created for each of
> the 50 fields.  So in my case, 50 fields * 100k per
> field = 5MB of memory was used for a single form being
> submitted.  Is this the normal behavior?


This was the behaviour in early versions of FileUpload 1.1-dev. It has been
fixed for a while now, though. Please try the latest version of FileUpload
together with Commons IO 1.1. Your problem should be resolved with those
versions.

  Also, the
> documentation states:
>
> # Uploaded items should be retained in memory as long
> as they are reasonably small.
> # Larger items should be written to a temporary file
> on disk.
> # Very large upload requests should not be permitted.
>
> Although I would imagine that "reasonably small" and
> "larger items" and "very large" would depend on the
> application, the memory available and the amount of
> concurrent users, are there any guidelines?


No, because those guidelines would be very specific to your application and
the hardware on which it is running.

--
Martin Cooper


Thanks,
> Josh
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>