You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Henri Yandell <ba...@generationjava.com> on 2003/11/16 01:31:29 UTC
Re: Fileupload: fails for forms with a large number of inputs with
OurOfMemoryError
Were any of the fileupload committers interested in this patch?
Can't see any obvious replies.
Hen
On Thu, 9 Oct 2003, Mike Samuel wrote:
> The fileupload package is well-architected, and a pleasure to use, but I ran
> into a problem. Attached is a test case servlet which repeats the problem with
> the Jun 24, 2003 fileupload-1.0 release.
>
> FileUploadBase.parseRequest runs out of memory when parsing a form with a large
> number of inputs. The cause seems to be DeferredFileOutputStream which
> allocates a ByteArrayOutputStream per input, each of which preallocates a buffer
> of length inMemoryThreshold. The in memory threshold defaults to 10k, but if it
> is made larger (> 1 M in our environment), then the vm quickly runs out of
> memory. Most of this memory is wasted since most files are ~5k, and almost all
> non-file inputs are less than 1k.
>
> I patched DeferredFileOutputStream to use a different underlying in memory
> stream. I don't really know why ByteArrayOutputStream uses a single byte[],
> since you can't do random access on the underlying buffer anyway.
> I think the patch is something that could be incorporated into the default file
> upload implementation without any noticable change in performance.
>
> If anyone's interested in the patch, should I send it to someone, or check it in
> myself? I've never contributed any code to apache before so I'm a little fuzzy
> on process.
>
> thanks,
> mike
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: Fileupload: fails for forms with a large number of inputs with
OurOfMemoryError
Posted by Mike Samuel <ms...@google.com>.
I filed this as bug: 24306 The patch is attached.
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=24306
mike
Henri Yandell wrote:
> Were any of the fileupload committers interested in this patch?
>
> Can't see any obvious replies.
>
> Hen
>
> On Thu, 9 Oct 2003, Mike Samuel wrote:
>
>
>>The fileupload package is well-architected, and a pleasure to use, but I ran
>>into a problem. Attached is a test case servlet which repeats the problem with
>>the Jun 24, 2003 fileupload-1.0 release.
>>
>>FileUploadBase.parseRequest runs out of memory when parsing a form with a large
>>number of inputs. The cause seems to be DeferredFileOutputStream which
>>allocates a ByteArrayOutputStream per input, each of which preallocates a buffer
>>of length inMemoryThreshold. The in memory threshold defaults to 10k, but if it
>>is made larger (> 1 M in our environment), then the vm quickly runs out of
>>memory. Most of this memory is wasted since most files are ~5k, and almost all
>>non-file inputs are less than 1k.
>>
>>I patched DeferredFileOutputStream to use a different underlying in memory
>>stream. I don't really know why ByteArrayOutputStream uses a single byte[],
>>since you can't do random access on the underlying buffer anyway.
>>I think the patch is something that could be incorporated into the default file
>>upload implementation without any noticable change in performance.
>>
>>If anyone's interested in the patch, should I send it to someone, or check it in
>>myself? I've never contributed any code to apache before so I'm a little fuzzy
>>on process.
>>
>>thanks,
>> mike
>>
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: Fileupload: fails for forms with a large number of inputs with
OurOfMemoryError
Posted by Martin Cooper <ma...@apache.org>.
On Sat, 15 Nov 2003, Henri Yandell wrote:
>
> Were any of the fileupload committers interested in this patch?
I'm aware of it, and have responded to the bug report. I'll try to address
it more fully when I have a little more time on my hands to pick up where
I left off on FileUpload.
--
Martin Cooper
>
> Can't see any obvious replies.
>
> Hen
>
> On Thu, 9 Oct 2003, Mike Samuel wrote:
>
> > The fileupload package is well-architected, and a pleasure to use, but I ran
> > into a problem. Attached is a test case servlet which repeats the problem with
> > the Jun 24, 2003 fileupload-1.0 release.
> >
> > FileUploadBase.parseRequest runs out of memory when parsing a form with a large
> > number of inputs. The cause seems to be DeferredFileOutputStream which
> > allocates a ByteArrayOutputStream per input, each of which preallocates a buffer
> > of length inMemoryThreshold. The in memory threshold defaults to 10k, but if it
> > is made larger (> 1 M in our environment), then the vm quickly runs out of
> > memory. Most of this memory is wasted since most files are ~5k, and almost all
> > non-file inputs are less than 1k.
> >
> > I patched DeferredFileOutputStream to use a different underlying in memory
> > stream. I don't really know why ByteArrayOutputStream uses a single byte[],
> > since you can't do random access on the underlying buffer anyway.
> > I think the patch is something that could be incorporated into the default file
> > upload implementation without any noticable change in performance.
> >
> > If anyone's interested in the patch, should I send it to someone, or check it in
> > myself? I've never contributed any code to apache before so I'm a little fuzzy
> > on process.
> >
> > thanks,
> > mike
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org