You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Nick Porter <ni...@tradar.com> on 2012/04/10 10:36:34 UTC
Problems uploading huge files >2GB to Tomcat app.
Hi all. New to Tomcat and not a developer but I have been trying to deploy a website using Debian6, Tomcat 6.0.35 hosting the OpenCMS 8.0.1 app. The website and the OpenCMS app has a facility to upload files however I only seem to be able to upload files that are less than 2GB. I did read that there used to be a 2GB bug in Tomcat a few versions ago but that this had since been resolved.
The errors I am seeing are a mixture of 400 Bad Request and 413 Request Entity too Large.
I'm connecting directly to Tomcat and the app/site on the HTTP/1.1 connector, port 8080.
I could be wrong but I can't see anything untoward in the logs located at /var/log/tomcat6/
The chaps over at the OpenCMS mail list assure me that OpenCMS is capable of handling files of this size so I'm guessing my Tomcat config is in some way causing the problem.
I've expanded the memory for the heap in the /etc/defaults/tomcat6 file with;
JAVA_OPTS="-Djava.awt.headless=true -Xmx512m -XX:+UseConcMarkSweepGC"
...and added the maxPostSize="0" to the HTTP connector but I'm still see these errors.
Here's my connector settings from server.xml
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
URIEncoding="UTF-8"
maxPostSize="0"
redirectPort="8443" />
The only other settings I found relating to the size of a POST were the packetSize attribute but believe this is only relevant to the AJP connector.
I'm at a loss as to what to try next so have come here as a last resort. Can anybody give me any pointers or advice on how I can further investigate and resolve this problem?
Thx.
--
np
--
Tradar Limited is a limited company registered in England and Wales. Registered number: 3431380.
Registered office: 20-21 Jockey's Fields, London. WC1R 4BW
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Problems uploading huge files >2GB to Tomcat app.
Posted by André Warnier <aw...@ice-sa.com>.
Just a note in the text below.
Nick Porter wrote:
>
...
>
> 1. Tomcat 6 does not handle file upload by itself (if you are using Content-Type="multipart/form-data"). It can only provide
> request.getInputStream() and something else has to consume it.
>
> Usually that will be Apache Commons Fileupload library, but you will have to check what OpenMS uses.
>
> [NP] I'm pretty sure OpenCMS is handling it. I don't have Apache installed in my test rig and can upload smaller files.
>
"Apache Commons Fileupload" is an add-on java library for Apache Tomcat (or other java
servlet engines), and has nothing to do with Apache httpd.
It may be used by openCMS to realise their file upload functionality.
That is why Konstantin was mentioning it. There may be a parameter to set, in order for
FileUpload to allow more than 2 GB.
See here : http://commons.apache.org/fileupload/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Problems uploading huge files >2GB to Tomcat app.
Posted by Konstantin Kolinko <kn...@gmail.com>.
2012/4/10 André Warnier <aw...@ice-sa.com>:
> Konstantin Kolinko wrote:
> ...
>>
>> This is from Tomcat 7 access log:
>>
>> [[[
>> 127.0.0.1 - - [10/Apr/2012:21:08:31 +0400] "POST
>>
>> /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=B5B161E96A5566CCE5FA70AB6477B2B9
>> HTTP/1.1" 400 - in:[-1980123022] out:[-]
>>
>> 127.0.0.1 - - [10/Apr/2012:21:09:24 +0400] "POST
>>
>> /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=2B89705754F848DE99DED1259DC376C1
>> HTTP/1.1" 400 - in:[-198012301] out:[-]
>> ]]]
>>
>> The first negative value is from Firefox. The second one is from IE.
>>
>>
> There is also a factor 10 difference between them, notice ?
> That cannot just be a different transfer-encoding. Or else Microsoft have
> invented a really good compression method.
>
;) They are negative values. So if you compare them the Firefox one is
smaller. I think that one would have to test with several files to
get several data points to confirm the trend.
Here are Firefox bugs regarding upload:of files >2Gb:
https://bugzilla.mozilla.org/show_bug.cgi?id=215450
https://bugzilla.mozilla.org/show_bug.cgi?id=660159
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Problems uploading huge files >2GB to Tomcat app.
Posted by André Warnier <aw...@ice-sa.com>.
Konstantin Kolinko wrote:
...
>
> This is from Tomcat 7 access log:
>
> [[[
> 127.0.0.1 - - [10/Apr/2012:21:08:31 +0400] "POST
> /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=B5B161E96A5566CCE5FA70AB6477B2B9
> HTTP/1.1" 400 - in:[-1980123022] out:[-]
>
> 127.0.0.1 - - [10/Apr/2012:21:09:24 +0400] "POST
> /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=2B89705754F848DE99DED1259DC376C1
> HTTP/1.1" 400 - in:[-198012301] out:[-]
> ]]]
>
> The first negative value is from Firefox. The second one is from IE.
>
>
There is also a factor 10 difference between them, notice ?
That cannot just be a different transfer-encoding. Or else Microsoft have invented a
really good compression method.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Problems uploading huge files >2GB to Tomcat app.
Posted by Konstantin Kolinko <kn...@gmail.com>.
2012/4/10 Konstantin Kolinko <kn...@gmail.com>:
> 2012/4/10 Nick Porter <ni...@tradar.com>:
>>
>> [NP] Yes. I can fathom no pattern to them. In fact, the only way I can actually see them is to packet trace the HTTP exchange.
>
> You do not have AccessLogValve configured?!
>
> I did the following to test how the standard Manager application
> handles it. In 6.0.x and then in current 7.0.x.
>
> 1. Configured AccessLogValve in server.xml as following, to print
> content-length headers of request and response:
>
> <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
> prefix="localhost_access_log." suffix=".txt"
> pattern='%h %l %u %t "%r" %s %b in:[%{content-length}i]
> out:[%{content-length}o]'
> />
>
> 2. Prepared a two files by concatenating several copies of a zip
> archive and renaming it to *.war.
>
> "test.war" is 127,586,991 bytes
> "test1.war" is 2,314,844,070 bytes
>
> 3. Started Tomcat, logged in into manager and tried to upload the
> files as new web applications.
>
> I am using Firefox 11.0, IE 8.0 both on Windows XP.
>
>
> The results:
> 1). Uploading test.war (127Mb) in Firefox 11
> (...)
> So everything is OK.
>
>
> 2). Uploading test1.war (>2Gb):
>
> Both Tomcat 6 and Tomcat 7 close request immediately, responding with
> status 400.
>
> Looking into access log I see wrong value of content-length header
> from the request.
> The value is different between Firefox and IE, but both are wrong,
> being negative.
>(...)
>
I've tested uploading the >2Gb file (test1.war) into Tomcat Manager
with current Google Chrome. Result: this browser handles it OK.
I had to change web.xml in manager app in Tomcat 7 to allow upload of
such a file. That is:
[[[
<multipart-config>
<max-file-size>-1</max-file-size>
<max-request-size>-1</max-request-size>
<file-size-threshold>0</file-size-threshold>
</multipart-config>
]]]
Result in Tomcat 6 - Failure. The manager displays error message:
FAIL - Deploy Upload Failed, Exception: the request was rejected
because it's size is unknown
In manager.<date>.log file:
[[[
12.04.2012 16:21:33 org.apache.catalina.core.ApplicationContext log
SEVERE: HTMLManager: FAIL - Deploy Upload Failed, Exception: the
request was rejected because it's size is unknown
org.apache.tomcat.util.http.fileupload.FileUploadBase$UnknownSizeException:
the request was rejected because it's size is unknown
at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:267)
at org.apache.catalina.manager.HTMLManagerServlet.doPost(HTMLManagerServlet.java:190)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
]]]
The code there is based on old version of commons fileupload and does
the following:
(FileUploadBase.java:267)
int requestSize = req.getContentLength();
if (requestSize == -1)
{
throw new UnknownSizeException(
"the request was rejected because it's size is unknown");
}
This code is used in the Manager application only, so I treat it as
WONTFIX, until there is a real need to deploy 2Gb wars through the
manager. ;) I am just saying that some old version of Commons
Fileupload did have an issue there.
In Tomcat 7 the code is newer and does handle "-1" properly. There it
more important, because that code is used to implement file upload
support for Servlet 3.0 applications.
In Tomcat 7 manager app (with the above mentioned configuration
change) the upload succeeded and the archive was unpacked.
Access log:
Tomcat 6:
[[[
127.0.0.1 - tomcat [12/Apr/2012:16:21:33 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=3F10D7BD7F9D5E31B56A4DB4EF35D96C
HTTP/1.1" 200 14661 in:[2314844272] out:[-]
]]]
Tomcat 7:
[[[
127.0.0.1 - tomcat [12/Apr/2012:17:22:34 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=E74BBB57E1F12736A9D0357B153BD4A7
HTTP/1.1" 200 17519 in:[2314844272] out:[-]
]]]
For reference:
File size was: 2314844070 bytes
Request size from Chrome was: 2314844272 bytes (= 0x89F9B870 = -1980123024)
Overhead from multipart/form-data was 202 bytes.
(Regarding broken Content-Length in IE: maybe it allocates only 10
chars for their number buffer, so when '-' occupies one character the
whole number gets truncated. Well, a negative value is a garbage
anyway).
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Problems uploading huge files >2GB to Tomcat app.
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Konstantin,
On 4/10/12 1:46 PM, Konstantin Kolinko wrote:
> 2). Uploading test1.war (>2Gb):
>
> Both Tomcat 6 and Tomcat 7 close request immediately, responding
> with status 400.
>
> Looking into access log I see wrong value of content-length header
> from the request. The value is different between Firefox and IE,
> but both are wrong, being negative. (Apparently the browsers have
> different coding for "multipart/form-data", with different
> overhead). The values are the same for Tomcat 6 and 7.
>
> This is from Tomcat 7 access log:
>
> [[[ 127.0.0.1 - - [10/Apr/2012:21:08:31 +0400] "POST
> /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=B5B161E96A5566CCE5FA70AB6477B2B9
>
>
HTTP/1.1" 400 - in:[-1980123022] out:[-]
>
> 127.0.0.1 - - [10/Apr/2012:21:09:24 +0400] "POST
> /manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=2B89705754F848DE99DED1259DC376C1
>
>
HTTP/1.1" 400 - in:[-198012301] out:[-]
> ]]]
>
> The first negative value is from Firefox. The second one is from
> IE.
This page seems to suggest that uploading files larger than 2GiB is
problematic in most web browsers:
http://www.motobit.com/help/scptutl/pa98.htm
Firefox definitely cannot currently upload files bigger than 2^31-1
bytes: https://bugzilla.mozilla.org/show_bug.cgi?id=215450
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk+Ee2IACgkQ9CaO5/Lv0PCfPgCcC8n6VoRjx6POYn+tOzx43HMM
0C8An2gfF3rNv8TCTjD5dUKPUWwYEwV0
=wd94
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Problems uploading huge files >2GB to Tomcat app.
Posted by Konstantin Kolinko <kn...@gmail.com>.
2012/4/10 Nick Porter <ni...@tradar.com>:
>
> [NP] Yes. I can fathom no pattern to them. In fact, the only way I can actually see them is to packet trace the HTTP exchange.
You do not have AccessLogValve configured?!
I did the following to test how the standard Manager application
handles it. In 6.0.x and then in current 7.0.x.
1. Configured AccessLogValve in server.xml as following, to print
content-length headers of request and response:
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log." suffix=".txt"
pattern='%h %l %u %t "%r" %s %b in:[%{content-length}i]
out:[%{content-length}o]'
/>
2. Prepared a two files by concatenating several copies of a zip
archive and renaming it to *.war.
"test.war" is 127,586,991 bytes
"test1.war" is 2,314,844,070 bytes
3. Started Tomcat, logged in into manager and tried to upload the
files as new web applications.
I am using Firefox 11.0, IE 8.0 both on Windows XP.
The results:
1). Uploading test.war (127Mb) in Firefox 11
Tomcat 6.0: OK
The first zip in concatenated file was unzipped as a new webapp.
Tomcat 7.0 Error 500, message
java.lang.IllegalStateException:
org.apache.tomcat.util.http.fileupload.FileUploadBase$SizeLimitExceededException:
the request was rejected because its size (127587196) exceeds the
configured maximum (52428800)
This behaviour of Tomcat 7 is expected, because of setting in its web.xml:
<multipart-config>
<!-- 50MB max -->
<max-file-size>52428800</max-file-size>
<max-request-size>52428800</max-request-size>
<file-size-threshold>0</file-size-threshold>
</multipart-config>
Access log:
Tomcat 6
127.0.0.1 - tomcat [10/Apr/2012:21:20:27 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=772A4FDB8109600ACA1BB00FBDE37F07
HTTP/1.1" 200 16044 in:[127587194] out:[-]
Tomcat 7
127.0.0.1 - username [10/Apr/2012:21:17:57 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=011354881097F245EB93726CF9F99CFE
HTTP/1.1" 500 2972 in:[127587196] out:[2972]
So everything is OK.
2). Uploading test1.war (>2Gb):
Both Tomcat 6 and Tomcat 7 close request immediately, responding with
status 400.
Looking into access log I see wrong value of content-length header
from the request.
The value is different between Firefox and IE, but both are wrong,
being negative.
(Apparently the browsers have different coding for
"multipart/form-data", with different overhead). The values are the
same for Tomcat 6 and 7.
This is from Tomcat 7 access log:
[[[
127.0.0.1 - - [10/Apr/2012:21:08:31 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=B5B161E96A5566CCE5FA70AB6477B2B9
HTTP/1.1" 400 - in:[-1980123022] out:[-]
127.0.0.1 - - [10/Apr/2012:21:09:24 +0400] "POST
/manager/html/upload?org.apache.catalina.filters.CSRF_NONCE=2B89705754F848DE99DED1259DC376C1
HTTP/1.1" 400 - in:[-198012301] out:[-]
]]]
The first negative value is from Firefox. The second one is from IE.
I'll try to debug it later.
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Problems uploading huge files >2GB to Tomcat app.
Posted by Nick Porter <ni...@tradar.com>.
Thanks for replying Konstantin. See my responses to your questions below.
--
np
--
Tradar Limited is a limited company registered in England and Wales. Registered number: 3431380.
Registered office: 20-21 Jockey's Fields, London. WC1R 4BW
-----Original Message-----
From: Konstantin Kolinko [mailto:knst.kolinko@gmail.com]
Sent: 10 April 2012 13:39
To: Tomcat Users List
Subject: Re: Problems uploading huge files >2GB to Tomcat app.
2012/4/10 Nick Porter <ni...@tradar.com>:
>
> Hi all. New to Tomcat and not a developer but I have been trying to deploy a website using Debian6, Tomcat 6.0.35 hosting the OpenCMS 8.0.1 app. The website and the OpenCMS app has a facility to upload files however I only seem to be able to upload files that are less than 2GB. I did read that there used to be a 2GB bug in Tomcat a few versions ago but that this had since been resolved.
>
> The errors I am seeing are a mixture of 400 Bad Request and 413 Request Entity too Large.
>
> I'm connecting directly to Tomcat and the app/site on the HTTP/1.1 connector, port 8080.
>
> I could be wrong but I can't see anything untoward in the logs located
> at /var/log/tomcat6/
>
> The chaps over at the OpenCMS mail list assure me that OpenCMS is capable of handling files of this size so I'm guessing my Tomcat config is in some way causing the problem.
>
> I've expanded the memory for the heap in the /etc/defaults/tomcat6
> file with;
>
> JAVA_OPTS="-Djava.awt.headless=true -Xmx512m -XX:+UseConcMarkSweepGC"
>
> ...and added the maxPostSize="0" to the HTTP connector but I'm still see these errors.
>
> Here's my connector settings from server.xml
>
> <Connector port="8080" protocol="HTTP/1.1"
> connectionTimeout="20000"
> URIEncoding="UTF-8"
> maxPostSize="0"
> redirectPort="8443" />
>
> The only other settings I found relating to the size of a POST were the packetSize attribute but believe this is only relevant to the AJP connector.
>
> I'm at a loss as to what to try next so have come here as a last resort. Can anybody give me any pointers or advice on how I can further investigate and resolve this problem?
>
1. Tomcat 6 does not handle file upload by itself (if you are using Content-Type="multipart/form-data"). It can only provide
request.getInputStream() and something else has to consume it.
Usually that will be Apache Commons Fileupload library, but you will have to check what OpenMS uses.
[NP] I'm pretty sure OpenCMS is handling it. I don't have Apache installed in my test rig and can upload smaller files.
I do not remember what default configuration of Fileupload library is.
Certainly such big files have to go into a temporary folder, but I do not remember whether it is configured by default. Tomcat startup script sets java.io.tmpdir property to point to "$CATALINA_BASE/temp".
Are you able to upload big files that are more than 512Mb, but less than 2Gb?
[NP] Yes. I've been able to upload a 1.9ish GB file but anything over 2GB and it fails.
The file upload form - does it come from OpenCMS?
[NP] Yes.
2. It is possible to configure "connectionUploadTimeout" and "disableUploadTimeout" on a Connector. I cannot say for sure how much relevant are they to those 400 and 413 errors.
[NP] I've tried these but they had no effect.
Do 400 vs 413 errors change randomly?
[NP] Yes. I can fathom no pattern to them. In fact, the only way I can actually see them is to packet trace the HTTP exchange.
3. If FORM authentication is used, it usually saves request before displaying the login form. This request is saved in memory and its size is very limited (several Kb).
If your session expires when you submit your file I think you will see an error because of that limit.
[NP] Don't think this is it. Not seeing any login issues.
4. If file size is >2Gb then the method
"response.setContentLength(int)" cannot be used. One has to use response.setHeader("Content-Length", Long.toString(length)); That is the only thing in API that I remember that is relevant to handling files > 2Gb.
[NP] Can't say on this one.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Problems uploading huge files >2GB to Tomcat app.
Posted by Konstantin Kolinko <kn...@gmail.com>.
2012/4/10 Nick Porter <ni...@tradar.com>:
>
> Hi all. New to Tomcat and not a developer but I have been trying to deploy a website using Debian6, Tomcat 6.0.35 hosting the OpenCMS 8.0.1 app. The website and the OpenCMS app has a facility to upload files however I only seem to be able to upload files that are less than 2GB. I did read that there used to be a 2GB bug in Tomcat a few versions ago but that this had since been resolved.
>
> The errors I am seeing are a mixture of 400 Bad Request and 413 Request Entity too Large.
>
> I'm connecting directly to Tomcat and the app/site on the HTTP/1.1 connector, port 8080.
>
> I could be wrong but I can't see anything untoward in the logs located at /var/log/tomcat6/
>
> The chaps over at the OpenCMS mail list assure me that OpenCMS is capable of handling files of this size so I'm guessing my Tomcat config is in some way causing the problem.
>
> I've expanded the memory for the heap in the /etc/defaults/tomcat6 file with;
>
> JAVA_OPTS="-Djava.awt.headless=true -Xmx512m -XX:+UseConcMarkSweepGC"
>
> ...and added the maxPostSize="0" to the HTTP connector but I'm still see these errors.
>
> Here's my connector settings from server.xml
>
> <Connector port="8080" protocol="HTTP/1.1"
> connectionTimeout="20000"
> URIEncoding="UTF-8"
> maxPostSize="0"
> redirectPort="8443" />
>
> The only other settings I found relating to the size of a POST were the packetSize attribute but believe this is only relevant to the AJP connector.
>
> I'm at a loss as to what to try next so have come here as a last resort. Can anybody give me any pointers or advice on how I can further investigate and resolve this problem?
>
1. Tomcat 6 does not handle file upload by itself (if you are using
Content-Type="multipart/form-data"). It can only provide
request.getInputStream() and something else has to consume it.
Usually that will be Apache Commons Fileupload library, but you will
have to check what OpenMS uses.
I do not remember what default configuration of Fileupload library is.
Certainly such big files have to go into a temporary folder, but I do
not remember whether it is configured by default. Tomcat startup
script sets java.io.tmpdir property to point to "$CATALINA_BASE/temp".
Are you able to upload big files that are more than 512Mb, but less than 2Gb?
The file upload form - does it come from OpenCMS?
2. It is possible to configure "connectionUploadTimeout" and
"disableUploadTimeout" on a Connector. I cannot say for sure how much
relevant are they to those 400 and 413 errors.
Do 400 vs 413 errors change randomly?
3. If FORM authentication is used, it usually saves request before
displaying the login form. This request is saved in memory and its
size is very limited (several Kb).
If your session expires when you submit your file I think you will see
an error because of that limit.
4. If file size is >2Gb then the method
"response.setContentLength(int)" cannot be used. One has to use
response.setHeader("Content-Length", Long.toString(length)); That is
the only thing in API that I remember that is relevant to handling
files > 2Gb.
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org