You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Jacob Hookom <ho...@uwec.edu> on 2002/06/25 19:25:30 UTC
Large File Transfer : Coyote Connector Crash
I'm running 4.0.4 with the Coyote connector. Part of my site has to do
with large file transfer-- uploaded via page form. When large files
are uploaded (>50MB), the Coyote connector along with Tomcat will just
crash. The uploading is done through the newest Struts release.
Has anyone run into this problem? Or, is there a setting in tomcat to
widen a buffer of some sorts?
Regards,
Jake Hookom
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.371 / Virus Database: 206 - Release Date: 6/13/2002
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Large File Transfer : Coyote Connector Crash [isHexDigit]
Posted by Remy Maucherat <re...@apache.org>.
Jacob Hookom wrote:
> Remy Maucherat wrote:
>
>>Are you actualy using a POST with a URL FORM encoding ? In which case,
>
>
>>sorry, but it's a really bad idea to do that."
>
>
> Are there alternatives? The goal of this process is to ease image
> transfer to a publishing server by people who haven't a clue on how to
> utilize ftp, or their mail servers do not permit large file attachments.
Many alternatives.
The problem is that when you do the form decoding you allocate an array
of the same size as the data (since you have to keep the decoded bytes
somewhere, right). So here, it's 50M * number of simultaneous uploads =
you're possibly in trouble.
Instead, you can use either a POST without the FORM encoding, or a PUT.
From a browser, the first solution looks better.
> My implementation is the same as the upload example included with the
> struts-upload.war. File transfers of 141MB will crash the VM at 114MB
> for example. So the file is reaching the server and midway through
> 'storing it', but crashes the VM midway through.
>
> The only thing I notice is that the Coyote connector throws
> java.io.CharConversionException: isHexDigit
Any line numbers ?
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Apache mod_expires and Tomcat
Posted by Will Hartung <wi...@msoft.com>.
Hi all,
We're running Apache backed by Tomcat. The primary reason we're running
Apache is to integrate with another custom module.
We have an app deployed as a single WAR.
However, we'd like to add Expires lines to the .GIFs in the WAR.
mod_expires can be configured to flag specific mime types (i.e. image/gif),
but I'm curious as to whether mod_expires even "sees" these requests?
It's not clear to me how much Apache can intercept and deal with items being
delivered out of the WAR deployed on Tomcat.
Can Tomcat be configured to set an Expire header (essentially, we're trying
to have them "never" expire)?
We're running Tomcat 3.2.
Does Apache simply "map" the WAR that Tomcat explodes and serve directly
from it, save for the configured Servlet and JSP details, or is Tomcat
itself serving these things?
The other alternative is to split the images out completely, but that may be
a bit invasive to all of the code at this time, so I'm hoping to do it
straight from the WAR file.
Any thoughts or tips would be appreciated.
Thanx!
Best Regards,
Will Hartung
(willh@msoft.com)
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Large File Transfer : Coyote Connector Crash [isHexDigit]
Posted by Remy Maucherat <re...@apache.org>.
Craig R. McClanahan wrote:
>
> On Tue, 25 Jun 2002, Jacob Hookom wrote:
>
>
>>Date: Tue, 25 Jun 2002 16:02:48 -0500
>>From: Jacob Hookom <ho...@uwec.edu>
>>Reply-To: Tomcat Users List <to...@jakarta.apache.org>
>>To: Tomcat Users List <to...@jakarta.apache.org>
>>Subject: RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
>>
>>Remy Maucherat wrote:
>>
>>>Are you actualy using a POST with a URL FORM encoding ? In which case,
>>
>>>sorry, but it's a really bad idea to do that."
>>
>>Are there alternatives? The goal of this process is to ease image
>>transfer to a publishing server by people who haven't a clue on how to
>>utilize ftp, or their mail servers do not permit large file attachments.
>>
>>My implementation is the same as the upload example included with the
>>struts-upload.war. File transfers of 141MB will crash the VM at 114MB
>>for example. So the file is reaching the server and midway through
>>'storing it', but crashes the VM midway through.
>>
>
>
> I hope you're not configuring things to load the entire incoming file into
> memory before saving it. That's obviously going to cause you memory
> problems if you don't have enough heap space allocated in your JVM.
>
>
>>The only thing I notice is that the Coyote connector throws
>>java.io.CharConversionException: isHexDigit
>>
>
>
> As Remy suggests, it's almost impossible to do much debugging without
> seeing the stack trace.
Here's the stack trace:
java.io.CharConversionException: isHexDigit
at
org.apache.tomcat.util.buf.UDecoder.convert(UDecoder.java:124)
at org.apache.tomcat.util.buf.UDecoder.convert(UDecoder.java:87)
at
org.apache.tomcat.util.http.Parameters.processParameters(Parameters.java
:408)
at
org.apache.coyote.tomcat4.CoyoteRequest.parseRequestParameters(CoyoteReq
uest.java:1944)
at
org.apache.coyote.tomcat4.CoyoteRequest.getParameterNames(CoyoteRequest.
java:942)
at
org.apache.coyote.tomcat4.CoyoteRequest.getParameterMap(CoyoteRequest.ja
va:922)
at
org.apache.coyote.tomcat4.CoyoteRequestFacade.getParameterMap(CoyoteRequ
estFacade.java:193)
at
org.apache.catalina.core.ApplicationHttpRequest.setRequest(ApplicationHt
tpRequest.java:526)
at
org.apache.catalina.core.ApplicationHttpRequest.<init>(ApplicationHttpRe
quest.java:126)
at
org.apache.catalina.core.ApplicationDispatcher.wrapRequest(ApplicationDi
spatcher.java:918)
at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDisp
atcher.java:414)
at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispat
cher.java:355)
at
org.apache.catalina.valves.ErrorDispatcherValve.custom(ErrorDispatcherVa
lve.java:391)
at
org.apache.catalina.valves.ErrorDispatcherValve.status(ErrorDispatcherVa
lve.java:305)
at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherVa
lve.java:180)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
:170)
at
org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.ja
va:564)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:468
)
That would seem to indicate that you're getting to an error page. Going
there triggers the parameters parsing, and so it causes trouble.
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
Posted by "Craig R. McClanahan" <cr...@apache.org>.
On Tue, 25 Jun 2002, Jacob Hookom wrote:
> Date: Tue, 25 Jun 2002 18:36:02 -0500
> From: Jacob Hookom <ho...@uwec.edu>
> Reply-To: Tomcat Users List <to...@jakarta.apache.org>
> To: 'Tomcat Users List' <to...@jakarta.apache.org>
> Subject: RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
>
> Craig R. McClanahan wrote:
> > I hope you're not configuring things to load the entire incoming file
> into
> > memory before saving it. That's obviously going to cause you memory
> > problems if you don't have enough heap space allocated in your JVM.
>
> This may be possibly more of a struts implementation question, but if I
> do follow the FormAction with a Bean that has an attribute of type
> org.apache.struts.upload.FormFile, then within the actual
> ResourceInsertAction.do:
>
Definitely sounds like it might be a Struts configuration or usage issue.
You'd probably do better asking folks on STRUTS-USER how they deal with
very large file uploads like this.
Craig
> // get FormFile
> FormFile file = form.getFile();
>
> /* contents below actually within try/catch to close streams */
>
> // setup streams
> baos = new ByteArrayOutputStream();
> stream = file.getInputStream(); //struts FormFile
> bos = new FileOutputStream(newFile); //desired path
>
> // copy contents from FormFile stream to newFile path
> int bytesRead = 0;
> byte[] buffer = new byte[8192];
> while ((bytesRead = stream.read(buffer, 0, 8192)) != -1)
> {
> bos.write(buffer, 0, bytesRead);
> }
>
> Somewhat unfamiliar with the java.io lib, does this essentially copy
> into memory? Memory usage seems to stay consistent throughout the
> transfer. When the VM does crash, there is no apparent spike in
> processor or memory usage.
>
> Would it be ideal for me to generate a separate servlet to handle file
> uploads as my implementation of Struts may be the problem?
>
> Best Regards,
> Jake Hookom
> Student, University of Wisconsin
>
> -----Original Message-----
> From: Craig R. McClanahan [mailto:craigmcc@apache.org]
> Sent: Tuesday, June 25, 2002 6:09 PM
> To: Tomcat Users List
> Subject: RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
>
>
>
> On Tue, 25 Jun 2002, Jacob Hookom wrote:
>
> > Date: Tue, 25 Jun 2002 16:02:48 -0500
> > From: Jacob Hookom <ho...@uwec.edu>
> > Reply-To: Tomcat Users List <to...@jakarta.apache.org>
> > To: Tomcat Users List <to...@jakarta.apache.org>
> > Subject: RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
> >
> > Remy Maucherat wrote:
> > > Are you actualy using a POST with a URL FORM encoding ? In which
> case,
> >
> > > sorry, but it's a really bad idea to do that."
> >
> > Are there alternatives? The goal of this process is to ease image
> > transfer to a publishing server by people who haven't a clue on how to
> > utilize ftp, or their mail servers do not permit large file
> attachments.
> >
> > My implementation is the same as the upload example included with the
> > struts-upload.war. File transfers of 141MB will crash the VM at 114MB
> > for example. So the file is reaching the server and midway through
> > 'storing it', but crashes the VM midway through.
> >
>
> I hope you're not configuring things to load the entire incoming file
> into
> memory before saving it. That's obviously going to cause you memory
> problems if you don't have enough heap space allocated in your JVM.
>
> > The only thing I notice is that the Coyote connector throws
> > java.io.CharConversionException: isHexDigit
> >
>
> As Remy suggests, it's almost impossible to do much debugging without
> seeing the stack trace.
>
> > Thanks,
> > Jake
> >
>
> Craig
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
> ---
> Incoming mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.371 / Virus Database: 206 - Release Date: 6/13/2002
>
>
> ---
> Outgoing mail is certified Virus Free.
> Checked by AVG anti-virus system (http://www.grisoft.com).
> Version: 6.0.371 / Virus Database: 206 - Release Date: 6/13/2002
>
>
>
> --
> To unsubscribe, e-mail: <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
Posted by Jacob Hookom <ho...@uwec.edu>.
Craig R. McClanahan wrote:
> I hope you're not configuring things to load the entire incoming file
into
> memory before saving it. That's obviously going to cause you memory
> problems if you don't have enough heap space allocated in your JVM.
This may be possibly more of a struts implementation question, but if I
do follow the FormAction with a Bean that has an attribute of type
org.apache.struts.upload.FormFile, then within the actual
ResourceInsertAction.do:
// get FormFile
FormFile file = form.getFile();
/* contents below actually within try/catch to close streams */
// setup streams
baos = new ByteArrayOutputStream();
stream = file.getInputStream(); //struts FormFile
bos = new FileOutputStream(newFile); //desired path
// copy contents from FormFile stream to newFile path
int bytesRead = 0;
byte[] buffer = new byte[8192];
while ((bytesRead = stream.read(buffer, 0, 8192)) != -1)
{
bos.write(buffer, 0, bytesRead);
}
Somewhat unfamiliar with the java.io lib, does this essentially copy
into memory? Memory usage seems to stay consistent throughout the
transfer. When the VM does crash, there is no apparent spike in
processor or memory usage.
Would it be ideal for me to generate a separate servlet to handle file
uploads as my implementation of Struts may be the problem?
Best Regards,
Jake Hookom
Student, University of Wisconsin
-----Original Message-----
From: Craig R. McClanahan [mailto:craigmcc@apache.org]
Sent: Tuesday, June 25, 2002 6:09 PM
To: Tomcat Users List
Subject: RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
On Tue, 25 Jun 2002, Jacob Hookom wrote:
> Date: Tue, 25 Jun 2002 16:02:48 -0500
> From: Jacob Hookom <ho...@uwec.edu>
> Reply-To: Tomcat Users List <to...@jakarta.apache.org>
> To: Tomcat Users List <to...@jakarta.apache.org>
> Subject: RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
>
> Remy Maucherat wrote:
> > Are you actualy using a POST with a URL FORM encoding ? In which
case,
>
> > sorry, but it's a really bad idea to do that."
>
> Are there alternatives? The goal of this process is to ease image
> transfer to a publishing server by people who haven't a clue on how to
> utilize ftp, or their mail servers do not permit large file
attachments.
>
> My implementation is the same as the upload example included with the
> struts-upload.war. File transfers of 141MB will crash the VM at 114MB
> for example. So the file is reaching the server and midway through
> 'storing it', but crashes the VM midway through.
>
I hope you're not configuring things to load the entire incoming file
into
memory before saving it. That's obviously going to cause you memory
problems if you don't have enough heap space allocated in your JVM.
> The only thing I notice is that the Coyote connector throws
> java.io.CharConversionException: isHexDigit
>
As Remy suggests, it's almost impossible to do much debugging without
seeing the stack trace.
> Thanks,
> Jake
>
Craig
--
To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail:
<ma...@jakarta.apache.org>
---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.371 / Virus Database: 206 - Release Date: 6/13/2002
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.371 / Virus Database: 206 - Release Date: 6/13/2002
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
Posted by "Craig R. McClanahan" <cr...@apache.org>.
On Tue, 25 Jun 2002, Jacob Hookom wrote:
> Date: Tue, 25 Jun 2002 16:02:48 -0500
> From: Jacob Hookom <ho...@uwec.edu>
> Reply-To: Tomcat Users List <to...@jakarta.apache.org>
> To: Tomcat Users List <to...@jakarta.apache.org>
> Subject: RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
>
> Remy Maucherat wrote:
> > Are you actualy using a POST with a URL FORM encoding ? In which case,
>
> > sorry, but it's a really bad idea to do that."
>
> Are there alternatives? The goal of this process is to ease image
> transfer to a publishing server by people who haven't a clue on how to
> utilize ftp, or their mail servers do not permit large file attachments.
>
> My implementation is the same as the upload example included with the
> struts-upload.war. File transfers of 141MB will crash the VM at 114MB
> for example. So the file is reaching the server and midway through
> 'storing it', but crashes the VM midway through.
>
I hope you're not configuring things to load the entire incoming file into
memory before saving it. That's obviously going to cause you memory
problems if you don't have enough heap space allocated in your JVM.
> The only thing I notice is that the Coyote connector throws
> java.io.CharConversionException: isHexDigit
>
As Remy suggests, it's almost impossible to do much debugging without
seeing the stack trace.
> Thanks,
> Jake
>
Craig
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Large File Transfer : Coyote Connector Crash [isHexDigit]
Posted by Jacob Hookom <ho...@uwec.edu>.
Remy Maucherat wrote:
> Are you actualy using a POST with a URL FORM encoding ? In which case,
> sorry, but it's a really bad idea to do that."
Are there alternatives? The goal of this process is to ease image
transfer to a publishing server by people who haven't a clue on how to
utilize ftp, or their mail servers do not permit large file attachments.
My implementation is the same as the upload example included with the
struts-upload.war. File transfers of 141MB will crash the VM at 114MB
for example. So the file is reaching the server and midway through
'storing it', but crashes the VM midway through.
The only thing I notice is that the Coyote connector throws
java.io.CharConversionException: isHexDigit
Thanks,
Jake
--
To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail:
<ma...@jakarta.apache.org>
---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.371 / Virus Database: 206 - Release Date: 6/13/2002
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.371 / Virus Database: 206 - Release Date: 6/13/2002
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Large File Transfer : Coyote Connector Crash
Posted by Remy Maucherat <re...@apache.org>.
Jacob Hookom wrote:
> I'm running 4.0.4 with the Coyote connector. Part of my site has to do
> with large file transfer-- uploaded via page form. When large files
> are uploaded (>50MB), the Coyote connector along with Tomcat will just
> crash. The uploading is done through the newest Struts release.
>
> Has anyone run into this problem? Or, is there a setting in tomcat to
> widen a buffer of some sorts?
If the VM crashes, then it's not Coyote's fault.
Coyote handles uploads very well in general. However, it depends then on
how the uploads are handled.
Are you actualy using a POST with a URL FORM encoding ? In which case,
sorry, but it's a really bad idea to do that.
Remy
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>