You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Andrew Milkowski <am...@unicast.com> on 2001/05/20 18:33:32 UTC

missing content-length from the header, Apache/Tomcat/AJP12 Connector spins out of control

Hi!

it will be best if I describe our configuration at first:

We are running Solaris 7, Tomcat 3.2.2 b4 (beta 4) + AJP12 Connector (Tomcat
configured with max of 500 threads), JVM: Sun 1.3.0_02 Hot Spot Native
Threads, Apache 1.3.19 (max of 1024 child processes)

We have had substantial traffic on our server (we have 7 servers load
balanced with the configuration specified above)

We found that if the incoming HTTP request has content-length missing from
the header, and the request gets channeled
via the AJP12 connector to the Tomcat and the servlet, it appears that all
subsequent request are causing AJP12 connections
to raise (All in ESTABLISHED mode) and Tomcat refuses to process any more
request , eventually eating up great deal of CPU time, we needed to do
restart and both Apache and the Tomcat to recover from this condition.

We have then decided to bypass all requests with the content-length missing
in the Ajp12ConnectionHandler.java

	int contentLength = reqA.getMimeHeaders().getIntHeader("content-length");
	if (contentLength != -1) {
		BufferedServletInputStream sis =
		    (BufferedServletInputStream)reqA.getInputStream();
		sis.setLimit(contentLength);
	 	}
+	else {
+		resA.finish();
+		socket.close();
+		return;
+	}

	    contextM.service( reqA, resA );
	    //resA.finish(); // is part of contextM !
	    socket.close();
	} catch (Exception e) {
            // XXX
	    // this isn't what we want, we want to log the problem somehow
	    System.out.println("HANDLER THREAD PROBLEM: " + e);
	    e.printStackTrace();
	}

Note that in the above socket.close() is not executed when an exception is
thrown in the service method

we are considering adding a "finally" clause to handle such a case: i.e

    contextM.service( reqA, resA );
	    //resA.finish(); // is part of contextM !
-	    socket.close();
	} catch (Exception e) {
            // XXX
	    // this isn't what we want, we want to log the problem somehow
	    System.out.println("HANDLER THREAD PROBLEM: " + e);
	    e.printStackTrace();
	}
+	} finally() {
+		if (socket != null)
+			socket.close();
+	}

----------------------------------------------------------------------------
-----------------------------

I believe that potential offending code could in the servlet application
that tries to read input stream "request.getInputStream() without the
content-length

following is what the servlet code looks like:

private int readInputStream(HttpServletRequest request) {

        int clientDayOfYear = 0;

        try {
            InputStreamReader isr=new
InputStreamReader(request.getInputStream());
            BufferedReader br=new BufferedReader(isr);
            clientDayOfYear = new
Integer(br.readLine().toString()).intValue();
	        isr.close();
	        isr=null;
            br.close();
            br=null;
        } catch (IOException e) {
            clientDayOfYear=0;
        }
        return clientDayOfYear;
    }


We know that our change to the Tomcat code is temporary at best (it allows
us to go on!) I am looking for the feedback
on the "quality" of our code change as well as maybe to a deeper reasons why
content-length missing from the header
together with request.getInputStream() would cause Apache/AJP12 to have
connections open (All in ESTABLISHED mode) and
eventually eat up all available CPU resources

Thanks for any feedback/help!



Re: missing content-length from the header, Apache/Tomcat/AJP12 Connector spins out of control

Posted by Rainer Jung <ra...@kippdata.de>.
Hi,

did you get any answers?

I would be very interested if anybody investigated your problem. If not 
consider logging a bug tu bugzilla.

Rainer Jung

At 18:33 20.05.01 , you wrote:

>Hi!
>
>it will be best if I describe our configuration at first:
>
>We are running Solaris 7, Tomcat 3.2.2 b4 (beta 4) + AJP12 Connector (Tomcat
>configured with max of 500 threads), JVM: Sun 1.3.0_02 Hot Spot Native
>Threads, Apache 1.3.19 (max of 1024 child processes)
>
>We have had substantial traffic on our server (we have 7 servers load
>balanced with the configuration specified above)
>
>We found that if the incoming HTTP request has content-length missing from
>the header, and the request gets channeled
>via the AJP12 connector to the Tomcat and the servlet, it appears that all
>subsequent request are causing AJP12 connections
>to raise (All in ESTABLISHED mode) and Tomcat refuses to process any more
>request , eventually eating up great deal of CPU time, we needed to do
>restart and both Apache and the Tomcat to recover from this condition.
>
>We have then decided to bypass all requests with the content-length missing
>in the Ajp12ConnectionHandler.java
>
>         int contentLength = 
> reqA.getMimeHeaders().getIntHeader("content-length");
>         if (contentLength != -1) {
>                 BufferedServletInputStream sis =
>                     (BufferedServletInputStream)reqA.getInputStream();
>                 sis.setLimit(contentLength);
>                 }
>+       else {
>+               resA.finish();
>+               socket.close();
>+               return;
>+       }
>
>             contextM.service( reqA, resA );
>             //resA.finish(); // is part of contextM !
>             socket.close();
>         } catch (Exception e) {
>             // XXX
>             // this isn't what we want, we want to log the problem somehow
>             System.out.println("HANDLER THREAD PROBLEM: " + e);
>             e.printStackTrace();
>         }
>
>Note that in the above socket.close() is not executed when an exception is
>thrown in the service method
>
>we are considering adding a "finally" clause to handle such a case: i.e
>
>     contextM.service( reqA, resA );
>             //resA.finish(); // is part of contextM !
>-           socket.close();
>         } catch (Exception e) {
>             // XXX
>             // this isn't what we want, we want to log the problem somehow
>             System.out.println("HANDLER THREAD PROBLEM: " + e);
>             e.printStackTrace();
>         }
>+       } finally() {
>+               if (socket != null)
>+                       socket.close();
>+       }
>
>----------------------------------------------------------------------------
>-----------------------------
>
>I believe that potential offending code could in the servlet application
>that tries to read input stream "request.getInputStream() without the
>content-length
>
>following is what the servlet code looks like:
>
>private int readInputStream(HttpServletRequest request) {
>
>         int clientDayOfYear = 0;
>
>         try {
>             InputStreamReader isr=new
>InputStreamReader(request.getInputStream());
>             BufferedReader br=new BufferedReader(isr);
>             clientDayOfYear = new
>Integer(br.readLine().toString()).intValue();
>                 isr.close();
>                 isr=null;
>             br.close();
>             br=null;
>         } catch (IOException e) {
>             clientDayOfYear=0;
>         }
>         return clientDayOfYear;
>     }
>
>
>We know that our change to the Tomcat code is temporary at best (it allows
>us to go on!) I am looking for the feedback
>on the "quality" of our code change as well as maybe to a deeper reasons why
>content-length missing from the header
>together with request.getInputStream() would cause Apache/AJP12 to have
>connections open (All in ESTABLISHED mode) and
>eventually eat up all available CPU resources
>
>Thanks for any feedback/help!
>


Re: JSR-096/Daemon/Invocation stuff...

Posted by kevin seguin <se...@motive.com>.
seems like this stuff is completely separate from tomcat...  so it seems
like it ought to go somewhere else.

just my $0.02 :)

"Pier P. Fumagalli" wrote:
> 
> I believe we're at a pretty good point right now with the development of the
> JSR-096, describing how Daemons are supposed to be working in the Java
> platform... So, even if it's not possible for me to write code on the actual
> discussion which took place on the expert group mailing list, I believe it's
> pretty safe to start writing some code that could be (in the future) used to
> achieve daemon functionality...
> 
> Now, my question is, where the hack do I put that stuff? My preference would
> be in the old jakarta-tomcat-4.0 repository in the invocation directory
> (where it was before), but since I consider myself a polite person :) :) I
> might ask before doing it... Soooo? What about it?
> 
>     Pier

JSR-096/Daemon/Invocation stuff...

Posted by "Pier P. Fumagalli" <pi...@betaversion.org>.
I believe we're at a pretty good point right now with the development of the
JSR-096, describing how Daemons are supposed to be working in the Java
platform... So, even if it's not possible for me to write code on the actual
discussion which took place on the expert group mailing list, I believe it's
pretty safe to start writing some code that could be (in the future) used to
achieve daemon functionality...

Now, my question is, where the hack do I put that stuff? My preference would
be in the old jakarta-tomcat-4.0 repository in the invocation directory
(where it was before), but since I consider myself a polite person :) :) I
might ask before doing it... Soooo? What about it?

    Pier