You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2011/10/31 14:53:44 UTC

BufferOverflowException in AjpNioProcessor

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

All,

I've only seen it once so far since switching-over to the NIO AJP
connector a few months back, but I figured I'd report it since I saw
it. I'll see if I can reproduce it if possible. I'm using a standard,
vanilla synchronized request.

Configuration:

$ java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode)
Tomcat 7.0.22

<Connector> configuration:

    <Connector port="8215"
               redirectPort="443"
               protocol="org.apache.coyote.ajp.AjpNioProtocol"
               URIEncoding="UTF-8" executor="tomcatThreadPool"
               requiredSecret="foo" />

Complete stack trace:

java.nio.BufferOverflowException
        at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:165)
        at
org.apache.coyote.ajp.AjpNioProcessor.output(AjpNioProcessor.java:281)
        at
org.apache.coyote.ajp.AbstractAjpProcessor$SocketOutputBuffer.doWrite(AbstractAjpProcessor.java:1081)
        at org.apache.coyote.Response.doWrite(Response.java:533)
        at
org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:368)
        at
org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:437)
        at
org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:321)
        at
org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:276)
        at
org.apache.catalina.connector.CoyoteWriter.close(CoyoteWriter.java:113)
        at
org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:422)
        at
org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:329)
        at
org.apache.struts.chain.commands.servlet.PerformForward.handleAsForward(PerformForward.java:113)
        at
org.apache.struts.chain.commands.servlet.PerformForward.perform(Perfo
rmForward.java:96)
        at
org.apache.struts.chain.commands.AbstractPerformForward.execute(Abstr
actPerformForward.java:54)
        at
org.apache.struts.chain.commands.ActionCommandBase.execute(ActionComm
andBase.java:51)
        at
org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:191)
        at
org.apache.commons.chain.generic.LookupCommand.execute(LookupCommand.
java:305)
        at
org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:191)
        at
org.apache.struts.chain.ComposableRequestProcessor.process(Composable
RequestProcessor.java:283)
        at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:191
3)
        at
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
icationFilterChain.java:304)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
ilterChain.java:210)
        at
org.childhealthcare.diagnosis.servlet.CredentialFilter.doFilter(Crede
ntialFilter.java:229)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
icationFilterChain.java:243)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationF
ilterChain.java:210)
        at
org.securityfilter.filter.SecurityFilter.doFilter(SecurityFilter.java
:231)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
icationFilterChain.java:243)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at
org.childhealthcare.diagnosis.servlet.BrokenLocaleFilter.doFilter(BrokenLocaleFilter.java:115)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at
org.childhealthcare.diagnosis.servlet.EncodingFilter.doFilter(EncodingFilter.java:64)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at
org.childhealthcare.diagnosis.servlet.ClientPreferencesFilter.doFilter(ClientPreferencesFilter.java:48)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at
org.childhealthcare.diagnosis.servlet.HttpResponseSplittingPreventionFilter.doFilter(HttpResponseSplittingPreventionFilter.java:49)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
        at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
        at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:405)
        at
org.apache.coyote.ajp.AjpNioProcessor.process(AjpNioProcessor.java:184)
        at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:515)
        at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1550)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6uqGgACgkQ9CaO5/Lv0PAecACbBOse4maAAF1M4agxwkJlouNN
lwYAn28hTza9mGdSwho1YezuhEL4TFxV
=eN1q
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: BufferOverflowException in AjpNioProcessor

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Konstantin,

On 10/31/2011 12:19 PM, Konstantin Kolinko wrote:
> 2011/10/31 Christopher Schultz <ch...@christopherschultz.net>:
>> 
>> PS Is there anything to be done about connector state after an
>> OOME? Chuck contends that OOMEs aren't certain death for an
>> application and that recovery is possible.
> 
> The problem is that with OOME you do not know what exactly remained
> in the broken state.
> 
> Generally, the thread that encountered any VirtualMachineError
> will rethrow the error up the whole stack die. As far as I remember
> my TC7 testing several months ago, for request processor threads it
> is not fatal, as new processors will be created.

The processors might be re-created, but some of the other shared,
re-usable resources (buffers?) might be re-used and in some kind of
broken state.

In the case I observed, here, can you think of a condition under which
the buffer would be left in a broken state and not re-set on a
subsequent request? I wonder if Tomcat could be slightly more
resilient under such circumstances.

Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6u5qcACgkQ9CaO5/Lv0PAqqACdFf5n3/AtG99xOfRwkBfv7671
vqYAn2b73wjpcBqKpgALGtPTQMZ/CIAi
=rOpJ
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: BufferOverflowException in AjpNioProcessor

Posted by Konstantin Kolinko <kn...@gmail.com>.
2011/10/31 Christopher Schultz <ch...@christopherschultz.net>:
>
> PS Is there anything to be done about connector state after an OOME?
> Chuck contends that OOMEs aren't certain death for an application and
> that recovery is possible.

The problem is that with OOME you do not know what exactly remained in
the broken state.

Generally, the thread that encountered any VirtualMachineError will
rethrow the error up the whole stack die. As far as I remember my TC7
testing several months ago, for request processor threads it is not
fatal, as new processors will be created.

There might be other threads (e.g. I think it applies to the
background tasks thread) where thread death can be more fatal.

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: BufferOverflowException in AjpNioProcessor

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

All,

On 10/31/2011 9:53 AM, Christopher Schultz wrote:
> I've only seen it once so far since switching-over to the NIO AJP 
> connector a few months back, but I figured I'd report it since I
> saw it.

I'm sure this was a fluke. After playing with the request that appears
to have caused this problem, I looked further back in my log files and
found a series of OOMEs which are almost certainly the source of the
problem.

Sorry for the noise.

- -chris

PS Is there anything to be done about connector state after an OOME?
Chuck contends that OOMEs aren't certain death for an application and
that recovery is possible.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6urSgACgkQ9CaO5/Lv0PCn0QCeJJ8kNYI5HNLxQPWwSPc3poLP
KeQAnAupcjJ5b2tCAJyppLPYhkjVg1a2
=WIxB
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org