You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by HU E <E....@alcatel-lucent.com> on 2014/02/18 10:31:18 UTC

NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

Dear all,

During performance and stability run, we encountered one tomcat issue, i.e. sometimes a NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket. This issue was happened very rarely.

Following is the stack:
PRS0: Http response exception : java.lang.NullPointerException
org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:463)
org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:800)
org.apache.coyote.http11.InternalNioOutputBuffer.endRequest(InternalNioOutputBuffer.java:412)
org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1097)
org.apache.coyote.Response.action(Response.java:183)
org.apache.coyote.Response.finish(Response.java:305)
org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:284)
org.apache.catalina.connector.CoyoteWriter.close(CoyoteWriter.java:108)
....

The tomcat version is 6.0.35. I checked the source code, the exception occurred at line 463 of InternalNioOutputBuffer.java, which is marked in red.

private synchronized int writeToSocket(ByteBuffer bytebuffer, boolean block, boolean flip) throws IOException {
......
......
try {
            written = getSelectorPool().write(bytebuffer, socket, selector, writeTimeout, block,lastWrite);
            //make sure we are flushed
            do {
                if (socket.flush(true,selector,writeTimeout,lastWrite)) break;
            }while ( true );
        }finally {
            if ( selector != null ) getSelectorPool().put(selector);
        }

......
......
}

Before executing line 460 (i.e. written = getSelectorPool().write(bytebuffer, socket, selector, writeTimeout, block,lastWrite)),socket object should not be null.

So I guess socket is recycled after executing line 460 so that NullPointException is thrown. But I am not sure why the socket is recycled. Client closes the TCP connection? TCP write timeout? TCP is reset?

I appreciate your help and thanks for your time in advance.

Thanks
Eric

Re: NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Eric,

On 2/26/14, 3:39 AM, HU E wrote:
> Chris, Mark, thanks for your suggestion. After setting the system 
> property org.apache.catalina.connector.RECYCLE_FACADES to true,
> this issue disappeared.

That suggests that your application is making a mistake somewhere. You
might want to fix that, as recycling facades gives you a bit of a
performance drop.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTD6NvAAoJEBzwKT+lPKRYeJAP+wRF5tzw6A6LaT0aqxS5zzdm
IsMdyKhT+IiqJNtUDuTZ4EkXiL6iP21pexigiPmPj4RBsetvzqKZBBv9ZYdpjuQE
7Ei8K1UmBndbFEbCiAYyb8UizArplxy0Dysmn6JjsG1wCqfhXGArLBVOJbe2YRWt
d39arANbV6Y14U5PCKo3zH3oN36FFSH0lbgA6CAqPkPyD843K2xSiIRGjPmySnXY
R8OkowijIr5tbKDJfe+vl6it04XA2m0pO0iVVNNIMHZkU3/zmptNIgw1n2cVT18/
kRMONFPyakZFzIXPbG32vBZWTjiHSzUpRbUBc+0NGUER5dYlkzF57UtmKk4inG7Y
wYEXVH8Wd+pdWD/NCIm9+9WM47U+fkVKOtOzAW3dn4psZ6GbrzNQ2X2oAKZakgH5
RRvPg804cgE3xEy8HsIqOO1Zx9xPzmf11LG6T33Xz9OZ7xp9ToQuJKVsYKzReMd4
I0c5nmcLnxn1K7v95YlkafOOuIopW3lnoHEbwK8LW9tAQ38ylNIp+GDErjLQbKin
alV7EAOJ4ZAZjqPizFWUJDEh3TJmOul8C9DH033K+32aZdWoOY5R+nL4+1rGelOw
YpVDrgaedikvG8f2awgkovBjZ/HmmFVBxVB63Vp/IdCWEJOEIkz02ZqWSYMvllN3
qPy/JXQPsgMwnwXWx00n
=4bJx
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

RE: NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

Posted by HU E <E....@alcatel-lucent.com>.
Chris, Mark, thanks for your suggestion. After setting the system property org.apache.catalina.connector.RECYCLE_FACADES to true, this issue disappeared.

-----Original Message-----
From: Christopher Schultz [mailto:chris@christopherschultz.net] 
Sent: Saturday, February 22, 2014 2:13 AM
To: Tomcat Users List
Subject: Re: NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Eric,

On 2/19/14, 2:40 AM, HU E wrote:
> Thanks for your response. I will try your suggestion, i.e. setting
> the system property org.apache.catalina.connector.RECYCLE_FACADES
> to true(I guess I can set this in catalina.properties).

I would set it as a command-line argument, probably in CATALINA_OPTS,
assuming that you are using scripts to launch Tomcat. I'd do it this
way because it's more obvious that it's being set. As I never modify
catalina.properties, it would be one of the last places I'd ever check
for customizations in case of a problem I was having.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTB5ckAAoJEBzwKT+lPKRYjX0P+wTpVmj+EKB0RBAXMckTaHf6
TXWTtl8kgcDkeuGTPcng3Hia4+o96zJ50XqdztgWudiTjqK6Q8RDPKjcr9IcVQCd
Uo+iT1ajlMvAtPSZqvw5bb0bt7hVaNB505WPE/9JdeHTTNAgTis0Q0muf/WhJQQU
G25Va2XOuOBnkU7swkkQABMSqW7pBrf+ojqhEV3xZISYGxCJER4SIvLDzaT/tC2K
NRQFlWWApxJwygrbzmgZfPV+STqvLfAOBEvuLH+ai4HXGnRkxlahQFhsubgDCgm9
1DE/aUxPLr/haufkNNmYZyDEr5NiDR3pOZaOuqwPU9KkVbFo8NlcqHKVNL+osRHN
oQYsv6KDoF1L1c9XACOx5O8Siecb5R+rK21ugRoccG2d3yB2gKiCF17Th23FJYbC
XPfGiJGMwm3zeKHKxgVwCu72+keaWN4tiXxA0Z/P1LP42+qzLwBCoEDLiZjaMQHq
BX63cw+fWeNSJ/Sp37pNyO3wPypguVwSp2wWO+YmfQwwp89nUEct+1qNfTIOpuqC
XTNLPzAlPNLACOp1yrGXYXabdhuhEEUPtE9wqHbu1XWXI6uuN796S0VH+Ffq+Vit
t5kf5uMNR2WyWCwcCN46r9rJFSNlYlN5oqE3LkdalkZJ9wEtUjsprWyfOANQJbqv
GzgdskRf07kucKSmorgw
=wagN
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Eric,

On 2/19/14, 2:40 AM, HU E wrote:
> Thanks for your response. I will try your suggestion, i.e. setting
> the system property org.apache.catalina.connector.RECYCLE_FACADES
> to true(I guess I can set this in catalina.properties).

I would set it as a command-line argument, probably in CATALINA_OPTS,
assuming that you are using scripts to launch Tomcat. I'd do it this
way because it's more obvious that it's being set. As I never modify
catalina.properties, it would be one of the last places I'd ever check
for customizations in case of a problem I was having.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJTB5ckAAoJEBzwKT+lPKRYjX0P+wTpVmj+EKB0RBAXMckTaHf6
TXWTtl8kgcDkeuGTPcng3Hia4+o96zJ50XqdztgWudiTjqK6Q8RDPKjcr9IcVQCd
Uo+iT1ajlMvAtPSZqvw5bb0bt7hVaNB505WPE/9JdeHTTNAgTis0Q0muf/WhJQQU
G25Va2XOuOBnkU7swkkQABMSqW7pBrf+ojqhEV3xZISYGxCJER4SIvLDzaT/tC2K
NRQFlWWApxJwygrbzmgZfPV+STqvLfAOBEvuLH+ai4HXGnRkxlahQFhsubgDCgm9
1DE/aUxPLr/haufkNNmYZyDEr5NiDR3pOZaOuqwPU9KkVbFo8NlcqHKVNL+osRHN
oQYsv6KDoF1L1c9XACOx5O8Siecb5R+rK21ugRoccG2d3yB2gKiCF17Th23FJYbC
XPfGiJGMwm3zeKHKxgVwCu72+keaWN4tiXxA0Z/P1LP42+qzLwBCoEDLiZjaMQHq
BX63cw+fWeNSJ/Sp37pNyO3wPypguVwSp2wWO+YmfQwwp89nUEct+1qNfTIOpuqC
XTNLPzAlPNLACOp1yrGXYXabdhuhEEUPtE9wqHbu1XWXI6uuN796S0VH+Ffq+Vit
t5kf5uMNR2WyWCwcCN46r9rJFSNlYlN5oqE3LkdalkZJ9wEtUjsprWyfOANQJbqv
GzgdskRf07kucKSmorgw
=wagN
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

RE: NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

Posted by HU E <E....@alcatel-lucent.com>.
Mark,

Thanks for your response. I will try your suggestion, i.e. setting the system property
org.apache.catalina.connector.RECYCLE_FACADES to true(I guess I can set this in catalina.properties).

Thanks
Eric

-----Original Message-----
From: Mark Thomas [mailto:markt@apache.org] 
Sent: Tuesday, February 18, 2014 6:30 PM
To: Tomcat Users List
Subject: Re: NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

On 18/02/2014 09:31, HU E wrote:

<snip/>

> PRS0: Http response exception : java.lang.NullPointerException
> org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:463)
> org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:800)
> org.apache.coyote.http11.InternalNioOutputBuffer.endRequest(InternalNioOutputBuffer.java:412)
> org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1097)
> org.apache.coyote.Response.action(Response.java:183)
> org.apache.coyote.Response.finish(Response.java:305)
> org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:284)
> org.apache.catalina.connector.CoyoteWriter.close(CoyoteWriter.java:108)
> ....
> 
> The tomcat version is 6.0.35.

<snip/>

> So I guess socket is recycled after executing line 460 so that NullPointException is thrown. But I am not sure why the socket is recycled. Client closes the TCP connection? TCP write timeout? TCP is reset?

It looks like there might be two threads trying to use the same socket
at the same time - that shouldn't happen.

It might be a Tomcat bug but a  common cause is if the application
retains a reference to the Response, the Writer or the OutputStream
between requests - e.g. in the session or in a Filter.

> I appreciate your help and thanks for your time in advance.

The first thing to do is to carefully check your code for references
being retained across requests that shouldn't be.

Setting the system property
org.apache.catalina.connector.RECYCLE_FACADES to true may help.

For further advice, search the archives for the above system property
and take a look at the related threads.

If you are able to create a repeatable test case then please open a bug
report and provide details of how to recreate this from a clean install
of the latest stable 6.0.x release.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: NullPointerException is thrown when executing org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket

Posted by Mark Thomas <ma...@apache.org>.
On 18/02/2014 09:31, HU E wrote:

<snip/>

> PRS0: Http response exception : java.lang.NullPointerException
> org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:463)
> org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:800)
> org.apache.coyote.http11.InternalNioOutputBuffer.endRequest(InternalNioOutputBuffer.java:412)
> org.apache.coyote.http11.Http11NioProcessor.action(Http11NioProcessor.java:1097)
> org.apache.coyote.Response.action(Response.java:183)
> org.apache.coyote.Response.finish(Response.java:305)
> org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:284)
> org.apache.catalina.connector.CoyoteWriter.close(CoyoteWriter.java:108)
> ....
> 
> The tomcat version is 6.0.35.

<snip/>

> So I guess socket is recycled after executing line 460 so that NullPointException is thrown. But I am not sure why the socket is recycled. Client closes the TCP connection? TCP write timeout? TCP is reset?

It looks like there might be two threads trying to use the same socket
at the same time - that shouldn't happen.

It might be a Tomcat bug but a  common cause is if the application
retains a reference to the Response, the Writer or the OutputStream
between requests - e.g. in the session or in a Filter.

> I appreciate your help and thanks for your time in advance.

The first thing to do is to carefully check your code for references
being retained across requests that shouldn't be.

Setting the system property
org.apache.catalina.connector.RECYCLE_FACADES to true may help.

For further advice, search the archives for the above system property
and take a look at the related threads.

If you are able to create a repeatable test case then please open a bug
report and provide details of how to recreate this from a clean install
of the latest stable 6.0.x release.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org