You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2016/04/29 17:04:19 UTC

[Bug 59398] New: org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

            Bug ID: 59398
           Summary: org.apache.catalina.connector.CoyoteWriter.close
                    throws org.apache.tomcat.jni.Error
           Product: Tomcat 8
           Version: 8.0.28
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Catalina
          Assignee: dev@tomcat.apache.org
          Reporter: james.silk@metaswitch.com

(This is the first bug I've raised in Tomcat so apologies if I've missed
something out!)

In responding to a request in tomcat using a HttpServletResponse object,
xiResponse, our server create a JSON string to send back and writes it to the
response with the following code:

...
      writer = xiResponse.getWriter();
      writer.write(jsonToSend);
      writer.close();
...

Which throws the following exception:

org.apache.tomcat.jni.Error: 88: Socket operation on non-socket
at org.apache.tomcat.jni.Socket.sendbb(Native Method)
at
org.apache.coyote.http11.InternalAprOutputBuffer.writeToSocket(InternalAprOutputBuffer.java:287)
at
org.apache.coyote.http11.InternalAprOutputBuffer.writeToSocket(InternalAprOutputBuffer.java:265)
at
org.apache.coyote.http11.InternalAprOutputBuffer.flushBuffer(InternalAprOutputBuffer.java:213)
at
org.apache.coyote.http11.AbstractOutputBuffer.endRequest(AbstractOutputBuffer.java:378)
at
org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:726)
at org.apache.coyote.Response.action(Response.java:177)
at org.apache.catalina.connector.OutputBuffer.close(OutputBuffer.java:306)
at org.apache.catalina.connector.CoyoteWriter.close(CoyoteWriter.java:108)
at dcl.mwp.servlet.EventProcessor.writeResponse(EventProcessor.java:684)
at dcl.mwp.servlet.EventProcessor.access$100(EventProcessor.java:53)
at
dcl.mwp.servlet.EventProcessor$AsyncListener.commandCompleted(EventProcessor.java:1129)
at
dcl.mwp.cstasoap.CSTASOAPCommandFactory$CSTASOAPAsyncCommand$CSTAAxis2Callback.onMessage(CSTASOAPCommandFactory.java:1754)
at
org.apache.axis2.description.OutInAxisOperationClient$NonBlockingInvocationWorker.run(OutInAxisOperation.java:471)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)

(Note that dcl.mwp.servlet.EventProcessor.writeResponse is the writer.close()
call above)

Catching this in a debugger shows that the writer.close() call failed as the
socket we're trying to close is '0' implying that something else has already
closed the socket. From this I think the exception thrown by sendbb is
perfectly valid but it shouldn't make it all the way up to the print writer.

>From the PrintWriter docs there should be no exception thrown by this method
and more over 'Closing a previously closed stream has no effect'.

Getting network capture from this event shows that the data we're trying to
send has been sent over the wire and the far end sent a 'fin' on the TCP
connection. Moving the writer.close() call above the writer.write() call
results in the same exception, indicating that the client is not responsible
for closing this under the feet of tomcat.

I believe something inside tomcat is recycling this socket under our feet but
even so I don't believe the writer.close call should fail in this way.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 59398] org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

Mark Thomas <ma...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|REOPENED                    |RESOLVED

--- Comment #3 from Mark Thomas <ma...@apache.org> ---
This looks like application mis-behaviour to me: continuing to access the
request/response object (or objects obtained from them) after processing has
completed for the original request/response pair. Given what can go wrong in
that case, an Exception is the least of your worries.

If you can provide a simple as possible test case that demonstrates this issue
without the application behaving as I describe above then please do re-open
this.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 59398] org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

james.silk@metaswitch.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |james.silk@metaswitch.com

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 59398] org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

--- Comment #6 from Remy Maucherat <re...@apache.org> ---
I can sympathize since a lot of the APR code is synced now, but this is really
wrong behavior.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 59398] org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

Remy Maucherat <re...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|NEW                         |RESOLVED

--- Comment #1 from Remy Maucherat <re...@apache.org> ---
This looks like a combination of using uncontrolled async processing (the
Servlet 3.0 specification specified that) and APR, which isn't very nice when
something bad happens, but it can't really be fixed.

Since you're using Tomcat 8.0, you can use the default NIO connector which will
give reasonable scalability and hopefully better edge case handling.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 59398] org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

james.silk@metaswitch.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|INVALID                     |---
             Status|RESOLVED                    |REOPENED

--- Comment #4 from james.silk@metaswitch.com ---
So I've done some more digging into this and the issue is that
org.apache.coyote.http11.InternalAprOutputBuffer.recycle is not synchronized.
This leaves the door open to the bytebuffer and socket being reset while data
is being written to the socket by
org.apache.coyote.http11.InternalAprOutputBuffer.writeToSocket, which is
synchronized, and this is what was biting me. So I propose the following fix:

--- tomcat-8-28-src/java/org/apache/coyote/http11/InternalAprOutputBuffer.java
(revision 10252)
+++ tomcat-8-28-src/java/org/apache/coyote/http11/InternalAprOutputBuffer.java
(working copy)
@@ -105,7 +105,7 @@
      * connection.
      */
     @Override
-    public void recycle() {
+    public synchronized void recycle() {

         super.recycle();


Note that I've checked and this method is not synchronized in tomcat 8.0.33 or
8.5 but I believe it should be. This is the first patch I've tried to submit to
tomcat so apologies if there is some process I've not followed (and if this is
the case please point me at it and I'll be happy to follow it!).

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 59398] org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

james.silk@metaswitch.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |---

--- Comment #2 from james.silk@metaswitch.com ---
Thanks for the suggestion. Moving to the NIO connector does get us round this
issue but we hit https://bz.apache.org/bugzilla/show_bug.cgi?id=58646 and while
we could upgrade to tomcat 8.0.33 to fix this we're nervous about switching
from the APR connector to NIO connectors as we moved to APR for performance
reasons and at the stage we're at in our release cycle we can't change this. 

It does feel like there is a fairly straight forward issue in tomcat though as
calling close on a PrintWriter twice should succeed, and shouldn't throw an
exception, as this is the documented behaviour.

(Also I don't feel like the fact that this is difficult to fix makes it
'invalid'... possibly it won't get fixed but we're not doing anything that
shouldn't be supported in tomcat as it's currently documented)

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


[Bug 59398] org.apache.catalina.connector.CoyoteWriter.close throws org.apache.tomcat.jni.Error

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59398

Mark Thomas <ma...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |INVALID
             Status|REOPENED                    |RESOLVED

--- Comment #5 from Mark Thomas <ma...@apache.org> ---
Section 2.3.3.4 of the Servlet specification is explicit that thread safe
access to container managed objects is the responsibility of the caller. This
includes the Writer.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org