You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2010/12/13 20:02:39 UTC

DO NOT REPLY [Bug 50467] New: Occasional NIO connector lockups on high load

https://issues.apache.org/bugzilla/show_bug.cgi?id=50467

           Summary: Occasional NIO connector lockups on high load
           Product: Tomcat 6
           Version: 6.0.29
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: Connectors
        AssignedTo: dev@tomcat.apache.org
        ReportedBy: hugg@fasterlight.com


We've been running Tomcat 6.0.29 on FC8 2.6.21 with tens of thousands of long
polling threads which usually work fine. Every few days though we will
experience a sudden lockup of the NIO connector and it has to be restarted.
These have been seen to be accompanied by the following stack trace:

Exception in thread "http-8082-ClientPoller-0" java.lang.NullPointerException
    at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1620)
    at java.lang.Thread.run(Thread.java:662)

Looking at the source it seems the issue is likely to be a race condition where
access() is called on a null attachment, probably while it's in the process of
being cancelled:

                    while (iterator != null && iterator.hasNext()) {
                        SelectionKey sk = (SelectionKey) iterator.next();
                        KeyAttachment attachment =
(KeyAttachment)sk.attachment();
/*NPE*/                 attachment.access();
                        iterator.remove();
                        processKey(sk, attachment);
                    }//while

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


DO NOT REPLY [Bug 50467] Occasional NIO connector lockups on high load

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50467

--- Comment #4 from Mark Thomas <ma...@apache.org> 2011-01-05 08:53:41 EST ---
The null check seems reasonable to me.

I have fixed this in 7.0.x and it will be included in 7.0.6 onwards.

I have also proposed the fix for 6.0.x.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


DO NOT REPLY [Bug 50467] Occasional NIO connector lockups on high load

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50467

--- Comment #1 from Christopher Schultz <ch...@christopherschultz.net> 2010-12-14 14:05:10 EST ---
Steven, would it be possible for you to upgrade to the latest (6.0.29) Tomcat
version? I seem to recall a recent fix to the NIO connector that fixes some
threading issues, though I can't seem to find a reference for it at the moment.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


DO NOT REPLY [Bug 50467] Occasional NIO connector lockups on high load

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50467

Mark Thomas <ma...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #5 from Mark Thomas <ma...@apache.org> 2011-01-07 13:40:10 EST ---
Fixed in 6.0.x and will be included in 6.0.30 onwards.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


DO NOT REPLY [Bug 50467] Occasional NIO connector lockups on high load

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50467

--- Comment #2 from Christopher Schultz <ch...@christopherschultz.net> 2010-12-14 14:06:43 EST ---
Ooh, sorry. I misread your version number. Duh.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org


DO NOT REPLY [Bug 50467] Occasional NIO connector lockups on high load

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=50467

--- Comment #3 from Steven Hugg <hu...@fasterlight.com> 2010-12-16 17:06:55 EST ---
I haven't reproduced it, but I would imagine that inserting a Thread.sleep()
after the call to key.attach(null) in cancelledKey() might do it.

For now I have just put a null check in the above loop like so:

                        if (attachment != null)
                        {
                            attachment.access();
                            iterator.remove();
                            processKey(sk, attachment);
                        } else {
                            log.warn("NioEndpoint: Attachment was null");
                            iterator.remove();
                        }

Not sure if that is correct, but better than the alternative ;)

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org