You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Germán Blanco (JIRA)" <ji...@apache.org> on 2015/11/24 15:41:10 UTC

[jira] [Commented] (ZOOKEEPER-2044) CancelledKeyException in zookeeper 3.4.5

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024611#comment-15024611 ] 

Germán Blanco commented on ZOOKEEPER-2044:
------------------------------------------

This is also visible in 3.4.6.
The fix seems quite straightforward. In NIOServerCnxn.java there seems to be a missing "sk.isValid()" before "sk.interestOps()":
{code:title=NIOServerCnxn.java|borderStyle=solid}
          if (bb != ServerCnxnFactory.closeConn) {
                 // We check if write interest here because if it is NOT set,
                 // nothing is queued, so we can try to send the buffer right
                 // away without waking up the selector
                 if ((sk.interestOps() & SelectionKey.OP_WRITE) == 0) { // <= SHOULD BE HERE
                    try \{
                        sock.write(bb);
                     } catch (IOException e) {
                         // we are just doing best effort right now
                     }
                 }
                 // if there is nothing left to send, we are done
                 if (bb.remaining() == 0) {
                     packetSent();
                     return;
                 }
             }
 
             synchronized(this.factory){
                 sk.selector().wakeup();
                 if (LOG.isTraceEnabled()) {
                     LOG.trace("Add a buffer to outgoingBuffers, sk " + sk
                             + " is valid: " + sk.isValid());
                 }
                 outgoingBuffers.add(bb);
                 if (sk.isValid()) {// <= AS IT IS IN HERE
                     sk.interestOps(sk.interestOps() | SelectionKey.OP_WRITE);
                 }
             }
{code}

> CancelledKeyException in zookeeper 3.4.5
> ----------------------------------------
>
>                 Key: ZOOKEEPER-2044
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2044
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: server
>    Affects Versions: 3.4.6
>         Environment: Red Hat Enterprise Linux Server release 6.2
>            Reporter: shamjith antholi
>            Assignee: Germán Blanco
>            Priority: Critical
>             Fix For: 3.4.7
>
>
> I am getting cancelled key exception in zookeeper (version 3.4.5). Please see the log below. When this error is thrown, the connected solr shard is going down by giving the error "Failed to index metadata in Solr,StackTrace=SolrError: HTTP status 503.Reason: {"responseHeader":{"status":503,"QTime":204},"error":{"msg":"ClusterState says we are the leader, but locally we don't think so","code":503"  and ultimately the current activity is going down. Could you please give a solution for this ?
> Zookeper log 
> ----------------------------------------------------------
> 2014-09-16 02:58:47,799 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@832] - Client attempting to renew session 0x24868e7ca980003 at /172.22.0.5:58587
> 2014-09-16 02:58:47,800 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:Learner@107] - Revalidating client: 0x24868e7ca980003
> 2014-09-16 02:58:47,802 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@588] - Invalid session 0x24868e7ca980003 for client /172.22.0.5:58587, probably expired
> 2014-09-16 02:58:47,803 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1001] - Closed socket connection for client /172.22.0.5:58587 which had sessionid 0x24868e7ca980003
> 2014-09-16 02:58:47,810 [myid:1] - ERROR [CommitProcessor:1:NIOServerCnxn@180] - Unexpected Exception:
> java.nio.channels.CancelledKeyException
>         at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
>         at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
>         at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:153)
>         at org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1076)
>         at org.apache.zookeeper.server.NIOServerCnxn.process(NIOServerCnxn.java:1113)
>         at org.apache.zookeeper.server.DataTree.setWatches(DataTree.java:1327)
>         at org.apache.zookeeper.server.ZKDatabase.setWatches(ZKDatabase.java:384)
>         at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:304)
>         at org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:74)
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)