[jira] [Created] (KAFKA-2595) Processor thread dies due to an uncaught NoSuchElementException

["Shaun Senecal (JIRA)" <ji...@apache.org>]

Shaun Senecal created KAFKA-2595:
------------------------------------

             Summary: Processor thread dies due to an uncaught NoSuchElementException
                 Key: KAFKA-2595
                 URL: https://issues.apache.org/jira/browse/KAFKA-2595
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 0.8.2.0
            Reporter: Shaun Senecal


We are getting uncaught exceptions which seem to kill the processor threads.  The end result is that we end up with a bunch of connections in CLOSE_WAIT and eventually the broker is unable to respond or hits the max open files ulimit.

{noformat}
[2015-09-23 09:54:33,687] ERROR Uncaught exception in thread 'kafka-network-thread-9092-2': (kafka.utils.Utils$)
java.util.NoSuchElementException: None.get
        at scala.None$.get(Option.scala:347)
        at scala.None$.get(Option.scala:345)
        at kafka.network.ConnectionQuotas.dec(SocketServer.scala:524)
        at kafka.network.AbstractServerThread.close(SocketServer.scala:165)
        at kafka.network.AbstractServerThread.close(SocketServer.scala:157)
        at kafka.network.Processor.close(SocketServer.scala:374)
        at kafka.network.Processor.processNewResponses(SocketServer.scala:406)
        at kafka.network.Processor.run(SocketServer.scala:318)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

The issue appears to be the same as KAFKA-1577, except that its not happening during shutdown.  We haven't been able to isolate when this happens, so we dont have a good way to reproduce the issue.

It also looks like KAFKA-2353 would work around the issue if it could be back-ported, but the root cause should probably be fixed as well.

- java version: 1.7.0_65
- kafka version: 0.8.2.0
- topics: 366
- partitions: ~550 (a few 20 partition topics, and a bunch of 1 partition topics)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)