[jira] [Created] (AMBARI-15091) Metrics collector running out of file descriptors

["Dmytro Sen (JIRA)" <ji...@apache.org>]

Dmytro Sen created AMBARI-15091:
-----------------------------------

             Summary: Metrics collector running out of file descriptors
                 Key: AMBARI-15091
                 URL: https://issues.apache.org/jira/browse/AMBARI-15091
             Project: Ambari
          Issue Type: Bug
    Affects Versions: 2.2.2
            Reporter: Dmytro Sen
            Assignee: Dmytro Sen
            Priority: Critical
             Fix For: 2.2.2


1200 node cluster:

{code}
[root@perf-a-2 ~]# lsof -p 4100 | wc -l
32928
{code}

Exception in collector logs:
{code}
2016-02-05 15:51:50,528 WARN org.mortbay.log: EXCEPTION
java.io.IOException: Too many open files
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.mortbay.jetty.nio.SelectChannelConnector$1.acceptChannel(SelectChannelConnector.java:75)
        at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:695)
        at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:193)
        at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
        at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{code}

The client side connections are also open and there are plenty of CLOSE_WAIT sockets.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)