[jira] [Commented] (HDFS-4251) NN connections can use up all fds leaving none for rolling journal files

["Kihwal Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-4251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508951#comment-13508951 ] 

Kihwal Lee commented on HDFS-4251:
----------------------------------

We've another case that affects edit logging: The edit rolling was successful, but after downloading checkpoint fsimage from the secondary namenode, FileJournalManager#purgeLogsOlderThan() failed due to the system's opendir() failed. 

bq. The rpc layer needs to limit the number of tcp connections to be a little less than the max fds.

There are also multiple log files and connections through servlets beside RPC requests. Do we want NN to figure out the limit automatically? Or leave it to users set a config variable? Automatic setting may be a bit tricky to do in a platform independent way.

The following is a breakdown of open file descriptors of a running namenode excluding sockets.

1 gc log
2 /dev/random, /dev/urandom
2 in-use lock files
2 edit log files
3 stdin, stdout, stderr
3 nn log, auth log, audit log
9 epoll fd
18 pipes
157 jar files

Limiting RPC layer to not accept beyond configured limit will be straightforward. 
                
> NN connections can use up all fds leaving none for rolling journal files
> ------------------------------------------------------------------------
>
>                 Key: HDFS-4251
>                 URL: https://issues.apache.org/jira/browse/HDFS-4251
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Sanjay Radia
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira