[GitHub] bhavinthaker commented on issue #8904: fix multiprocessing too many open files

[GitBox <gi...@apache.org>]

bhavinthaker commented on issue #8904: fix multiprocessing too many open files
URL: https://github.com/apache/incubator-mxnet/pull/8904#issuecomment-349420885
 
 
   I have documented the following in the MXNet 1.0 Release Notes here: https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+1.0+Release+Notes (see this Release notes for up-to-date information)
   
   Problem: The data-loader has a file-descriptor leak when the number of files being opened exceeds the ulimit of the user session.
   Workaround: Increase the ulimit of the user session or at the OS level using any of the following steps:
   $ ulimit -n unlimited  (to make the fd limit as unlimited), OR
   $ ulimit -n 60000  (to make the fd limit as 60,000 files assuming you have around but less than 60,000 files to be opened) OR
   $ echo 65535 > /proc/sys/fs/file-max (to increase the system-wide file limit) along with the ulimit command.
   Further details: https://docs.oracle.com/cd/E19623-01/820-6168/file-descriptor-requirements.html

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services