: Failed to create file system watcher service: User limit of inotify instances reached or too many open files

["Polisetti, Venkata Siva Rama Gopala Krishna" <vp...@spglobal.com>]

Hi,

When I am doing calculations for example 700 listID's it is saving only some 50 rows and then getting some random exceptions

Getting below exception when I try to do calculations on huge data and try to save huge data . Please let me know if any suggestions.

Sample Code :

    I have some lakhs ListID's making group of RDD  rdd.groupBy(row => row.getInt(6)).  And then using .map doing all ranking calculations

Step1 : groupedPartitionRdd =  rdd.groupBy(row => row.getInt(6))
Step2 :  val outputObjectForGainersAndLosers =  groupedPartitionRdd.map(grp =>
      {
                                                //ranking logic and some calculations

      }
Step 3 :    outputObjectForGainersAndLosers.saveToCassandra("tablename",somecolumns ).


Am getting some random exceptions every time in spark- submit not able to debug and facing lot of issues.

Caused by: java.lang.RuntimeException: Failed to create file system watcher service: User limit of inotify instances reached or too many open files
Caused by: java.io.IOException: User limit of inotify instances reached or too many open files
Failed to create file system watcher service: User limit of inotify instances reached or too many open files

Error while invoking RpcHandler#receive() for one-way message.
org.apache.spark.SparkException: Could not find CoarseGrainedScheduler.
        at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:154)
        at org.apache.spark.rpc.netty.Dispatcher.postOneWayMessage(Dispatcher.scala:134)
        at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:644)
        at org.apache.spark.network.server.TransportRequestHandler.processOneWayMessage(TransportRequestHandler.java:178)
        at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:107)
        at org.apache.spark.network.server.TransportChannelHandler.channelRead

Thanks,
Gopi

________________________________

The information contained in this message is intended only for the recipient, and may be a confidential attorney-client communication or may otherwise be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, please be aware that any dissemination or copying of this communication is strictly prohibited. If you have received this communication in error, please immediately notify us by replying to the message and deleting it from your computer. S&P Global Inc. reserves the right, subject to applicable local law, to monitor, review and process the content of any electronic message or information sent to or from S&P Global Inc. e-mail addresses without informing the sender or recipient of the message. By sending electronic message or information to S&P Global Inc. e-mail addresses you, as the sender, are consenting to S&P Global Inc. processing any of your personal data therein.