IOException: MkDirs Failed to Create in Spark

[Anuj Wadehra <an...@yahoo.co.in>]

Hi
 
We are using Spark with Cassandra. While using rdd.saveAsTextFile("/tmp/dr"), we are getting following error when we run the application with root access. Spark is able to create two level of directories but fails after that with Exception:

16/03/01 22:59:48 WARN TaskSetManager: Lost task 73.3 in stage 0.0 (TID 144, host1): java.io.IOException: Mkdirs failed to create file:/tmp/dr/_temporary/0/_temporary/attempt_201603012259_0000_m_000073_144
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:438)
        at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:424)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:906)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:799)
        at org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:123)
        at org.apache.spark.SparkHadoopWriter.open(SparkHadoopWriter.scala:90)
        at org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1056)
        at org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1047)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
        at org.apache.spark.scheduler.Task.run(Task.scala:56)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

		
Permissions on /tmp:
chmod -R 777 /tmp has been executed and permissions look like:
drwxrwxrwx.  31 root root 1.2K Mar  1 22:54 tmp

Forgive me for raising this question in Cassandra Mailing list. I think Spark & Cassandra user base is overlapping, so I expected help here.
I am not yet part of Spark mailing list.

Thanks
Anuj