[jira] [Created] (SPARK-16989) fileSystem.getFileStatus throw exception in EventLoggingListener

["Hong Shen (JIRA)" <ji...@apache.org>]

Hong Shen created SPARK-16989:
---------------------------------

             Summary: fileSystem.getFileStatus throw exception in EventLoggingListener
                 Key: SPARK-16989
                 URL: https://issues.apache.org/jira/browse/SPARK-16989
             Project: Spark
          Issue Type: Bug
    Affects Versions: 2.0.0
            Reporter: Hong Shen


16/05/02 22:24:22 ERROR spark.SparkContext: Error initializing SparkContext.
java.io.FileNotFoundException: File file:/data/tdwadmin/tdwenv/tdwgaia/logs/sparkhistory/intermediate-done-dir does not exist
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:520)
	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:398)
	at org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:109)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:612)
	at org.apache.spark.deploy.yarn.SQLApplicationMaster.<init>(SQLApplicationMaster.scala:78)
	at org.apache.spark.deploy.yarn.SQLApplicationMaster.<init>(SQLApplicationMaster.scala:46)
	at org.apache.spark.deploy.yarn.SQLApplicationMaster$$anonfun$main$1.apply$mcV$sp(SQLApplicationMaster.scala:311)
	at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:70)
	at org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:69)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
	at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:69)
	at org.apache.spark.deploy.yarn.SQLApplicationMaster$.main(SQLApplicationMaster.scala:310)
	at org.apache.spark.deploy.yarn.SQLApplicationMaster.main(SQLApplicationMaster.scala)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org