You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/02/03 03:26:34 UTC
[jira] [Commented] (SPARK-5545) [STREAMING] DStream#saveAs**Files
can fail after app restarts
[ https://issues.apache.org/jira/browse/SPARK-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14302673#comment-14302673 ]
Apache Spark commented on SPARK-5545:
-------------------------------------
User 'harishreedharan' has created a pull request for this issue:
https://github.com/apache/spark/pull/4322
> [STREAMING] DStream#saveAs**Files can fail after app restarts
> -------------------------------------------------------------
>
> Key: SPARK-5545
> URL: https://issues.apache.org/jira/browse/SPARK-5545
> Project: Spark
> Issue Type: Bug
> Reporter: Hari Shreedharan
> Priority: Critical
>
> After an app restarts, sometimes the saveAs**Files can fail. This happens if the driver dies while the RDD was being written to HDFS. At this point the rdd-<timestamp> directory has already been created but we have not marked it as completely processed. This causes the RDD to get written after we restart, into the same directory. This can cause the underlying MR api to throw an exception that looks like this:
> {code}
> 15/02/02 13:16:41 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: Output directory hdfs://wypoon-cdhx-1.ent.cloudera.com:8020/user/systest/flumetest/rdd-1422911774000 already exists)
> Exception in thread "Driver" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://wypoon-cdhx-1.ent.cloudera.com:8020/user/systest/flumetest/rdd-1422911774000 already exists
> at org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:132)
> at org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopDataset(PairRDDFunctions.scala:1041)
> at org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:940)
> at org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:849)
> at org.apache.spark.rdd.RDD.saveAsTextFile(RDD.scala:1164)
> ...
> {code}
> Thanks to [~wypoon] for finding this issue!
> I have a PR coming up for this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org