You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2016/03/10 05:35:40 UTC

[jira] [Commented] (SPARK-13793) PipeRDD doesn't propagate exceptions while reading parent RDD

    [ https://issues.apache.org/jira/browse/SPARK-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188660#comment-15188660 ] 

Apache Spark commented on SPARK-13793:
--------------------------------------

User 'tejasapatil' has created a pull request for this issue:
https://github.com/apache/spark/pull/11628

> PipeRDD doesn't propagate exceptions while reading parent RDD
> -------------------------------------------------------------
>
>                 Key: SPARK-13793
>                 URL: https://issues.apache.org/jira/browse/SPARK-13793
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.6.0
>            Reporter: Tejas Patil
>            Priority: Minor
>
> PipeRDD creates a process to run the command and spawns a thread to feed the input data to the process's stdin. If there is any exception in the child thread which gets the input data from the parent RDD, the child thread does not propagate that exception to the main thread. eg. In event of fetch failures, since the exception is not be propagated, the entire stage fails. The correct behaviour would be to recompute the parent(s) and then relaunch the stage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org