You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/11/30 22:59:11 UTC
[jira] [Assigned] (SPARK-11328) Correctly propagate error message
in the case of failures when writing parquet
[ https://issues.apache.org/jira/browse/SPARK-11328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-11328:
------------------------------------
Assignee: Apache Spark (was: Nong Li)
> Correctly propagate error message in the case of failures when writing parquet
> ------------------------------------------------------------------------------
>
> Key: SPARK-11328
> URL: https://issues.apache.org/jira/browse/SPARK-11328
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Yin Huai
> Assignee: Apache Spark
> Priority: Critical
>
> When saving data to S3 (e.g. saving to parquet), if there is an error during the query execution, the partial file generated by the failed task will be uploaded to S3 and the retries of this task will throw file already exist error. It is very confusing to users because they may think that file already exist error is the error causing the job failure. They can only find the real error in the spark ui (in the stage page).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org