You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2017/10/26 20:53:00 UTC

[jira] [Commented] (SPARK-22366) Support ignoreMissingFiles flag parallel to ignoreCorruptFiles

    [ https://issues.apache.org/jira/browse/SPARK-22366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221225#comment-16221225 ] 

Apache Spark commented on SPARK-22366:
--------------------------------------

User 'joseph-torres' has created a pull request for this issue:
https://github.com/apache/spark/pull/19581

> Support ignoreMissingFiles flag parallel to ignoreCorruptFiles
> --------------------------------------------------------------
>
>                 Key: SPARK-22366
>                 URL: https://issues.apache.org/jira/browse/SPARK-22366
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.2.0
>            Reporter: Jose Torres
>            Priority: Minor
>
> There's an existing flag "spark.sql.files.ignoreCorruptFiles" that will quietly ignore attempted reads from files that have been corrupted, but it still allows the query to fail on missing files. Being able to ignore missing files too is useful in some replication scenarios.
> We should add a "spark.sql.files.ignoreMissingFiles" to fill out the functionality.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org