You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:23:23 UTC

[jira] [Updated] (SPARK-16790) Warn (or fail) when user query reads no data due to pruning

     [ https://issues.apache.org/jira/browse/SPARK-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon updated SPARK-16790:
---------------------------------
    Labels: bulk-closed  (was: )

> Warn (or fail) when user query reads no data due to pruning
> -----------------------------------------------------------
>
>                 Key: SPARK-16790
>                 URL: https://issues.apache.org/jira/browse/SPARK-16790
>             Project: Spark
>          Issue Type: Wish
>          Components: SQL
>    Affects Versions: 1.6.2, 2.0.0
>            Reporter: Tejas Patil
>            Priority: Minor
>              Labels: bulk-closed
>
> User query might refer to partitions of the input table which might be existing at some point (OR might not have been produced when the query execution starts). In such cases, if we let the computation run, then it might produce empty / unexpected results. To clarify, as far as SQL semantics is concerned, it would be producing correct output. But from end users' perspective, that might not be desired results. We could add a config option to let the query fail in such cases. Irrespective of that, have some logging to indicate this to the end user.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org