[jira] [Created] (SPARK-19913) Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach

["Kousuke Saruta (JIRA)" <ji...@apache.org>]

Kousuke Saruta created SPARK-19913:
--------------------------------------

             Summary: Log warning rather than throw AnalysisException when output is partitioned although format is memory, console or foreach
                 Key: SPARK-19913
                 URL: https://issues.apache.org/jira/browse/SPARK-19913
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 2.2.0
            Reporter: Kousuke Saruta
            Priority: Minor


When batches are executed with memory, console or foreach format, `assertNotPartitioned` will check whether output is not partitioned and throw AnalysisException in case it is.

But I wonder it's better to log warning rather than throw the exception because partitioning does not affect output for those formats but also does not bring any negative impacts.

Also, this assertion is not applied when the format is `console`. I think in this case too, we should assert that .

By fixing them, we can easily switch the format to memory or console for debug purposes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org