You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Tathagata Das (JIRA)" <ji...@apache.org> on 2016/12/01 08:37:58 UTC

[jira] [Created] (SPARK-18671) Add tests to ensure stability of that all Structured Streaming log formats

Tathagata Das created SPARK-18671:
-------------------------------------

             Summary: Add tests to ensure stability of that all Structured Streaming log formats
                 Key: SPARK-18671
                 URL: https://issues.apache.org/jira/browse/SPARK-18671
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
            Reporter: Tathagata Das
            Priority: Critical


To be able to restart StreamingQueries across Spark version, we have already made the logs (offset log, file source log, file sink log) use json. We should added tests with actual json files in the Spark such that any incompatible changes in reading the logs is immediately caught. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org