You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Tathagata Das (JIRA)" <ji...@apache.org> on 2016/12/06 01:49:58 UTC

[jira] [Updated] (SPARK-18671) Add tests to ensure stability of that all Structured Streaming log formats

     [ https://issues.apache.org/jira/browse/SPARK-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tathagata Das updated SPARK-18671:
----------------------------------
    Priority: Major  (was: Critical)

> Add tests to ensure stability of that all Structured Streaming log formats
> --------------------------------------------------------------------------
>
>                 Key: SPARK-18671
>                 URL: https://issues.apache.org/jira/browse/SPARK-18671
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>            Reporter: Tathagata Das
>
> To be able to restart StreamingQueries across Spark version, we have already made the logs (offset log, file source log, file sink log) use json. We should added tests with actual json files in the Spark such that any incompatible changes in reading the logs is immediately caught. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org