[jira] [Created] (SPARK-21192) Preserve State Store provider class configuration across restarts

["Tathagata Das (JIRA)" <ji...@apache.org>]

Tathagata Das created SPARK-21192:
-------------------------------------

             Summary: Preserve State Store provider class configuration across restarts
                 Key: SPARK-21192
                 URL: https://issues.apache.org/jira/browse/SPARK-21192
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 2.3.0
            Reporter: Tathagata Das
            Assignee: Tathagata Das


If the SQL conf for StateStore provider class is changed between restarts (i.e. query started with providerClass1 and attempted to restart using providerClass2), then the query will fail in a unpredictable way as files saved by one provider class cannot be used by the newer one. 

Ideally, the provider class used to start the query should be used to restart the query, and the configuration in the session where it is being restarted should be ignored. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org