You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Jungtaek Lim (JIRA)" <ji...@apache.org> on 2018/05/17 21:17:00 UTC

[jira] [Created] (SPARK-24311) Refactor HDFSBackedStateStoreProvide to remove duplicated logic between operations on delta file and operations on snapshot file

Jungtaek Lim created SPARK-24311:
------------------------------------

             Summary: Refactor HDFSBackedStateStoreProvide to remove duplicated logic between operations on delta file and operations on snapshot file
                 Key: SPARK-24311
                 URL: https://issues.apache.org/jira/browse/SPARK-24311
             Project: Spark
          Issue Type: Improvement
          Components: Structured Streaming
    Affects Versions: 2.3.0
            Reporter: Jungtaek Lim


The structure of delta file and snapshot file is same, but the operations are defined as individual methods which incurs duplicated logic between logic for delta file and logic for snapshot file.

We can refactor to remove duplicated logic to ensure readability, as well as guaranteeing that the structure of delta file and snapshot file keeps same.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org