[jira] [Created] (SPARK-23154) Document backwards compatibility guarantees for ML persistence

["Joseph K. Bradley (JIRA)" <ji...@apache.org>]

Joseph K. Bradley created SPARK-23154:
-----------------------------------------

             Summary: Document backwards compatibility guarantees for ML persistence
                 Key: SPARK-23154
                 URL: https://issues.apache.org/jira/browse/SPARK-23154
             Project: Spark
          Issue Type: Documentation
          Components: Documentation, ML
    Affects Versions: 2.3.0
            Reporter: Joseph K. Bradley
            Assignee: Joseph K. Bradley


We have (as far as I know) maintained backwards compatibility for ML persistence, but this is not documented anywhere.  I'd like us to document it (for spark.ml, not for spark.mllib).

I'd recommend something like:
{quote}
In general, MLlib maintains backwards compatibility for ML persistence.  I.e., if you save an ML model or Pipeline in one version of Spark, then you should be able to load it back and use it in a future version of Spark.  However, there are rare exceptions, described below.

Model persistence: Is a model or Pipeline saved using Apache Spark ML persistence in Spark version X loadable by Spark version Y?
* Major versions: No guarantees, but best-effort.
* Minor and patch versions: Yes; these are backwards compatible.
* Note about the format: There are no guarantees for a stable persistence format, but model loading itself is designed to be backwards compatible.

Model behavior: Does a model or Pipeline in Spark version X behave identically in Spark version Y?
* Major versions: No guarantees, but best-effort.
* Minor and patch versions: Identical behavior, except for bug fixes.

For both model persistence and model behavior, any breaking changes across a minor version or patch version are reported in the Spark version release notes. If a breakage is not reported in release notes, then it should be treated as a bug to be fixed.
{quote}

How does this sound?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org