You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Bryan Cutler (JIRA)" <ji...@apache.org> on 2018/01/19 18:54:00 UTC

[jira] [Created] (SPARK-23159) Update Cloudpickle to match version 0.4.2

Bryan Cutler created SPARK-23159:
------------------------------------

             Summary: Update Cloudpickle to match version 0.4.2
                 Key: SPARK-23159
                 URL: https://issues.apache.org/jira/browse/SPARK-23159
             Project: Spark
          Issue Type: Improvement
          Components: PySpark
    Affects Versions: 2.3.0
            Reporter: Bryan Cutler


Update PySpark's version of Cloudpickle to match version 0.4.2.  The reasons for doing this are:
 * Pick up bug fixes, improvements with newer version
 * Match a specific version as close as possible (Spark has additional changes that might be necessary) to make future upgrades easier

There are newer versions of Cloudpickle that can fix bugs with NamedTuple pickling (that Spark currently has workarounds for), but these include other changes that need some verification before bringing into Spark.  Upgrading first to 0.4.2 will help make this verification easier.

Discussion on the mailing list: http://apache-spark-developers-list.1001551.n3.nabble.com/Thoughts-on-Cloudpickle-Update-td23188.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org