You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Nicholas Chammas (JIRA)" <ji...@apache.org> on 2015/02/06 20:50:35 UTC

[jira] [Updated] (SPARK-729) Closures not always serialized at capture time

     [ https://issues.apache.org/jira/browse/SPARK-729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nicholas Chammas updated SPARK-729:
-----------------------------------
    Component/s: Spark Core

> Closures not always serialized at capture time
> ----------------------------------------------
>
>                 Key: SPARK-729
>                 URL: https://issues.apache.org/jira/browse/SPARK-729
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 0.7.0, 0.7.1
>            Reporter: Matei Zaharia
>            Assignee: William Benton
>
> As seen in https://groups.google.com/forum/?fromgroups=#!topic/spark-users/8pTchwuP2Kk and its corresponding fix on https://github.com/mesos/spark/commit/adba773fab6294b5764d101d248815a7d3cb3558, it is possible for a closure referencing a var to see the latest version of that var, instead of the version that was there when the closure was passed to Spark. This is not good when failures or recomputations happen. We need to serialize the closures on capture if possible, perhaps as part of ClosureCleaner.clean.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org