You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Shixiong Zhu (JIRA)" <ji...@apache.org> on 2015/12/17 07:12:46 UTC

[jira] [Resolved] (SPARK-11904) pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing

     [ https://issues.apache.org/jira/browse/SPARK-11904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shixiong Zhu resolved SPARK-11904.
----------------------------------
       Resolution: Fixed
         Assignee: David Tolpin
    Fix Version/s: 2.0.0

> pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing
> ---------------------------------------------------------------------
>
>                 Key: SPARK-11904
>                 URL: https://issues.apache.org/jira/browse/SPARK-11904
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 1.5.2
>            Reporter: David Tolpin
>            Assignee: David Tolpin
>             Fix For: 2.0.0
>
>
> When invFunc argument of reduceByKeyAndWindow is None, checkpointing should not be required. Scala implementation of reduceByKeyAndWindow handles this correctly and does not require checkpointing. Python version requires that checkpoint directory be specified, it is a bug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org