You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Shixiong Zhu (JIRA)" <ji...@apache.org> on 2015/12/17 07:13:46 UTC
[jira] [Updated] (SPARK-11904) pyspark reduceByKeyAndWindow with
invFunc=None requires checkpointing
[ https://issues.apache.org/jira/browse/SPARK-11904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu updated SPARK-11904:
---------------------------------
Affects Version/s: 1.6.0
> pyspark reduceByKeyAndWindow with invFunc=None requires checkpointing
> ---------------------------------------------------------------------
>
> Key: SPARK-11904
> URL: https://issues.apache.org/jira/browse/SPARK-11904
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 1.5.2, 1.6.0
> Reporter: David Tolpin
> Assignee: David Tolpin
> Fix For: 2.0.0
>
>
> When invFunc argument of reduceByKeyAndWindow is None, checkpointing should not be required. Scala implementation of reduceByKeyAndWindow handles this correctly and does not require checkpointing. Python version requires that checkpoint directory be specified, it is a bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org