You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (Jira)" <ji...@apache.org> on 2019/08/30 16:38:00 UTC
[jira] [Resolved] (SPARK-28866) Persist item factors RDD when
checkpointing in ALS
[ https://issues.apache.org/jira/browse/SPARK-28866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-28866.
-------------------------------
Fix Version/s: 3.0.0
Resolution: Fixed
Issue resolved by pull request 25576
[https://github.com/apache/spark/pull/25576]
> Persist item factors RDD when checkpointing in ALS
> --------------------------------------------------
>
> Key: SPARK-28866
> URL: https://issues.apache.org/jira/browse/SPARK-28866
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 3.0.0
> Reporter: Liang-Chi Hsieh
> Assignee: Liang-Chi Hsieh
> Priority: Minor
> Fix For: 3.0.0
>
>
> In ALS ML implementation, if `implicitPrefs` is false, we checkpoint the RDD of item factors, between intervals. Before checkpointing and materializing RDD, this RDD was not persisted. It causes recomputation. In an experiment, there is performance difference between persisting and no persisting before checkpointing the RDD.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org