You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by amghost <zh...@outlook.com> on 2015/04/23 06:03:10 UTC
Re: StackOverflow Error when run ALS with 100 iterations
Hi, would you please how to checkpoint the training set rdd since all things
are done in ALS.train method.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/StackOverflow-Error-when-run-ALS-with-100-iterations-tp4296p22619.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: StackOverflow Error when run ALS with 100 iterations
Posted by Xiangrui Meng <me...@gmail.com>.
ALS.setCheckpointInterval was added in Spark 1.3.1. You need to
upgrade Spark to use this feature. -Xiangrui
On Wed, Apr 22, 2015 at 9:03 PM, amghost <zh...@outlook.com> wrote:
> Hi, would you please how to checkpoint the training set rdd since all things
> are done in ALS.train method.
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/StackOverflow-Error-when-run-ALS-with-100-iterations-tp4296p22619.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: StackOverflow Error when run ALS with 100 iterations
Posted by LeoB <le...@gmail.com>.
Just wanted to add a comment to the Jira ticket but I don't think I have
permission to do so, so answering here instead. I am encountering the same
issue with a stackOverflow Exception.
I would like to point out that there is a
localCheckpoint
<https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-rdd-checkpointing.html>
method which does not require HDFS to be installed. We could use this
instead of Checkpoint to cut down the lineage.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org