You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by amghost <zh...@outlook.com> on 2015/04/23 06:03:10 UTC

Re: StackOverflow Error when run ALS with 100 iterations

Hi, would you please how to checkpoint the training set rdd since all things
are done in ALS.train method.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/StackOverflow-Error-when-run-ALS-with-100-iterations-tp4296p22619.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Re: StackOverflow Error when run ALS with 100 iterations

Posted by Xiangrui Meng <me...@gmail.com>.
ALS.setCheckpointInterval was added in Spark 1.3.1. You need to
upgrade Spark to use this feature. -Xiangrui

On Wed, Apr 22, 2015 at 9:03 PM, amghost <zh...@outlook.com> wrote:
> Hi, would you please how to checkpoint the training set rdd since all things
> are done in ALS.train method.
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/StackOverflow-Error-when-run-ALS-with-100-iterations-tp4296p22619.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Re: StackOverflow Error when run ALS with 100 iterations

Posted by LeoB <le...@gmail.com>.
Just wanted to add a comment to the Jira ticket but I don't think I have
permission to do so, so answering here instead. I am encountering the same
issue with a stackOverflow Exception. 
I would like to point out that there is a  
localCheckpoint
<https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-rdd-checkpointing.html>  
method which does not require HDFS to be installed. We could use this
instead of Checkpoint to cut down the lineage. 



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org