You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Debasish Das <de...@gmail.com> on 2014/08/15 21:30:57 UTC
ALS checkpoint performance
Hi,
Are there any experiments detailing the performance hit due to HDFS
checkpoint in ALS ?
As we scale to large ranks with more ratings, I believe we have to cut the
RDD lineage to safe guard against the lineage issue...
Thanks.
Deb
Re: ALS checkpoint performance
Posted by Xiangrui Meng <me...@gmail.com>.
Guoqiang reported some results in his PRs
https://github.com/apache/spark/pull/828 and
https://github.com/apache/spark/pull/929 . But this is really
problem-dependent. -Xiangrui
On Fri, Aug 15, 2014 at 12:30 PM, Debasish Das <de...@gmail.com> wrote:
> Hi,
>
> Are there any experiments detailing the performance hit due to HDFS
> checkpoint in ALS ?
>
> As we scale to large ranks with more ratings, I believe we have to cut the
> RDD lineage to safe guard against the lineage issue...
>
> Thanks.
> Deb
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org