StackOverflow Problem with 1.3 mllib ALS

[Justin Yip <yi...@prediction.io>]

Hello,

I have been using Mllib's ALS in 1.2 and it works quite well. I have just
upgraded to 1.3 and I encountered stackoverflow problem.

After some digging, I realized that when the iteration > ~35, I will get
overflow problem. However, I can get at least 80 iterations with ALS in 1.2.

Is there any change to the ALS algorithm? And are there any ways to achieve
more iterations?

Thanks.

Justin

Re: StackOverflow Problem with 1.3 mllib ALS

[Nick Pentreath <ni...@gmail.com>]

Fair enough but I'd say you hit that diminishing return after 20 iterations
or so... :)

On Thu, Apr 2, 2015 at 9:39 AM, Justin Yip <yi...@gmail.com> wrote:

> Thanks Xiangrui,
>
> I used 80 iterations to demonstrates the marginal diminishing return in
> prediction quality :)
>
> Justin
> On Apr 2, 2015 00:16, "Xiangrui Meng" <me...@gmail.com> wrote:
>
>> I think before 1.3 you also get stackoverflow problem in > ~35
>> iterations. In 1.3.x, please use setCheckpointInterval to solve this
>> problem, which is available in the current master and 1.3.1 (to be
>> released soon). Btw, do you find 80 iterations are needed for
>> convergence? -Xiangrui
>>
>> On Wed, Apr 1, 2015 at 11:54 PM, Justin Yip <yi...@prediction.io>
>> wrote:
>> > Hello,
>> >
>> > I have been using Mllib's ALS in 1.2 and it works quite well. I have
>> just
>> > upgraded to 1.3 and I encountered stackoverflow problem.
>> >
>> > After some digging, I realized that when the iteration > ~35, I will get
>> > overflow problem. However, I can get at least 80 iterations with ALS in
>> 1.2.
>> >
>> > Is there any change to the ALS algorithm? And are there any ways to
>> achieve
>> > more iterations?
>> >
>> > Thanks.
>> >
>> > Justin
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>

Re: StackOverflow Problem with 1.3 mllib ALS

[Justin Yip <yi...@gmail.com>]

Thanks Xiangrui,

I used 80 iterations to demonstrates the marginal diminishing return in
prediction quality :)

Justin
On Apr 2, 2015 00:16, "Xiangrui Meng" <me...@gmail.com> wrote:

> I think before 1.3 you also get stackoverflow problem in > ~35
> iterations. In 1.3.x, please use setCheckpointInterval to solve this
> problem, which is available in the current master and 1.3.1 (to be
> released soon). Btw, do you find 80 iterations are needed for
> convergence? -Xiangrui
>
> On Wed, Apr 1, 2015 at 11:54 PM, Justin Yip <yi...@prediction.io>
> wrote:
> > Hello,
> >
> > I have been using Mllib's ALS in 1.2 and it works quite well. I have just
> > upgraded to 1.3 and I encountered stackoverflow problem.
> >
> > After some digging, I realized that when the iteration > ~35, I will get
> > overflow problem. However, I can get at least 80 iterations with ALS in
> 1.2.
> >
> > Is there any change to the ALS algorithm? And are there any ways to
> achieve
> > more iterations?
> >
> > Thanks.
> >
> > Justin
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: StackOverflow Problem with 1.3 mllib ALS

[Xiangrui Meng <me...@gmail.com>]

I think before 1.3 you also get stackoverflow problem in > ~35
iterations. In 1.3.x, please use setCheckpointInterval to solve this
problem, which is available in the current master and 1.3.1 (to be
released soon). Btw, do you find 80 iterations are needed for
convergence? -Xiangrui

On Wed, Apr 1, 2015 at 11:54 PM, Justin Yip <yi...@prediction.io> wrote:
> Hello,
>
> I have been using Mllib's ALS in 1.2 and it works quite well. I have just
> upgraded to 1.3 and I encountered stackoverflow problem.
>
> After some digging, I realized that when the iteration > ~35, I will get
> overflow problem. However, I can get at least 80 iterations with ALS in 1.2.
>
> Is there any change to the ALS algorithm? And are there any ways to achieve
> more iterations?
>
> Thanks.
>
> Justin

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org