PLSI/EM in pig. Getting stuck in local minima ?

[prasenjit mukherjee <pr...@gmail.com>]

Hi,
   I have written a pig script which attempts to implement  the EM algo for
PLSI. For extremely simple case with the following data :
transaction_log = (u1,s1), (u2,s2)
z = 2

if I initialize the p(s|z) and p(z|u)  with  equal probabilities (all 0.5 )
I was expecting a convergence of results towards ==> p(z1|u1) ~ 1  and
p(z2|u2) ~ 1.

But it seems to get stuck with the same values after the first iteration. Is
it because I am working with too less a data and getting stuck in a local
minima or there is indeed a problem with my implementation which needs to be
fixed.

I also tried manually computing with those iterations and even then I am
getting same values for p(s|z) and p(z|u) with which I had started.

Any help is greatly appreciated.

-Thanks,
Prasen

Re: PLSI/EM in pig. Getting stuck in local minima ?

[prasenjit mukherjee <pr...@gmail.com>]

Thanks Pallavi for reminding :)

Yep it works when random probabilities are used.

-Prasen

On Tue, Feb 10, 2009 at 2:20 PM, Pallavi Palleti <
pallavi.palleti@corp.aol.com> wrote:

> Hi Prasen,
>  I faced the  same issue and figured out that, when you use equal
> probabilities for both P(s/z) and P(z/u), it will average out and finally
> end up with the same probability that you have started with. In order to
> make it work, one of it should be random either P(s/z) and P(z/u). Then it
> will converge.
>
> Thanks
> Pallavi
>
> prasenjit mukherjee wrote:
>
>> Hi,
>>   I have written a pig script which attempts to implement  the EM algo for
>> PLSI. For extremely simple case with the following data :
>> transaction_log = (u1,s1), (u2,s2)
>> z = 2
>>
>> if I initialize the p(s|z) and p(z|u)  with  equal probabilities (all 0.5
>> )
>> I was expecting a convergence of results towards ==> p(z1|u1) ~ 1  and
>> p(z2|u2) ~ 1.
>>
>> But it seems to get stuck with the same values after the first iteration.
>> Is
>> it because I am working with too less a data and getting stuck in a local
>> minima or there is indeed a problem with my implementation which needs to
>> be
>> fixed.
>>
>> I also tried manually computing with those iterations and even then I am
>> getting same values for p(s|z) and p(z|u) with which I had started.
>>
>> Any help is greatly appreciated.
>>
>> -Thanks,
>> Prasen
>>
>>
>>
>
>

Re: PLSI/EM in pig. Getting stuck in local minima ?

[Pallavi Palleti <pa...@corp.aol.com>]

Hi Prasen,
   I faced the  same issue and figured out that, when you use equal 
probabilities for both P(s/z) and P(z/u), it will average out and 
finally end up with the same probability that you have started with. In 
order to make it work, one of it should be random either P(s/z) and 
P(z/u). Then it will converge.

Thanks
Pallavi
prasenjit mukherjee wrote:
> Hi,
>    I have written a pig script which attempts to implement  the EM algo for
> PLSI. For extremely simple case with the following data :
> transaction_log = (u1,s1), (u2,s2)
> z = 2
>
> if I initialize the p(s|z) and p(z|u)  with  equal probabilities (all 0.5 )
> I was expecting a convergence of results towards ==> p(z1|u1) ~ 1  and
> p(z2|u2) ~ 1.
>
> But it seems to get stuck with the same values after the first iteration. Is
> it because I am working with too less a data and getting stuck in a local
> minima or there is indeed a problem with my implementation which needs to be
> fixed.
>
> I also tried manually computing with those iterations and even then I am
> getting same values for p(s|z) and p(z|u) with which I had started.
>
> Any help is greatly appreciated.
>
> -Thanks,
> Prasen
>
>