You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Michael Lazarus (JIRA)" <ji...@apache.org> on 2010/05/24 19:18:24 UTC
[jira] Created: (MAHOUT-399) LDA on Mahout 0.3 does not converge to
correct solution for overlapping pyramids toy problem.
LDA on Mahout 0.3 does not converge to correct solution for overlapping pyramids toy problem.
---------------------------------------------------------------------------------------------
Key: MAHOUT-399
URL: https://issues.apache.org/jira/browse/MAHOUT-399
Project: Mahout
Issue Type: Bug
Components: Classification
Affects Versions: 0.3
Environment: Mac OS X 10.6.2, Hadoop 0.20.2, Mahout 0.3.
Reporter: Michael Lazarus
Priority: Critical
Hello,
Apologies if I have not labeled this correctly.
I have run a toy problem on Mahout 0.3 (locally) for LDA that I used to test Blei's c version of LDA that he posts on his site. It has an exact solution that the LDA should converge to. Please see attached PDF that describes the intended output.
Is LDA working? The following output indicates some sort of collapsing behavior to me.
T0 T1 T2 T3 T4
x w x u x
u u g j n
l r i m l
j q h h p
v p e i q
e t f g v
d s d f o
b c b n k
y f c l m
w v u v u
c d p y t
k o l r r
i b j k j
f e k e f
g x y s y
t y w b w
h i s p s
o l v x d
q j t d i
n k o t b
The intended output is (again, please see attached):
D I N S X
d i n s x
c h m t y
e j o r w
b k l u v
f g p q a
a f k p b
g l q v u
h m j w t
y u r o c
n s d d i
s e x f f
r q i i n
m v w c o
o w u a h
q n s h g
p t c x d
t x f e l
x d e j s
w y g b j
i r y n r
u o h y m
k b t l e
v c a m k
j a b g p
l p v k q
What tests do you run to make sure the output is correct?
Thank you,
Mike.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-399) LDA on Mahout 0.3 does not converge to
correct solution for overlapping pyramids toy problem.
Posted by "Michael Lazarus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Lazarus updated MAHOUT-399:
-----------------------------------
Attachment: Overlapping Pyramids Toy Dataset.pdf
Please see attached pdf that decribes the toy problem and expected output. Also please find attached a sample dataset including 10,000 small documents created according to the method described in the dataset.
> LDA on Mahout 0.3 does not converge to correct solution for overlapping pyramids toy problem.
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-399
> URL: https://issues.apache.org/jira/browse/MAHOUT-399
> Project: Mahout
> Issue Type: Bug
> Components: Classification
> Affects Versions: 0.3
> Environment: Mac OS X 10.6.2, Hadoop 0.20.2, Mahout 0.3.
> Reporter: Michael Lazarus
> Priority: Critical
> Attachments: Overlapping Pyramids Toy Dataset.pdf
>
>
> Hello,
> Apologies if I have not labeled this correctly.
> I have run a toy problem on Mahout 0.3 (locally) for LDA that I used to test Blei's c version of LDA that he posts on his site. It has an exact solution that the LDA should converge to. Please see attached PDF that describes the intended output.
> Is LDA working? The following output indicates some sort of collapsing behavior to me.
> T0 T1 T2 T3 T4
> x w x u x
> u u g j n
> l r i m l
> j q h h p
> v p e i q
> e t f g v
> d s d f o
> b c b n k
> y f c l m
> w v u v u
> c d p y t
> k o l r r
> i b j k j
> f e k e f
> g x y s y
> t y w b w
> h i s p s
> o l v x d
> q j t d i
> n k o t b
> The intended output is (again, please see attached):
> D I N S X
> d i n s x
> c h m t y
> e j o r w
> b k l u v
> f g p q a
> a f k p b
> g l q v u
> h m j w t
> y u r o c
> n s d d i
> s e x f f
> r q i i n
> m v w c o
> o w u a h
> q n s h g
> p t c x d
> t x f e l
> x d e j s
> w y g b j
> i r y n r
> u o h y m
> k b t l e
> v c a m k
> j a b g p
> l p v k q
> What tests do you run to make sure the output is correct?
> Thank you,
> Mike.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-399) LDA on Mahout 0.3 does not converge
to correct solution for overlapping pyramids toy problem.
Posted by "Michael Lazarus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870845#action_12870845 ]
Michael Lazarus commented on MAHOUT-399:
----------------------------------------
Hi Jeff,
I would be happy to give it a try. I will take a look at the existing unit tests.
Thanks,
Mike.
> LDA on Mahout 0.3 does not converge to correct solution for overlapping pyramids toy problem.
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-399
> URL: https://issues.apache.org/jira/browse/MAHOUT-399
> Project: Mahout
> Issue Type: Bug
> Components: Classification
> Affects Versions: 0.3
> Environment: Mac OS X 10.6.2, Hadoop 0.20.2, Mahout 0.3.
> Reporter: Michael Lazarus
> Priority: Critical
> Attachments: olt.tar, Overlapping Pyramids Toy Dataset.pdf
>
>
> Hello,
> Apologies if I have not labeled this correctly.
> I have run a toy problem on Mahout 0.3 (locally) for LDA that I used to test Blei's c version of LDA that he posts on his site. It has an exact solution that the LDA should converge to. Please see attached PDF that describes the intended output.
> Is LDA working? The following output indicates some sort of collapsing behavior to me.
> T0 T1 T2 T3 T4
> x w x u x
> u u g j n
> l r i m l
> j q h h p
> v p e i q
> e t f g v
> d s d f o
> b c b n k
> y f c l m
> w v u v u
> c d p y t
> k o l r r
> i b j k j
> f e k e f
> g x y s y
> t y w b w
> h i s p s
> o l v x d
> q j t d i
> n k o t b
> The intended output is (again, please see attached):
> D I N S X
> d i n s x
> c h m t y
> e j o r w
> b k l u v
> f g p q a
> a f k p b
> g l q v u
> h m j w t
> y u r o c
> n s d d i
> s e x f f
> r q i i n
> m v w c o
> o w u a h
> q n s h g
> p t c x d
> t x f e l
> x d e j s
> w y g b j
> i r y n r
> u o h y m
> k b t l e
> v c a m k
> j a b g p
> l p v k q
> What tests do you run to make sure the output is correct?
> Thank you,
> Mike.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-399) LDA on Mahout 0.3 does not converge to
correct solution for overlapping pyramids toy problem.
Posted by "Michael Lazarus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Lazarus updated MAHOUT-399:
-----------------------------------
Attachment: olt.tar
The file olt.tar contains a 10,000 document corpus generated according to the method described in pdf.
> LDA on Mahout 0.3 does not converge to correct solution for overlapping pyramids toy problem.
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-399
> URL: https://issues.apache.org/jira/browse/MAHOUT-399
> Project: Mahout
> Issue Type: Bug
> Components: Classification
> Affects Versions: 0.3
> Environment: Mac OS X 10.6.2, Hadoop 0.20.2, Mahout 0.3.
> Reporter: Michael Lazarus
> Priority: Critical
> Attachments: olt.tar, Overlapping Pyramids Toy Dataset.pdf
>
>
> Hello,
> Apologies if I have not labeled this correctly.
> I have run a toy problem on Mahout 0.3 (locally) for LDA that I used to test Blei's c version of LDA that he posts on his site. It has an exact solution that the LDA should converge to. Please see attached PDF that describes the intended output.
> Is LDA working? The following output indicates some sort of collapsing behavior to me.
> T0 T1 T2 T3 T4
> x w x u x
> u u g j n
> l r i m l
> j q h h p
> v p e i q
> e t f g v
> d s d f o
> b c b n k
> y f c l m
> w v u v u
> c d p y t
> k o l r r
> i b j k j
> f e k e f
> g x y s y
> t y w b w
> h i s p s
> o l v x d
> q j t d i
> n k o t b
> The intended output is (again, please see attached):
> D I N S X
> d i n s x
> c h m t y
> e j o r w
> b k l u v
> f g p q a
> a f k p b
> g l q v u
> h m j w t
> y u r o c
> n s d d i
> s e x f f
> r q i i n
> m v w c o
> o w u a h
> q n s h g
> p t c x d
> t x f e l
> x d e j s
> w y g b j
> i r y n r
> u o h y m
> k b t l e
> v c a m k
> j a b g p
> l p v k q
> What tests do you run to make sure the output is correct?
> Thank you,
> Mike.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-399) LDA on Mahout 0.3 does not converge
to correct solution for overlapping pyramids toy problem.
Posted by "David Hall (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870954#action_12870954 ]
David Hall commented on MAHOUT-399:
-----------------------------------
LDA, as most unsupervised learning problems are, does not present as a convex problem. Given different starting points, minor (but still correct) implementation differences, you can arrive at different results.
That's not to say that there isn't a bug here, but just because we don't converge to the same value doesn't mean that there is a bug.
> LDA on Mahout 0.3 does not converge to correct solution for overlapping pyramids toy problem.
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-399
> URL: https://issues.apache.org/jira/browse/MAHOUT-399
> Project: Mahout
> Issue Type: Bug
> Components: Classification
> Affects Versions: 0.3
> Environment: Mac OS X 10.6.2, Hadoop 0.20.2, Mahout 0.3.
> Reporter: Michael Lazarus
> Priority: Critical
> Attachments: olt.tar, Overlapping Pyramids Toy Dataset.pdf
>
>
> Hello,
> Apologies if I have not labeled this correctly.
> I have run a toy problem on Mahout 0.3 (locally) for LDA that I used to test Blei's c version of LDA that he posts on his site. It has an exact solution that the LDA should converge to. Please see attached PDF that describes the intended output.
> Is LDA working? The following output indicates some sort of collapsing behavior to me.
> T0 T1 T2 T3 T4
> x w x u x
> u u g j n
> l r i m l
> j q h h p
> v p e i q
> e t f g v
> d s d f o
> b c b n k
> y f c l m
> w v u v u
> c d p y t
> k o l r r
> i b j k j
> f e k e f
> g x y s y
> t y w b w
> h i s p s
> o l v x d
> q j t d i
> n k o t b
> The intended output is (again, please see attached):
> D I N S X
> d i n s x
> c h m t y
> e j o r w
> b k l u v
> f g p q a
> a f k p b
> g l q v u
> h m j w t
> y u r o c
> n s d d i
> s e x f f
> r q i i n
> m v w c o
> o w u a h
> q n s h g
> p t c x d
> t x f e l
> x d e j s
> w y g b j
> i r y n r
> u o h y m
> k b t l e
> v c a m k
> j a b g p
> l p v k q
> What tests do you run to make sure the output is correct?
> Thank you,
> Mike.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-399) LDA on Mahout 0.3 does not converge
to correct solution for overlapping pyramids toy problem.
Posted by "Michael Lazarus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871039#action_12871039 ]
Michael Lazarus commented on MAHOUT-399:
----------------------------------------
Hi David,
Thank you for responding. I agree that you can arrive at different results with slightly different implementations. And also sometimes LDA gets stuck in a local minimum. In the pdf, for example, looking at the log-likelihood of the corpus given the model across models with different numbers of topics, you can see that Blei's implementation of LDA gets stuck on a ten topic model.
This dataset was designed to have just enough structure to demonstrate some of the behavior of the algorithm while at the same time defining an underlying model that should be readily discoverable by an implementation.
I have found it useful in debugging various systems that use LDA.
Thanks,
Mike.
> LDA on Mahout 0.3 does not converge to correct solution for overlapping pyramids toy problem.
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-399
> URL: https://issues.apache.org/jira/browse/MAHOUT-399
> Project: Mahout
> Issue Type: Bug
> Components: Classification
> Affects Versions: 0.3
> Environment: Mac OS X 10.6.2, Hadoop 0.20.2, Mahout 0.3.
> Reporter: Michael Lazarus
> Priority: Critical
> Attachments: olt.tar, Overlapping Pyramids Toy Dataset.pdf
>
>
> Hello,
> Apologies if I have not labeled this correctly.
> I have run a toy problem on Mahout 0.3 (locally) for LDA that I used to test Blei's c version of LDA that he posts on his site. It has an exact solution that the LDA should converge to. Please see attached PDF that describes the intended output.
> Is LDA working? The following output indicates some sort of collapsing behavior to me.
> T0 T1 T2 T3 T4
> x w x u x
> u u g j n
> l r i m l
> j q h h p
> v p e i q
> e t f g v
> d s d f o
> b c b n k
> y f c l m
> w v u v u
> c d p y t
> k o l r r
> i b j k j
> f e k e f
> g x y s y
> t y w b w
> h i s p s
> o l v x d
> q j t d i
> n k o t b
> The intended output is (again, please see attached):
> D I N S X
> d i n s x
> c h m t y
> e j o r w
> b k l u v
> f g p q a
> a f k p b
> g l q v u
> h m j w t
> y u r o c
> n s d d i
> s e x f f
> r q i i n
> m v w c o
> o w u a h
> q n s h g
> p t c x d
> t x f e l
> x d e j s
> w y g b j
> i r y n r
> u o h y m
> k b t l e
> v c a m k
> j a b g p
> l p v k q
> What tests do you run to make sure the output is correct?
> Thank you,
> Mike.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-399) LDA on Mahout 0.3 does not converge
to correct solution for overlapping pyramids toy problem.
Posted by "Jeff Eastman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870840#action_12870840 ]
Jeff Eastman commented on MAHOUT-399:
-------------------------------------
The LDA unit tests are in org.apache.mahout.clustering.lda and are not extensive. It would be really nice to get the same answer. Can you turn your example into another test case?
> LDA on Mahout 0.3 does not converge to correct solution for overlapping pyramids toy problem.
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-399
> URL: https://issues.apache.org/jira/browse/MAHOUT-399
> Project: Mahout
> Issue Type: Bug
> Components: Classification
> Affects Versions: 0.3
> Environment: Mac OS X 10.6.2, Hadoop 0.20.2, Mahout 0.3.
> Reporter: Michael Lazarus
> Priority: Critical
> Attachments: olt.tar, Overlapping Pyramids Toy Dataset.pdf
>
>
> Hello,
> Apologies if I have not labeled this correctly.
> I have run a toy problem on Mahout 0.3 (locally) for LDA that I used to test Blei's c version of LDA that he posts on his site. It has an exact solution that the LDA should converge to. Please see attached PDF that describes the intended output.
> Is LDA working? The following output indicates some sort of collapsing behavior to me.
> T0 T1 T2 T3 T4
> x w x u x
> u u g j n
> l r i m l
> j q h h p
> v p e i q
> e t f g v
> d s d f o
> b c b n k
> y f c l m
> w v u v u
> c d p y t
> k o l r r
> i b j k j
> f e k e f
> g x y s y
> t y w b w
> h i s p s
> o l v x d
> q j t d i
> n k o t b
> The intended output is (again, please see attached):
> D I N S X
> d i n s x
> c h m t y
> e j o r w
> b k l u v
> f g p q a
> a f k p b
> g l q v u
> h m j w t
> y u r o c
> n s d d i
> s e x f f
> r q i i n
> m v w c o
> o w u a h
> q n s h g
> p t c x d
> t x f e l
> x d e j s
> w y g b j
> i r y n r
> u o h y m
> k b t l e
> v c a m k
> j a b g p
> l p v k q
> What tests do you run to make sure the output is correct?
> Thank you,
> Mike.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.