You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org> on 2011/04/17 22:48:05 UTC
[jira] [Created] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution vectors
Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution vectors
---------------------------------------------------------------------------------------------
Key: MAHOUT-673
URL: https://issues.apache.org/jira/browse/MAHOUT-673
Project: Mahout
Issue Type: Improvement
Affects Versions: 0.4
Reporter: Dmitriy Lyubimov
Assignee: Dmitriy Lyubimov
Priority: Minor
Fix For: 0.6
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian().
I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Attachment: MAHOUT-673-1.patch
Ok... This should be better.
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
> Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Description:
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian().
I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
was:
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian().
I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Status: Patch Available (was: Open)
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
> Attachments: MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020880#comment-13020880 ]
Dmitriy Lyubimov commented on MAHOUT-673:
-----------------------------------------
Oh. This of course is not right.
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
> Attachments: MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Fix Version/s: 0.5
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.5, 0.6
>
> Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13024888#comment-13024888 ]
Dmitriy Lyubimov commented on MAHOUT-673:
-----------------------------------------
Ok, i am closing it. I really hoped Ted could pitch in with review, so if anybody wants to add something in regards to this change, please post to the list.
Thank you.
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.5, 0.6
>
> Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.5, 0.6
>
> Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Description:
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
was:
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian().
I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023157#comment-13023157 ]
Hudson commented on MAHOUT-673:
-------------------------------
Integrated in Mahout-Quality #769 (See [https://builds.apache.org/hudson/job/Mahout-Quality/769/])
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
> Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Summary: Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements (was: Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution vectors)
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use
64bit murmur hash to produce uniform distribution matrix elements
Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------
Attachment: MAHOUT-673-1.patch
Done. This is implied to go on top of not-yet-committed MAHOUT-638 patch since it applies some changes on top. But should be fine for review.
Yes-- Omega matrix could be generalized to implement Matrix API but for now it is kept specialized to stochastic projection. Future work may derive such generalization for the rest of the code to use.
> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-673
> URL: https://issues.apache.org/jira/browse/MAHOUT-673
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.4
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Priority: Minor
> Fix For: 0.6
>
> Attachments: MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian().
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this.
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira