You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org> on 2011/04/17 22:48:05 UTC

[jira] [Created] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution vectors

Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution vectors
---------------------------------------------------------------------------------------------

                 Key: MAHOUT-673
                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
             Project: Mahout
          Issue Type: Improvement
    Affects Versions: 0.4
            Reporter: Dmitriy Lyubimov
            Assignee: Dmitriy Lyubimov
            Priority: Minor
             Fix For: 0.6


So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian(). 

I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 

The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Attachment: MAHOUT-673-1.patch

Ok... This should be better.

> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Description: 
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian(). 

I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 

The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.

Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

  was:
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian(). 

I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 

The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.


> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Status: Patch Available  (was: Open)

> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020880#comment-13020880 ] 

Dmitriy Lyubimov commented on MAHOUT-673:
-----------------------------------------

Oh. This of course is not right.

> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Fix Version/s: 0.5

> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.5, 0.6
>
>         Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13024888#comment-13024888 ] 

Dmitriy Lyubimov commented on MAHOUT-673:
-----------------------------------------

Ok, i am closing it. I really hoped Ted could pitch in with review, so if anybody wants to add something in regards to this change, please post to the list. 
Thank you. 


> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.5, 0.6
>
>         Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.5, 0.6
>
>         Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Description: 
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 

I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 

The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.

Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

  was:
So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian(). 

I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 

The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.

Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.


> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023157#comment-13023157 ] 

Hudson commented on MAHOUT-673:
-------------------------------

Integrated in Mahout-Quality #769 (See [https://builds.apache.org/hudson/job/Mahout-Quality/769/])
    

> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: MAHOUT-673-1.patch, MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Summary: Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements  (was: Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution vectors)

> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval [-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-673) Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements

Posted by "Dmitriy Lyubimov (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-673:
------------------------------------

    Attachment: MAHOUT-673-1.patch

Done. This is implied to go on top of not-yet-committed MAHOUT-638 patch since it applies some changes on top. But should be fine for review. 

Yes-- Omega matrix could be generalized to implement Matrix API but for now it is kept specialized to stochastic projection. Future work may derive such generalization for the rest of the code to use.


> Stochastic projection (SSVD) to use 64bit murmur hash to produce uniform distribution matrix elements
> -----------------------------------------------------------------------------------------------------
>
>                 Key: MAHOUT-673
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-673
>             Project: Mahout
>          Issue Type: Improvement
>    Affects Versions: 0.4
>            Reporter: Dmitriy Lyubimov
>            Assignee: Dmitriy Lyubimov
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: MAHOUT-673-1.patch
>
>
> So, per earlier discussion on the list: for random matrix Omega in stochastic projection, let's use murmur hash to generate uniformly distributed elements in a closed interval (-1,+1] instead of using Random.nextGaussian(). 
> I am not sure if there's really compelling mathematical reason to do this but maybe it's just faster and more inline with practice accepted in Mahout for all this. 
> The murmur 64bit value is already in the code. I just need to figure the optimal way to convert it into a uniform distribution.
> Github url for this issue tree: https://github.com/dlyubimov/mahout-commits/branches/MAHOUT-673, pull requests are welcome.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira