You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Ted Dunning (JIRA)" <ji...@apache.org> on 2012/08/17 22:23:37 UTC

[jira] [Created] (MAHOUT-1059) New matrix extensions

Ted Dunning created MAHOUT-1059:
-----------------------------------

             Summary: New matrix extensions
                 Key: MAHOUT-1059
                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
             Project: Mahout
          Issue Type: Improvement
          Components: Math
            Reporter: Ted Dunning
             Fix For: 0.8


The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ted Dunning updated MAHOUT-1059:
--------------------------------

    Attachment: 0002-MAHOUT-1059-Stylistic-cleanups.patch
                0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch

Updated patches including indentation fixes.
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Comment Edited] (MAHOUT-1059) New matrix extensions

Posted by "Paritosh Ranjan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437861#comment-13437861 ] 

Paritosh Ranjan edited comment on MAHOUT-1059 at 8/21/12 12:48 AM:
-------------------------------------------------------------------

I looked at the code. The code is new, and I believe readable unit tests are door to new code. However, test methods like testUpdate() of CentroidTest having multiple asserts in it does not give much insight into what update is doing until someone goes through the whole code and understands it. I also think that the test case for WeightedVector is actually testing the DelegatingVector.

I did not find any test class for DelegatingVector, which I am attaching. To run this test case, EasyMock's dependency will be needed in math module.




                
      was (Author: paritoshranjan):
    I looked at the code. The code is new, and I believe readable unit tests are door to new code. However, test methods like testUpdate() of CentroidTest having multiple asserts in it does not give much insight into what update is doing until someone goes through the whole code and understands it. I also think that the test case for WeightedVector is actually testing the DelegatingVector.

I did not find any test class for DelegatingVector, which I am attaching.




                  
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Resolved] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ted Dunning resolved MAHOUT-1059.
---------------------------------

    Resolution: Fixed

Committed this given no further discussion.
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ted Dunning updated MAHOUT-1059:
--------------------------------

    Attachment: 0004-MAHOUT-1059-Indentation.patch
                0003-MAHOUT-1060-Move-distance-measures-to-math-as-much-a.patch
                0003-MAHOUT-1059-Add-generic-vector-test.patch
                0002-MAHOUT-1059-Stylistic-cleanups.patch
                0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch

Here are some revised patches with a proposed AbstractVectorTest.  This style allows a lot of redundant code to be eliminated and doesn't require mocking magic.
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1060-Move-distance-measures-to-math-as-much-a.patch, 0004-MAHOUT-1059-Indentation.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438197#comment-13438197 ] 

Ted Dunning commented on MAHOUT-1059:
-------------------------------------

Good comments.  Your comment about which class is being tested is pretty much on target.

I would like to build an AbstractVectorTest for testing vectors in general.  That should help a bit.

Is the mocking necessary?  Adding the a mocking package as a test dependency is fine, but perhaps not yet needed.
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAHOUT-1059) New matrix extensions

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447468#comment-13447468 ] 

Hudson commented on MAHOUT-1059:
--------------------------------

Integrated in Mahout-Quality #1649 (See [https://builds.apache.org/job/Mahout-Quality/1649/])
    MAHOUT-1059 - Needed for IndianBuffet test. (Revision 1380437)
MAHOUT-1059 - Additional test for weighted vectors. (Revision 1380433)
MAHOUT-1059 - Abstract the idea of a cached length. (Revision 1380432)
MAHOUT-1059 - Indentation (Revision 1380431)
MAHOUT-1059 - Add generic vector test. (Revision 1380430)
MAHOUT-1059 - Stylistic cleanups (Revision 1380429)
MAHOUT-1059 - Added Centroid, WeightedVector, DelegatingVector and several random thing generators. (Revision 1380428)

     Result = FAILURE
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380437
Files : 
* /mahout/trunk/math/src/test/resources/words.txt

tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380433
Files : 
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java

tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380432
Files : 
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/AbstractVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/LengthCachingVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/MatrixVectorView.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/AbstractVectorTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedSparseBinaryMatrixTest.java

tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380431
Files : 
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java

tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380430
Files : 
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/AbstractVectorTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/CentroidTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java

tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380429
Files : 
* /mahout/trunk/core/src/main/java/org/apache/mahout/clustering/meanshift/MeanShiftCanopyMapper.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/CentroidTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedMatrixTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java

tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380428
Files : 
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/Centroid.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/FileBasedMatrix.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/WeightedVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/AbstractSamplerFunction.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/ChineseRestaurant.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Empirical.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/IndianBuffet.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Missing.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/MultiNormal.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Multinomial.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Normal.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/PoissonSampler.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Sampler.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/SyntheticVariable.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/WeightedThing.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/CentroidTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedMatrixTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/ChineseRestaurantTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/EmpiricalTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/IndianBuffetTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/MultiNormalTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/MultinomialTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/NormalTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/PoissonSamplerTest.java

                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ted Dunning updated MAHOUT-1059:
--------------------------------

    Attachment: 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch
                0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch
                0004-MAHOUT-1059-Indentation.patch
                0003-MAHOUT-1059-Add-generic-vector-test.patch
                0002-MAHOUT-1059-Stylistic-cleanups.patch
                0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch

Updated patches
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAHOUT-1059) New matrix extensions

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450197#comment-13450197 ] 

Hudson commented on MAHOUT-1059:
--------------------------------

Integrated in Mahout-Quality #1653 (See [https://builds.apache.org/job/Mahout-Quality/1653/])
    MAHOUT-1059 - Remove memory hungry test (Revision 1381780)
MAHOUT-1059 - Oops.  Added test without the class.  This class isn't really ready, but the build should work. (Revision 1381778)

     Result = FAILURE
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381780
Files : 
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedSparseBinaryMatrixTest.java

tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381778
Files : 
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/FileBasedSparseBinaryMatrix.java

                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ted Dunning updated MAHOUT-1059:
--------------------------------

    Attachment: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch

Here is a patch that provides the matrix and vectors needed for the clustering stuff.

The missing bits are that I want to add a few more tests to DelegatingVector, Centroid and WeightedVector.  We should also resolve the name conflict with the other Centroid class we have somewhere in Mahout (possibly by letting the conflict remain).
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447442#comment-13447442 ] 

Ted Dunning commented on MAHOUT-1059:
-------------------------------------

I have added a fix/enhancement to the basic matrix to make handling of size caching more generic.  The basic issue was that caching the squared length of a vector makes distance calculations with L_2 faster for sparse vectors.  This should apply (or not) to all view-like vectors like DelegatingVector.  So now it does.
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAHOUT-1059) New matrix extensions

Posted by "Paritosh Ranjan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438400#comment-13438400 ] 

Paritosh Ranjan commented on MAHOUT-1059:
-----------------------------------------

I thought Mocking was the best way to test DelegatingVector. 

I also think its necessary to test DelegatingVector ( though it looks pretty simple ), as there are lots of methods, and it would be really easy to inject bug but would be difficult to debug it.

I personally do not believe in creating generalized classes for testing. I think class ATest should be used to test A, BTest for B and so on. However, A should only test A's code, B should only test B's code and so on. Generalizing a class for writing test cases for a particular genre makes it difficult for a new guy to find the test cases.

However, in the end, what actually matters is that the code should work, so, all approaches are fine.
                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAHOUT-1059) New matrix extensions

Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ted Dunning updated MAHOUT-1059:
--------------------------------

    Attachment:     (was: 0003-MAHOUT-1060-Move-distance-measures-to-math-as-much-a.patch)
    
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (MAHOUT-1059) New matrix extensions

Posted by "Paritosh Ranjan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Paritosh Ranjan updated MAHOUT-1059:
------------------------------------

    Attachment: DelegatingVectorTest.java

I looked at the code. The code is new, and I believe readable unit tests are door to new code. However, test methods like testUpdate() of CentroidTest having multiple asserts in it does not give much insight into what update is doing until someone goes through the whole code and understands it. I also think that the test case for WeightedVector is actually testing the DelegatingVector.

I did not find any test class for DelegatingVector, which I am attaching.




                
> New matrix extensions
> ---------------------
>
>                 Key: MAHOUT-1059
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1059
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>            Reporter: Ted Dunning
>             Fix For: 0.8
>
>         Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations.  These include some matrix extensions for adding behaviors to different kinds of matrices.  Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory.  Since this is off-heap and shared between processes, it can seriously help some programs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira