You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Ted Dunning (JIRA)" <ji...@apache.org> on 2012/08/17 22:23:37 UTC
[jira] [Created] (MAHOUT-1059) New matrix extensions
Ted Dunning created MAHOUT-1059:
-----------------------------------
Summary: New matrix extensions
Key: MAHOUT-1059
URL: https://issues.apache.org/jira/browse/MAHOUT-1059
Project: Mahout
Issue Type: Improvement
Components: Math
Reporter: Ted Dunning
Fix For: 0.8
The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning updated MAHOUT-1059:
--------------------------------
Attachment: 0002-MAHOUT-1059-Stylistic-cleanups.patch
0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch
Updated patches including indentation fixes.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MAHOUT-1059) New matrix extensions
Posted by "Paritosh Ranjan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437861#comment-13437861 ]
Paritosh Ranjan edited comment on MAHOUT-1059 at 8/21/12 12:48 AM:
-------------------------------------------------------------------
I looked at the code. The code is new, and I believe readable unit tests are door to new code. However, test methods like testUpdate() of CentroidTest having multiple asserts in it does not give much insight into what update is doing until someone goes through the whole code and understands it. I also think that the test case for WeightedVector is actually testing the DelegatingVector.
I did not find any test class for DelegatingVector, which I am attaching. To run this test case, EasyMock's dependency will be needed in math module.
was (Author: paritoshranjan):
I looked at the code. The code is new, and I believe readable unit tests are door to new code. However, test methods like testUpdate() of CentroidTest having multiple asserts in it does not give much insight into what update is doing until someone goes through the whole code and understands it. I also think that the test case for WeightedVector is actually testing the DelegatingVector.
I did not find any test class for DelegatingVector, which I am attaching.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning resolved MAHOUT-1059.
---------------------------------
Resolution: Fixed
Committed this given no further discussion.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning updated MAHOUT-1059:
--------------------------------
Attachment: 0004-MAHOUT-1059-Indentation.patch
0003-MAHOUT-1060-Move-distance-measures-to-math-as-much-a.patch
0003-MAHOUT-1059-Add-generic-vector-test.patch
0002-MAHOUT-1059-Stylistic-cleanups.patch
0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch
Here are some revised patches with a proposed AbstractVectorTest. This style allows a lot of redundant code to be eliminated and doesn't require mocking magic.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1060-Move-distance-measures-to-math-as-much-a.patch, 0004-MAHOUT-1059-Indentation.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438197#comment-13438197 ]
Ted Dunning commented on MAHOUT-1059:
-------------------------------------
Good comments. Your comment about which class is being tested is pretty much on target.
I would like to build an AbstractVectorTest for testing vectors in general. That should help a bit.
Is the mocking necessary? Adding the a mocking package as a test dependency is fine, but perhaps not yet needed.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1059) New matrix extensions
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447468#comment-13447468 ]
Hudson commented on MAHOUT-1059:
--------------------------------
Integrated in Mahout-Quality #1649 (See [https://builds.apache.org/job/Mahout-Quality/1649/])
MAHOUT-1059 - Needed for IndianBuffet test. (Revision 1380437)
MAHOUT-1059 - Additional test for weighted vectors. (Revision 1380433)
MAHOUT-1059 - Abstract the idea of a cached length. (Revision 1380432)
MAHOUT-1059 - Indentation (Revision 1380431)
MAHOUT-1059 - Add generic vector test. (Revision 1380430)
MAHOUT-1059 - Stylistic cleanups (Revision 1380429)
MAHOUT-1059 - Added Centroid, WeightedVector, DelegatingVector and several random thing generators. (Revision 1380428)
Result = FAILURE
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380437
Files :
* /mahout/trunk/math/src/test/resources/words.txt
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380433
Files :
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380432
Files :
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/AbstractVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/LengthCachingVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/MatrixVectorView.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/AbstractVectorTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedSparseBinaryMatrixTest.java
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380431
Files :
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380430
Files :
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/AbstractVectorTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/CentroidTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380429
Files :
* /mahout/trunk/core/src/main/java/org/apache/mahout/clustering/meanshift/MeanShiftCanopyMapper.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/CentroidTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedMatrixTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1380428
Files :
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/Centroid.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/DelegatingVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/FileBasedMatrix.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/WeightedVector.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/AbstractSamplerFunction.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/ChineseRestaurant.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Empirical.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/IndianBuffet.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Missing.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/MultiNormal.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Multinomial.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Normal.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/PoissonSampler.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/Sampler.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/SyntheticVariable.java
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/random/WeightedThing.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/CentroidTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedMatrixTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/WeightedVectorTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/ChineseRestaurantTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/EmpiricalTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/IndianBuffetTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/MultiNormalTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/MultinomialTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/NormalTest.java
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/random/PoissonSamplerTest.java
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning updated MAHOUT-1059:
--------------------------------
Attachment: 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch
0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch
0004-MAHOUT-1059-Indentation.patch
0003-MAHOUT-1059-Add-generic-vector-test.patch
0002-MAHOUT-1059-Stylistic-cleanups.patch
0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch
Updated patches
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1059) New matrix extensions
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450197#comment-13450197 ]
Hudson commented on MAHOUT-1059:
--------------------------------
Integrated in Mahout-Quality #1653 (See [https://builds.apache.org/job/Mahout-Quality/1653/])
MAHOUT-1059 - Remove memory hungry test (Revision 1381780)
MAHOUT-1059 - Oops. Added test without the class. This class isn't really ready, but the build should work. (Revision 1381778)
Result = FAILURE
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381780
Files :
* /mahout/trunk/math/src/test/java/org/apache/mahout/math/FileBasedSparseBinaryMatrixTest.java
tdunning : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381778
Files :
* /mahout/trunk/math/src/main/java/org/apache/mahout/math/FileBasedSparseBinaryMatrix.java
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning updated MAHOUT-1059:
--------------------------------
Attachment: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch
Here is a patch that provides the matrix and vectors needed for the clustering stuff.
The missing bits are that I want to add a few more tests to DelegatingVector, Centroid and WeightedVector. We should also resolve the name conflict with the other Centroid class we have somewhere in Mahout (possibly by letting the conflict remain).
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447442#comment-13447442 ]
Ted Dunning commented on MAHOUT-1059:
-------------------------------------
I have added a fix/enhancement to the basic matrix to make handling of size caching more generic. The basic issue was that caching the squared length of a vector makes distance calculations with L_2 faster for sparse vectors. This should apply (or not) to all view-like vectors like DelegatingVector. So now it does.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, 0004-MAHOUT-1059-Indentation.patch, 0005-MAHOUT-1059-Abstract-the-idea-of-a-cached-length.patch, 0006-MAHOUT-1059-Additional-test-for-weighted-vectors.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-1059) New matrix extensions
Posted by "Paritosh Ranjan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438400#comment-13438400 ]
Paritosh Ranjan commented on MAHOUT-1059:
-----------------------------------------
I thought Mocking was the best way to test DelegatingVector.
I also think its necessary to test DelegatingVector ( though it looks pretty simple ), as there are lots of methods, and it would be really easy to inject bug but would be difficult to debug it.
I personally do not believe in creating generalized classes for testing. I think class ATest should be used to test A, BTest for B and so on. However, A should only test A's code, B should only test B's code and so on. Generalizing a class for writing test cases for a particular genre makes it difficult for a new guy to find the test cases.
However, in the end, what actually matters is that the code should work, so, all approaches are fine.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-1059) New matrix extensions
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Dunning updated MAHOUT-1059:
--------------------------------
Attachment: (was: 0003-MAHOUT-1060-Move-distance-measures-to-math-as-much-a.patch)
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, 0003-MAHOUT-1059-Add-generic-vector-test.patch, 0004-MAHOUT-1059-Indentation.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-1059) New matrix extensions
Posted by "Paritosh Ranjan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paritosh Ranjan updated MAHOUT-1059:
------------------------------------
Attachment: DelegatingVectorTest.java
I looked at the code. The code is new, and I believe readable unit tests are door to new code. However, test methods like testUpdate() of CentroidTest having multiple asserts in it does not give much insight into what update is doing until someone goes through the whole code and understands it. I also think that the test case for WeightedVector is actually testing the DelegatingVector.
I did not find any test class for DelegatingVector, which I am attaching.
> New matrix extensions
> ---------------------
>
> Key: MAHOUT-1059
> URL: https://issues.apache.org/jira/browse/MAHOUT-1059
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Reporter: Ted Dunning
> Fix For: 0.8
>
> Attachments: 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0001-MAHOUT-1059-Added-Centroid-WeightedVector-Delegating.patch, 0002-MAHOUT-1059-Stylistic-cleanups.patch, DelegatingVectorTest.java
>
>
> The upcoming clustering needs several capabilities to support different operations. These include some matrix extensions for adding behaviors to different kinds of matrices. Also there is a file based matrix that uses mmap to access a file as if it were a matrix in shared memory. Since this is off-heap and shared between processes, it can seriously help some programs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira