You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Grant Ingersoll (Created) (JIRA)" <ji...@apache.org> on 2011/11/04 18:13:51 UTC
[jira] [Created] (MAHOUT-873) Provide MapReduce job for creating
Encoded Vectors from sequence files
Provide MapReduce job for creating Encoded Vectors from sequence files
----------------------------------------------------------------------
Key: MAHOUT-873
URL: https://issues.apache.org/jira/browse/MAHOUT-873
Project: Mahout
Issue Type: Improvement
Reporter: Grant Ingersoll
Assignee: Grant Ingersoll
Fix For: 0.6
Similar to SparseVectorsFromSequenceFiles, provide a version that can do encoded vectors. Start simple by handling basic text, but this could easily evolve to handle pluggable Vectorizer's that can better deal with features (numerics, etc.).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-873) Provide MapReduce job for creating
Encoded Vectors from sequence files
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144547#comment-13144547 ]
Hudson commented on MAHOUT-873:
-------------------------------
Integrated in Mahout-Quality #1151 (See [https://builds.apache.org/job/Mahout-Quality/1151/])
MAHOUT-873: baseline of simple vectorization encoding capabilities
gsingers : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1197839
Files :
* /mahout/trunk/core/src/main/java/org/apache/mahout/common/AbstractJob.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/common/ClassUtils.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/common/HadoopUtil.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/vectorizer/DictionaryVectorizer.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/vectorizer/EncodedVectorsFromSequenceFiles.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/vectorizer/EncodingMapper.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/vectorizer/SimpleTextEncodingVectorizer.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/vectorizer/SparseVectorsFromSequenceFiles.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/vectorizer/Vectorizer.java
* /mahout/trunk/core/src/main/java/org/apache/mahout/vectorizer/VectorizerConfig.java
* /mahout/trunk/core/src/test/java/org/apache/mahout/vectorizer/EncodedVectorsFromSequenceFilesTest.java
* /mahout/trunk/examples/bin/build-asf-email.sh
* /mahout/trunk/src/conf/driver.classes.props
> Provide MapReduce job for creating Encoded Vectors from sequence files
> ----------------------------------------------------------------------
>
> Key: MAHOUT-873
> URL: https://issues.apache.org/jira/browse/MAHOUT-873
> Project: Mahout
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Labels: MAHOUT_INTRO_CONTRIBUTE
> Fix For: 0.6
>
> Attachments: MAHOUT-873.patch, MAHOUT-873.patch, MAHOUT-873.patch
>
>
> Similar to SparseVectorsFromSequenceFiles, provide a version that can do encoded vectors. Start simple by handling basic text, but this could easily evolve to handle pluggable Vectorizer's that can better deal with features (numerics, etc.).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAHOUT-873) Provide MapReduce job for creating
Encoded Vectors from sequence files
Posted by "Grant Ingersoll (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll resolved MAHOUT-873.
------------------------------------
Resolution: Fixed
> Provide MapReduce job for creating Encoded Vectors from sequence files
> ----------------------------------------------------------------------
>
> Key: MAHOUT-873
> URL: https://issues.apache.org/jira/browse/MAHOUT-873
> Project: Mahout
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Labels: MAHOUT_INTRO_CONTRIBUTE
> Fix For: 0.6
>
> Attachments: MAHOUT-873.patch, MAHOUT-873.patch, MAHOUT-873.patch
>
>
> Similar to SparseVectorsFromSequenceFiles, provide a version that can do encoded vectors. Start simple by handling basic text, but this could easily evolve to handle pluggable Vectorizer's that can better deal with features (numerics, etc.).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-873) Provide MapReduce job for creating
Encoded Vectors from sequence files
Posted by "Grant Ingersoll (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated MAHOUT-873:
-----------------------------------
Attachment: MAHOUT-873.patch
Patch that does basic work. Also refactors AbstractJob and HadoopUtil a bit to make it easier to use PrepareJob
> Provide MapReduce job for creating Encoded Vectors from sequence files
> ----------------------------------------------------------------------
>
> Key: MAHOUT-873
> URL: https://issues.apache.org/jira/browse/MAHOUT-873
> Project: Mahout
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Fix For: 0.6
>
> Attachments: MAHOUT-873.patch
>
>
> Similar to SparseVectorsFromSequenceFiles, provide a version that can do encoded vectors. Start simple by handling basic text, but this could easily evolve to handle pluggable Vectorizer's that can better deal with features (numerics, etc.).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-873) Provide MapReduce job for creating
Encoded Vectors from sequence files
Posted by "Grant Ingersoll (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated MAHOUT-873:
-----------------------------------
Attachment: MAHOUT-873.patch
Progress. Extracted a Vectorizer interface, made the encoder pluggable, various other goodness.
Also started to flesh out hooking in seq2encoded to build-asf-email so as to run SGD over the ASF email archive. Now just need train test per MAHOUT-851.
> Provide MapReduce job for creating Encoded Vectors from sequence files
> ----------------------------------------------------------------------
>
> Key: MAHOUT-873
> URL: https://issues.apache.org/jira/browse/MAHOUT-873
> Project: Mahout
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Fix For: 0.6
>
> Attachments: MAHOUT-873.patch, MAHOUT-873.patch
>
>
> Similar to SparseVectorsFromSequenceFiles, provide a version that can do encoded vectors. Start simple by handling basic text, but this could easily evolve to handle pluggable Vectorizer's that can better deal with features (numerics, etc.).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-873) Provide MapReduce job for creating
Encoded Vectors from sequence files
Posted by "Grant Ingersoll (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144481#comment-13144481 ]
Grant Ingersoll commented on MAHOUT-873:
----------------------------------------
I've checked in some baseline functionality here. Going to leave this open, as I think we can really take this to an interesting level of capabilities.
> Provide MapReduce job for creating Encoded Vectors from sequence files
> ----------------------------------------------------------------------
>
> Key: MAHOUT-873
> URL: https://issues.apache.org/jira/browse/MAHOUT-873
> Project: Mahout
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Fix For: 0.6
>
> Attachments: MAHOUT-873.patch, MAHOUT-873.patch, MAHOUT-873.patch
>
>
> Similar to SparseVectorsFromSequenceFiles, provide a version that can do encoded vectors. Start simple by handling basic text, but this could easily evolve to handle pluggable Vectorizer's that can better deal with features (numerics, etc.).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-873) Provide MapReduce job for creating
Encoded Vectors from sequence files
Posted by "Grant Ingersoll (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated MAHOUT-873:
-----------------------------------
Attachment: MAHOUT-873.patch
Starts the conversion of DictionaryVectorizer. I now think we could fold SparseVectorsFromSequenceFiles and EncodedVectorsFromSequenceFiles, perhaps, into one class, if we use appropriate command line grouping of options.
> Provide MapReduce job for creating Encoded Vectors from sequence files
> ----------------------------------------------------------------------
>
> Key: MAHOUT-873
> URL: https://issues.apache.org/jira/browse/MAHOUT-873
> Project: Mahout
> Issue Type: Improvement
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Fix For: 0.6
>
> Attachments: MAHOUT-873.patch, MAHOUT-873.patch, MAHOUT-873.patch
>
>
> Similar to SparseVectorsFromSequenceFiles, provide a version that can do encoded vectors. Start simple by handling basic text, but this could easily evolve to handle pluggable Vectorizer's that can better deal with features (numerics, etc.).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira