You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Karl Wettin (JIRA)" <ji...@apache.org> on 2008/07/08 20:24:31 UTC
[jira] Updated: (MAHOUT-61) Text problem matrix builder
[ https://issues.apache.org/jira/browse/MAHOUT-61?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated MAHOUT-61:
------------------------------
Attachment: MAHOUT-61.txt
This is what it is now:
1. InstanceHandler gathers instances
2. TokenizationMapper, Reducer and Combiner create one intermediate MapWritiable instance (see [4]). These are reduced down to unique feature names and class values.
3. The features and class values are placed in maps, assigned column index and numeric values, and stored as MapFile on DFS.
4. VectorBuilderMapper is a Mapping only job that use the results from [2] and [3] to produce sparse vectors.
> Text problem matrix builder
> ----------------------------
>
> Key: MAHOUT-61
> URL: https://issues.apache.org/jira/browse/MAHOUT-61
> Project: Mahout
> Issue Type: New Feature
> Reporter: Karl Wettin
> Assignee: Karl Wettin
> Priority: Minor
> Attachments: MAHOUT-61.txt, MAHOUT-61.txt, MAHOUT-61.txt
>
>
> A set of classes that builds matrices from text.
> Currently the API consists of TokenMatrixBuilder and TokenInstanceBuilder. Should be thread safe.
> PostReader imports 20news-bydate. This takes several GB heap. It would be nice to bounce the data via JDBM or perhaps using the PersistentHashMap in MAHOUT-19.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.