You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Robin Anil (JIRA)" <ji...@apache.org> on 2010/02/07 19:52:28 UTC
[jira] Created: (MAHOUT-277) Increase number of entries in memory
per chunk of dictionary
Increase number of entries in memory per chunk of dictionary
-------------------------------------------------------------
Key: MAHOUT-277
URL: https://issues.apache.org/jira/browse/MAHOUT-277
Project: Mahout
Issue Type: Improvement
Affects Versions: 0.3
Reporter: Robin Anil
Assignee: Robin Anil
Fix For: 0.3
Java HashMap was having a huge overhead that was decreasing the number of entries per chunk hence increases number of passes over the data
Dictionary was using Text,LongWritable key. Integer is enough to keep the feature ids.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-277) Increase number of entries in memory
per chunk of dictionary
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Anil resolved MAHOUT-277.
-------------------------------
Resolution: Fixed
Closing for now. Will reopen later
> Increase number of entries in memory per chunk of dictionary
> -------------------------------------------------------------
>
> Key: MAHOUT-277
> URL: https://issues.apache.org/jira/browse/MAHOUT-277
> Project: Mahout
> Issue Type: Improvement
> Components: Utils
> Affects Versions: 0.3
> Reporter: Robin Anil
> Assignee: Robin Anil
> Fix For: 0.3
>
> Attachments: MAHOUT-277.patch
>
>
> Java HashMap was having a huge overhead that was decreasing the number of entries per chunk hence increases number of passes over the data
> Dictionary was using Text,LongWritable key. Integer is enough to keep the feature ids.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-277) Increase number of entries in memory
per chunk of dictionary
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Anil updated MAHOUT-277:
------------------------------
Component/s: Utils
> Increase number of entries in memory per chunk of dictionary
> -------------------------------------------------------------
>
> Key: MAHOUT-277
> URL: https://issues.apache.org/jira/browse/MAHOUT-277
> Project: Mahout
> Issue Type: Improvement
> Components: Utils
> Affects Versions: 0.3
> Reporter: Robin Anil
> Assignee: Robin Anil
> Fix For: 0.3
>
> Attachments: MAHOUT-277.patch
>
>
> Java HashMap was having a huge overhead that was decreasing the number of entries per chunk hence increases number of passes over the data
> Dictionary was using Text,LongWritable key. Integer is enough to keep the feature ids.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-277) Increase number of entries in memory
per chunk of dictionary
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830782#action_12830782 ]
Sean Owen commented on MAHOUT-277:
----------------------------------
Seems fine to commit. I feel that JIRA issues should have defined end points, so rather than keep this open (especially as its marked as blocking 0.3) I'd say reopen a new issue later if you think of more changes.
> Increase number of entries in memory per chunk of dictionary
> -------------------------------------------------------------
>
> Key: MAHOUT-277
> URL: https://issues.apache.org/jira/browse/MAHOUT-277
> Project: Mahout
> Issue Type: Improvement
> Components: Utils
> Affects Versions: 0.3
> Reporter: Robin Anil
> Assignee: Robin Anil
> Fix For: 0.3
>
> Attachments: MAHOUT-277.patch
>
>
> Java HashMap was having a huge overhead that was decreasing the number of entries per chunk hence increases number of passes over the data
> Dictionary was using Text,LongWritable key. Integer is enough to keep the feature ids.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-277) Increase number of entries in memory
per chunk of dictionary
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Anil updated MAHOUT-277:
------------------------------
Attachment: MAHOUT-277.patch
Uses our supercharged OpenObjectIntHashMap and changed feature id to int. Now fits double the number of entries given a fixed chunk size
> Increase number of entries in memory per chunk of dictionary
> -------------------------------------------------------------
>
> Key: MAHOUT-277
> URL: https://issues.apache.org/jira/browse/MAHOUT-277
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.3
> Reporter: Robin Anil
> Assignee: Robin Anil
> Fix For: 0.3
>
> Attachments: MAHOUT-277.patch
>
>
> Java HashMap was having a huge overhead that was decreasing the number of entries per chunk hence increases number of passes over the data
> Dictionary was using Text,LongWritable key. Integer is enough to keep the feature ids.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-277) Increase number of entries in memory
per chunk of dictionary
Posted by "Robin Anil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830744#action_12830744 ]
Robin Anil commented on MAHOUT-277:
-----------------------------------
Keeping it open for further improvements.
> Increase number of entries in memory per chunk of dictionary
> -------------------------------------------------------------
>
> Key: MAHOUT-277
> URL: https://issues.apache.org/jira/browse/MAHOUT-277
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.3
> Reporter: Robin Anil
> Assignee: Robin Anil
> Fix For: 0.3
>
> Attachments: MAHOUT-277.patch
>
>
> Java HashMap was having a huge overhead that was decreasing the number of entries per chunk hence increases number of passes over the data
> Dictionary was using Text,LongWritable key. Integer is enough to keep the feature ids.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.