You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Ning Li (JIRA)" <ji...@apache.org> on 2009/03/13 23:44:50 UTC
[jira] Created: (HADOOP-5491) Better control memory usage in
contrib/index
Better control memory usage in contrib/index
--------------------------------------------
Key: HADOOP-5491
URL: https://issues.apache.org/jira/browse/HADOOP-5491
Project: Hadoop Core
Issue Type: Bug
Reporter: Ning Li
Priority: Minor
The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting reassigned HADOOP-5491:
------------------------------------
Assignee: Ning Li
> Hmm, "submit patch" is not shown as an available workflow action.
How about now that the issue is assigned to you?
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Status: In Progress (was: Patch Available)
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Attachment: hadoop-5491.patch
The patch fixes memory usage problem, so there is no new or modified test.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682335#action_12682335 ]
Ning Li commented on HADOOP-5491:
---------------------------------
Hmm, "submit patch" is not shown as an available workflow action.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682412#action_12682412 ]
Hadoop QA commented on HADOOP-5491:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12402288/hadoop-5491.patch
against trunk revision 754927.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/91/console
This message is automatically generated.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Attachment: hadoop-5491.patch
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Status: Patch Available (was: In Progress)
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-5491:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Ning!
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688927#action_12688927 ]
Hadoop QA commented on HADOOP-5491:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12403541/hadoop-5491.patch
against trunk revision 757958.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/132/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/132/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/132/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/132/console
This message is automatically generated.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681930#action_12681930 ]
Hadoop QA commented on HADOOP-5491:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12402173/hadoop-5491.patch
against trunk revision 753449.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-minerva.apache.org/64/console
This message is automatically generated.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Status: Patch Available (was: In Progress)
"Submit patch" is available again now that I'm assignee. Thanks, Doug.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688961#action_12688961 ]
Ning Li commented on HADOOP-5491:
---------------------------------
The failed test is not related to this patch.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Status: In Progress (was: Patch Available)
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Attachment: hadoop-5491.patch
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682755#action_12682755 ]
Doug Cutting commented on HADOOP-5491:
--------------------------------------
> I used dos2unix, but Hudson still cannot apply patch.
The problem is that the committed code uses CRLF and your patch uses only LF.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Attachment: hadoop-5491.patch
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Li updated HADOOP-5491:
----------------------------
Status: Patch Available (was: Open)
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681926#action_12681926 ]
Ning Li commented on HADOOP-5491:
---------------------------------
A configuration parameter "sea.max.ramsize.bytes" is added to limit the size of a ram index created by a combiner.
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5491) Better control memory usage in
contrib/index
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682516#action_12682516 ]
Ning Li commented on HADOOP-5491:
---------------------------------
I used dos2unix, but Hudson still cannot apply patch. Anyone has an idea why?
> Better control memory usage in contrib/index
> --------------------------------------------
>
> Key: HADOOP-5491
> URL: https://issues.apache.org/jira/browse/HADOOP-5491
> Project: Hadoop Core
> Issue Type: Bug
> Reporter: Ning Li
> Assignee: Ning Li
> Priority: Minor
> Attachments: hadoop-5491.patch, hadoop-5491.patch, hadoop-5491.patch
>
>
> The combiner was originally designed to work only on the map side. When used on the reduce side, it may use too much memory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.