You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org> on 2011/06/30 07:32:28 UTC
[jira] [Created] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Avoid calling the partitioner when the numReduceTasks is 1.
-----------------------------------------------------------
Key: MAPREDUCE-2632
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: tasktracker
Affects Versions: 0.23.0
Reporter: Ravi Teja Ch N V
We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057857#comment-13057857 ]
Ravi Teja Ch N V commented on MAPREDUCE-2632:
---------------------------------------------
The failing tests are not related to this patch.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Teja Ch N V reassigned MAPREDUCE-2632:
-------------------------------------------
Assignee: Ravi Teja Ch N V
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Assignee: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632-1.patch, MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Teja Ch N V updated MAPREDUCE-2632:
----------------------------------------
Attachment: MAPREDUCE-2632.patch
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057633#comment-13057633 ]
Harsh J commented on MAPREDUCE-2632:
------------------------------------
Ravi,
Sorry for much noise! Not sure I understand your comment above… Do you intend to only apply fixes to some of the other remaining Partitioner implementations or is this change going to be deeper than that (as I'd thought)?
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Teja Ch N V updated MAPREDUCE-2632:
----------------------------------------
Status: Patch Available (was: Open)
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632-1.patch, MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Teja Ch N V updated MAPREDUCE-2632:
----------------------------------------
Attachment: MAPREDUCE-2632-1.patch
Rebased the patch to the latest version.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632-1.patch, MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100119#comment-13100119 ]
Hadoop QA commented on MAPREDUCE-2632:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12493554/MAPREDUCE-2632-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.server.nodemanager.TestNMAuditLogger
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/637//console
This message is automatically generated.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632-1.patch, MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Mostafa Elhemali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508424#comment-13508424 ]
Mostafa Elhemali commented on MAPREDUCE-2632:
---------------------------------------------
A few comments:
1. Stylistic nit-picks: else should be on same line as closing brace, and should have a space after. And "part=0" needs more space.
2. Can you put a comment (and test) that partitioner can be null if there's only one reader? One of the nice things about MAPREDUCE-1287 is that it doesn't even construct the partitioner if it's not needed, so we should help client code do that as well.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Assignee: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632-1.patch, MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-2632:
-------------------------------
Hadoop Flags: [Incompatible change]
This would be an incompatible change since API users may always expect a partitioner logic to get called despite reducer # settings, and the change may not even initialize it.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057634#comment-13057634 ]
Ravi Teja Ch N V commented on MAPREDUCE-2632:
---------------------------------------------
Yes, As part of MAPREDUCE-1287, It has handled for all the partitioner implementations but in this case, we can avoid the call for the partitioner itself.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Teja Ch N V updated MAPREDUCE-2632:
----------------------------------------
Status: Patch Available (was: Open)
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057820#comment-13057820 ]
Hadoop QA commented on MAPREDUCE-2632:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12484759/MAPREDUCE-2632.patch
against trunk revision 1140942.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.cli.TestMRCLI
org.apache.hadoop.fs.TestFileSystem
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/433//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/433//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/433//console
This message is automatically generated.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100143#comment-13100143 ]
Ravi Teja Ch N V commented on MAPREDUCE-2632:
---------------------------------------------
The and findbugs testcase failures are not related to this patch.Thanks.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632-1.patch, MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Ravi Teja Ch N V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057630#comment-13057630 ]
Ravi Teja Ch N V commented on MAPREDUCE-2632:
---------------------------------------------
The fix was missed in this case.
{code:title=MapFileOutputFormat.java|borderStyle=solid}
public static <K extends WritableComparable<?>, V extends Writable>
Writable getEntry(MapFile.Reader[] readers,
Partitioner<K, V> partitioner, K key, V value) throws IOException {
int part = partitioner.getPartition(key, value, readers.length);
return readers[part].get(key, value);
}
{code}
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2632:
-------------------------------------
Status: Open (was: Patch Available)
Sorry to come in late, the patch has gone stale. Can you please rebase? Thanks.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
> Attachments: MAPREDUCE-2632.patch
>
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057626#comment-13057626 ]
Harsh J commented on MAPREDUCE-2632:
------------------------------------
This went in as an Incompatible Change in 0.21.0. Has it been reverted away to demand a re-entry?
MAPREDUCE-1287 did the change earlier I think.
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2632) Avoid calling the partitioner
when the numReduceTasks is 1.
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057637#comment-13057637 ]
Harsh J commented on MAPREDUCE-2632:
------------------------------------
Thanks for clarifying Ravi :)
> Avoid calling the partitioner when the numReduceTasks is 1.
> -----------------------------------------------------------
>
> Key: MAPREDUCE-2632
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2632
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: tasktracker
> Affects Versions: 0.23.0
> Reporter: Ravi Teja Ch N V
>
> We can avoid the call to the partitioner when the number of reducers is 1.This will avoid the unnecessary computations by the partitioner.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira