You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Siddharth Seth (Created) (JIRA)" <ji...@apache.org> on 2012/02/07 20:48:59 UTC
[jira] [Created] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
---------------------------------------------------------------------------------------------------------------
Key: MAPREDUCE-3834
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Siddharth Seth
Priority: Critical
Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203142#comment-13203142 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #1689 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1689/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241734
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3834:
--------------------------------------
Assignee: Siddharth Seth
Status: Patch Available (was: Open)
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Attachments: MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203155#comment-13203155 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Common-0.23-Commit #517 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/517/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
svn merge --ignore-ancestry -c 1241734 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241735
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202915#comment-13202915 ]
Hadoop QA commented on MAPREDUCE-3834:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513671/MR3834.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.mapred.TestJobCounters
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1816//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1816//console
This message is automatically generated.
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Attachments: MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203198#comment-13203198 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Commit #521 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/521/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
svn merge --ignore-ancestry -c 1241734 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241735
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203154#comment-13203154 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Commit #505 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/505/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
svn merge --ignore-ancestry -c 1241734 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241735
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3834:
-----------------------------------------------
Status: Open (was: Patch Available)
Looks good overall. Very minor nits:
- I was confused by the test as to how the hosts all get resolved to the same rack. May be you can comment that they resolve to the default rack.
- when you do a racks.toArray(new String[0]), you could instead do racks.toArray(new String[racks.size()]). The underlying code does the resizing immediately, we could as well give the correct size.
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Attachments: MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203125#comment-13203125 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3834:
----------------------------------------------------
+!. lgtm now. Pushing this in.
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3834:
-----------------------------------------------
Resolution: Fixed
Release Note: Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
I just committed this to trunk and branch-0.23. Thanks Sid!
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203595#comment-13203595 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #983 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/983/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241734
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203570#comment-13203570 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Mapreduce-0.23-Build #185 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/185/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
svn merge --ignore-ancestry -c 1241734 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241735
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203005#comment-13203005 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3834:
----------------------------------------------------
Looking at this for review/commit.
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Attachments: MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3834:
--------------------------------------
Attachment: MR3834.txt
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Priority: Critical
> Attachments: MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203139#comment-13203139 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1763 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1763/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241734
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3834:
--------------------------------------
Attachment: MR3834.txt
Done. Thanks for the review.
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3834:
-----------------------------------------------
Component/s: mr-am
Fix Version/s: 0.23.1
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203525#comment-13203525 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #950 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/950/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241734
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203195#comment-13203195 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #1700 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1700/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241734
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203111#comment-13203111 ]
Hadoop QA commented on MAPREDUCE-3834:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513730/MR3834.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.mapred.TestJobCounters
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1819//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1819//console
This message is automatically generated.
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203546#comment-13203546 ]
Hudson commented on MAPREDUCE-3834:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #163 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/163/])
MAPREDUCE-3834. Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack. Contributed by Siddarth Seth.
svn merge --ignore-ancestry -c 1241734 ../../trunk/
vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1241735
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskAttempt.java
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Siddharth Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202702#comment-13202702 ]
Siddharth Seth commented on MAPREDUCE-3834:
-------------------------------------------
Applies on top of 3815. Will submit patch after that's done.
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Priority: Critical
> Attachments: MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3834) If multiple hosts for a split
belong to the same rack, the rack is added multiple times in the AM request
table
Posted by "Siddharth Seth (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated MAPREDUCE-3834:
--------------------------------------
Status: Patch Available (was: Open)
> If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3834
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3834
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am, mrv2
> Affects Versions: 0.23.0
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Priority: Critical
> Fix For: 0.23.1
>
> Attachments: MR3834.txt, MR3834.txt
>
>
> Should be added only once - so that the RM doesn't think there's multiple rack local requests for that particular rack.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira