You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ahmed Radwan (Created) (JIRA)" <ji...@apache.org> on 2011/12/01 10:37:40 UTC
[jira] [Created] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Add the default mapreduce.shuffle.port property to mapred-default.xml
---------------------------------------------------------------------
Key: MAPREDUCE-3493
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Ahmed Radwan
Assignee: Ahmed Radwan
Priority: Minor
I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284915#comment-13284915 ]
Hudson commented on MAPREDUCE-3493:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2369 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2369/])
MAPREDUCE-3493. Add the default mapreduce.shuffle.port property to mapred-default.xml. Contributed by Madhukara Phatak. (harsh) (Revision 1343804)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343804
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "madhukara phatak (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhukara phatak updated MAPREDUCE-3493:
----------------------------------------
Status: Patch Available (was: Open)
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250449#comment-13250449 ]
Harsh J commented on MAPREDUCE-3493:
------------------------------------
Yes thats right, but it is also a service rather than an instance every reducer runs. So perhaps a better description could be "Default port that the ShuffleHandler will run on. ShuffleHandler is a service run at the NodeManager, to facilitate transfers of intermediate Map outputs to requesting Reducers."
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "madhukara phatak (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhukara phatak updated MAPREDUCE-3493:
----------------------------------------
Attachment: MAPREDUCE-3493.patch
Changed mapred-default.xml to have this property
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "madhukara phatak (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhukara phatak updated MAPREDUCE-3493:
----------------------------------------
Attachment: MAPREDUCE-3493-3.patch
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13247051#comment-13247051 ]
Hadoop QA commented on MAPREDUCE-3493:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12521459/MAPREDUCE-3493.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2151//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2151//console
This message is automatically generated.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284940#comment-13284940 ]
Hudson commented on MAPREDUCE-3493:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2315 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2315/])
MAPREDUCE-3493. Add the default mapreduce.shuffle.port property to mapred-default.xml. Contributed by Madhukara Phatak. (harsh) (Revision 1343804)
Result = FAILURE
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343804
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "madhukara phatak (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250431#comment-13250431 ]
madhukara phatak commented on MAPREDUCE-3493:
---------------------------------------------
As per my knowledge shuffle handler is used by reducer to copy the intermediate map results from mapper nodes to the node where reducer runs. is that correct ?
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285590#comment-13285590 ]
Hudson commented on MAPREDUCE-3493:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1061 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1061/])
MAPREDUCE-3493. Add the default mapreduce.shuffle.port property to mapred-default.xml. Contributed by Madhukara Phatak. (harsh) (Revision 1343804)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343804
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250749#comment-13250749 ]
Hadoop QA commented on MAPREDUCE-3493:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522114/MAPREDUCE-3493-2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2182//console
This message is automatically generated.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250119#comment-13250119 ]
Harsh J commented on MAPREDUCE-3493:
------------------------------------
Looks good. Can you also elaborate a bit more on what the ShuffleHandler is, in the description field? May come handy to some.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Ahmed Radwan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmed Radwan updated MAPREDUCE-3493:
------------------------------------
Component/s: mrv2
Affects Version/s: 0.23.0
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285672#comment-13285672 ]
Hudson commented on MAPREDUCE-3493:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1095 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1095/])
MAPREDUCE-3493. Add the default mapreduce.shuffle.port property to mapred-default.xml. Contributed by Madhukara Phatak. (harsh) (Revision 1343804)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343804
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "madhukara phatak (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhukara phatak updated MAPREDUCE-3493:
----------------------------------------
Attachment: MAPREDUCE-3493-2.patch
Updated the description.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284803#comment-13284803 ]
Hadoop QA commented on MAPREDUCE-3493:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12530040/MAPREDUCE-3943-4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2422//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2422//console
This message is automatically generated.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "madhukara phatak (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252205#comment-13252205 ]
madhukara phatak commented on MAPREDUCE-3493:
---------------------------------------------
It strange that some times core test pass but some times they fail.Is that because some other process is running on port 8080 when jenkins runs the build??
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-3493:
-------------------------------
Attachment: MAPREDUCE-3943-4.patch
Your last patch had some issues:
# Your patch's targeted file was an old, obsolete one. The new ones lie under the client
# The patch also had a relative, sub-project path that isn't acceptable. The patch must be diffed from the root of the svn checkout.
I've submitted a new patch with these fixes, and will commit once QA gives a +1 again.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-3493:
-------------------------------
Attachment: MAPREDUCE-3943-4.patch
Re-upping patch cause these tests seem to pass locally. Lets try again.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252512#comment-13252512 ]
Hadoop QA commented on MAPREDUCE-3493:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522419/MAPREDUCE-3943-4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell
org.apache.hadoop.yarn.server.TestDiskFailures
org.apache.hadoop.yarn.server.TestContainerManagerSecurity
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapreduce.v2.TestMRJobs
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
org.apache.hadoop.mapred.TestMiniMRBringup
org.apache.hadoop.mapred.TestMiniMRChildTask
org.apache.hadoop.mapred.TestReduceFetch
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
org.apache.hadoop.mapred.TestJobCounters
org.apache.hadoop.mapreduce.TestChild
org.apache.hadoop.mapred.TestMiniMRClientCluster
org.apache.hadoop.ipc.TestSocketFactory
org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService
org.apache.hadoop.mapreduce.v2.TestMROldApiJobs
org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
org.apache.hadoop.mapred.TestClientRedirect
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapred.TestJobCleanup
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
org.apache.hadoop.mapreduce.v2.TestMRAppWithCombiner
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.mapreduce.v2.TestRMNMInfo
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
org.apache.hadoop.mapreduce.v2.TestNonExistentJob
org.apache.hadoop.mapred.TestJobSysDirWithDFS
org.apache.hadoop.mapreduce.v2.TestUberAM
org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser
org.apache.hadoop.mapred.TestJobName
org.apache.hadoop.mapreduce.security.TestJHSSecurity
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2212//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2212//console
This message is automatically generated.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284908#comment-13284908 ]
Hudson commented on MAPREDUCE-3493:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2296 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2296/])
MAPREDUCE-3493. Add the default mapreduce.shuffle.port property to mapred-default.xml. Contributed by Madhukara Phatak. (harsh) (Revision 1343804)
Result = SUCCESS
harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1343804
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-3493:
-------------------------------
Target Version/s: 2.0.1-alpha, 3.0.0 (was: 2.0.0-alpha)
Affects Version/s: (was: 0.23.0)
2.0.0-alpha
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "madhukara phatak (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
madhukara phatak updated MAPREDUCE-3493:
----------------------------------------
Attachment: MAPREDUCE-3493-1.patch
Fixed core tests errors.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-3493:
-------------------------------
Target Version/s: 2.0.0
Assignee: (was: Ahmed Radwan)
Hadoop Flags: Reviewed
+1. Patch looks good to me. Will commit it in shortly. Failing tests are unrelated to this doc change.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-3493:
-------------------------------
Resolution: Fixed
Fix Version/s: 2.0.1-alpha
Target Version/s: (was: 2.0.1-alpha, 3.0.0)
Status: Resolved (was: Patch Available)
Committed to trunk and backported to branch-2. Thanks for your contributions Madhukara!
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.1-alpha
>
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250781#comment-13250781 ]
Hadoop QA commented on MAPREDUCE-3493:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522120/MAPREDUCE-3493-3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2184//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2184//console
This message is automatically generated.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250098#comment-13250098 ]
Hadoop QA commented on MAPREDUCE-3493:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12521983/MAPREDUCE-3493-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2176//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2176//console
This message is automatically generated.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284897#comment-13284897 ]
Harsh J commented on MAPREDUCE-3493:
------------------------------------
bq. -1 tests included.
No tests are required as this is a documentation additive.
Committing to trunk shortly.
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-3493) Add the default
mapreduce.shuffle.port property to mapred-default.xml
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-3493.
------------------------------------
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-3493
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.0-alpha
> Reporter: Ahmed Radwan
> Priority: Minor
> Labels: newbie
> Fix For: 2.0.2-alpha
>
> Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493-2.patch, MAPREDUCE-3493-3.patch, MAPREDUCE-3493.patch, MAPREDUCE-3943-4.patch, MAPREDUCE-3943-4.patch
>
>
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node. The default value for this property is hardcoded in the ShuffleHandler.java class so it results in port conflicts. The issue is resolved if you set the property value in your conf files. But the absence of this property from *-default.xml files is confusing. So It'll be cleaner to move this property to mapred-default.xml, so its default value can be easily identified and changed if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira