You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org> on 2011/07/29 08:44:09 UTC
[jira] [Created] (MAPREDUCE-2750) [MR-279] NodeManager should start
its servers on ephemeral ports
[MR-279] NodeManager should start its servers on ephemeral ports
-----------------------------------------------------------------
Key: MAPREDUCE-2750
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Vinod Kumar Vavilapalli
Fix For: 0.23.0
There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2750) [MR-279] NodeManager should
start its servers on ephemeral ports
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13076288#comment-13076288 ]
Steve Loughran commented on MAPREDUCE-2750:
-------------------------------------------
How about making it an option to start on an ephemeral port, as in production it will only complicate firewall setup.
> [MR-279] NodeManager should start its servers on ephemeral ports
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2750
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
>
> There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
> The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2750) [MR-279] NodeManager should
start its servers on ephemeral ports
Posted by "Anupam Seth (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth reassigned MAPREDUCE-2750:
--------------------------------------
Assignee: Anupam Seth
> [MR-279] NodeManager should start its servers on ephemeral ports
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2750
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Anupam Seth
> Fix For: 0.23.0
>
>
> There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
> The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2750) [MR-279] NodeManager should
start its servers on ephemeral ports
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078857#comment-13078857 ]
Allen Wittenauer commented on MAPREDUCE-2750:
---------------------------------------------
So basically the argument for ephemeral ports is that it is too hard to provide different config files to run the 2+ services on the same box for testing?
I would think for testing predictability would be a great quality to have...
But yes, out of the box this needs to have fix port for production purposes. (... and not 8080! That's just stunningly bad.) For testing, it makes sense to provide a different config that turns on ephemeral-ness if that is really desired. (Which doesn't make sense to me at all, but whatever.)
> [MR-279] NodeManager should start its servers on ephemeral ports
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2750
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
>
> There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
> The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2750) [MR-279] NodeManager should
start its servers on ephemeral ports
Posted by "Vinod Kumar Vavilapalli (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli resolved MAPREDUCE-2750.
------------------------------------------------
Resolution: Duplicate
> [MR-279] NodeManager should start its servers on ephemeral ports
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2750
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Anupam Seth
> Fix For: 0.23.0
>
>
> There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
> The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2750) [MR-279] NodeManager should
start its servers on ephemeral ports
Posted by "Anupam Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117423#comment-13117423 ]
Anupam Seth commented on MAPREDUCE-2750:
----------------------------------------
Patch to fix this clubbed with MAPREDUCE-2986. Up for review.
> [MR-279] NodeManager should start its servers on ephemeral ports
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2750
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Anupam Seth
> Fix For: 0.23.0
>
>
> There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
> The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2750) [MR-279] NodeManager should
start its servers on ephemeral ports
Posted by "Vinod Kumar Vavilapalli (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13078762#comment-13078762 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2750:
----------------------------------------------------
bq. Is there a reason someone would want to run multiple NodeManagers on a single host in a production environment?
Two cases where I need this: (1) for benchmarks where I want to run many nodemanagers on a single node and (2) tests so that multiple NMs can be started in MiniYARNClusters.
bq. Hadoop has way too many ephemeral ports as it is....
Any down-sides to this? I know Steve mentioned firewall issues, anything else too?
bq. How about making it an option to start on an ephemeral port, as in production it will only complicate firewall setup
Thanks for that idea, that could work if we deem ephemeral ports as too limiting.
> [MR-279] NodeManager should start its servers on ephemeral ports
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2750
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
>
> There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
> The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2750) [MR-279] NodeManager should
start its servers on ephemeral ports
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072876#comment-13072876 ]
Allen Wittenauer commented on MAPREDUCE-2750:
---------------------------------------------
Is there a reason someone would want to run multiple NodeManagers on a single host in a production environment? Hadoop has way too many ephemeral ports as it is....
> [MR-279] NodeManager should start its servers on ephemeral ports
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-2750
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2750
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Vinod Kumar Vavilapalli
> Fix For: 0.23.0
>
>
> There is absolutely no need for NM to start ContainerManager on a standard port, it should bind to an ephemeral port. Binding on ephemeral ports will help us start multiple NMs on a single node without any issues.
> The same holds for the ResourceLocalizationService's server port and NM Http server's port.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira