You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Philip Zeyliger (JIRA)" <ji...@apache.org> on 2011/03/18 00:44:29 UTC
[jira] Created: (HADOOP-7198) Hadoop defaults for web UI ports
often fall smack in the middle of Linux ephemeral port range
Hadoop defaults for web UI ports often fall smack in the middle of Linux ephemeral port range
---------------------------------------------------------------------------------------------
Key: HADOOP-7198
URL: https://issues.apache.org/jira/browse/HADOOP-7198
Project: Hadoop Common
Issue Type: Wish
Reporter: Philip Zeyliger
Priority: Trivial
It turns out (see http://en.wikipedia.org/wiki/Ephemeral_port and /proc/sys/net/ipv4/ip_local_port_range) that when you bind to port 0, Linux chooses an ephemeral port. On my default-ridden Ubuntu Maverick box and on CentOS 5.5, that range is 32768-61000. So, when HBase binds to 60030 or when mapReduce binds to 50070, there's a small chance that you'll conflict with, say, an FTP session, or with some other Hadoop daemon that's had a listening address configured as :0.
I don't know that there's a practical resolution here, since changing the defaults seems like an ill-fated effort, but if you have any ephemeral port use, you can run into this. We've now run into it once.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7198) Hadoop defaults for web UI ports
often fall smack in the middle of Linux ephemeral port range
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451374#comment-13451374 ]
Harsh J commented on HADOOP-7198:
---------------------------------
bq. Any port that is picked is going to conflict with something when talking above the 1k port line.
For Linux this may not hold true though, it seems to limit the ephemeral range from 32k onwards these days. But yeah, thats non-standard.
> Hadoop defaults for web UI ports often fall smack in the middle of Linux ephemeral port range
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-7198
> URL: https://issues.apache.org/jira/browse/HADOOP-7198
> Project: Hadoop Common
> Issue Type: Wish
> Reporter: Philip Zeyliger
> Priority: Trivial
>
> It turns out (see http://en.wikipedia.org/wiki/Ephemeral_port and /proc/sys/net/ipv4/ip_local_port_range) that when you bind to port 0, Linux chooses an ephemeral port. On my default-ridden Ubuntu Maverick box and on CentOS 5.5, that range is 32768-61000. So, when HBase binds to 60030 or when mapReduce binds to 50070, there's a small chance that you'll conflict with, say, an FTP session, or with some other Hadoop daemon that's had a listening address configured as :0.
> I don't know that there's a practical resolution here, since changing the defaults seems like an ill-fated effort, but if you have any ephemeral port use, you can run into this. We've now run into it once.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7198) Hadoop defaults for web UI ports
often fall smack in the middle of Linux ephemeral port range
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049900#comment-13049900 ]
Harsh J commented on HADOOP-7198:
---------------------------------
I guess we could make a documentation note at most for this.
Do you have any other ideas in mind?
> Hadoop defaults for web UI ports often fall smack in the middle of Linux ephemeral port range
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-7198
> URL: https://issues.apache.org/jira/browse/HADOOP-7198
> Project: Hadoop Common
> Issue Type: Wish
> Reporter: Philip Zeyliger
> Priority: Trivial
>
> It turns out (see http://en.wikipedia.org/wiki/Ephemeral_port and /proc/sys/net/ipv4/ip_local_port_range) that when you bind to port 0, Linux chooses an ephemeral port. On my default-ridden Ubuntu Maverick box and on CentOS 5.5, that range is 32768-61000. So, when HBase binds to 60030 or when mapReduce binds to 50070, there's a small chance that you'll conflict with, say, an FTP session, or with some other Hadoop daemon that's had a listening address configured as :0.
> I don't know that there's a practical resolution here, since changing the defaults seems like an ill-fated effort, but if you have any ephemeral port use, you can run into this. We've now run into it once.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7198) Hadoop defaults for web UI ports
often fall smack in the middle of Linux ephemeral port range
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008226#comment-13008226 ]
Allen Wittenauer commented on HADOOP-7198:
------------------------------------------
We often run into this problem when the local named caching daemon fires up first. The best solution is to pretty much make sure that Hadoop comes up first or configure the other service to use a limited set of ports.
Any port that is picked is going to conflict with *something* when talking above the 1k port line.
> Hadoop defaults for web UI ports often fall smack in the middle of Linux ephemeral port range
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-7198
> URL: https://issues.apache.org/jira/browse/HADOOP-7198
> Project: Hadoop Common
> Issue Type: Wish
> Reporter: Philip Zeyliger
> Priority: Trivial
>
> It turns out (see http://en.wikipedia.org/wiki/Ephemeral_port and /proc/sys/net/ipv4/ip_local_port_range) that when you bind to port 0, Linux chooses an ephemeral port. On my default-ridden Ubuntu Maverick box and on CentOS 5.5, that range is 32768-61000. So, when HBase binds to 60030 or when mapReduce binds to 50070, there's a small chance that you'll conflict with, say, an FTP session, or with some other Hadoop daemon that's had a listening address configured as :0.
> I don't know that there's a practical resolution here, since changing the defaults seems like an ill-fated effort, but if you have any ephemeral port use, you can run into this. We've now run into it once.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira