You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Susheel Kumar Gadalay <sk...@gmail.com> on 2014/08/10 16:24:57 UTC

How to restrict ephemral ports used by Yarn App Master

Hi,

I have a question.

How do I selectively open port range for Hadoop Yarn App Master on a cluster.

I have seen the jira issue in
http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-issues/201204.mbox/%3C74835698.75.1335357881103.JavaMail.tomcat@hel.zones.apache.org%3E
fixed in version 0.23.2 and accordingly downloaded the latest version
2.4.1.

I have added the following property in mapred-site.xml
        <property>
                <name>yarn.app.mapreduce.am.job.client.port-range</name>
                <value>50100-50200</value>
        </property>

and accordingly opened the port range 50100-50200 in iptables.

After submitting the hadoop job it is using the port 50100 but it is
also using one ephemeral port for IPC server.
I have noticed different ephemeral ports used for IPC server for each
job submission.

Pl let me know what property can be set to restrict this random port range.

Thanks
Susheel Kumar

Re: How to restrict ephemral ports used by Yarn App Master

Posted by David <da...@gmail.com>.
Susheel Kumar Gadalay <sk...@...> writes:

> 
> Hi,
> 
> I have a question.
> 
> How do I selectively open port range for Hadoop Yarn App Master on a cluster.
> 
> I have seen the jira issue in
>
http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-issues/201204.mbox/%3C74835698.75.1335357881103.JavaMail.tomcat-RumystHAhCSyP0Er04PKA2D2FQJk+8+b
<at> public.gmane.org%3E
> fixed in version 0.23.2 and accordingly downloaded the latest version
> 2.4.1.
> 
> I have added the following property in mapred-site.xml
>         <property>
>                 <name>yarn.app.mapreduce.am.job.client.port-range</name>
>                 <value>50100-50200</value>
>         </property>
> 
> and accordingly opened the port range 50100-50200 in iptables.
> 
> After submitting the hadoop job it is using the port 50100 but it is
> also using one ephemeral port for IPC server.
> I have noticed different ephemeral ports used for IPC server for each
> job submission.
> 
> Pl let me know what property can be set to restrict this random port range.
> 
> Thanks
> Susheel Kumar
> 
> 


Hello,

I have the same kind of problems.
I added theses values into yarn-site.xml also the error is not exactly the
same when adding this property in this file but the map/reduce job is
failing because the connection is done on other ports (not in the range) and
closed by a firewall rule.
I think there is an issue on the map/reduce hadoop truck 2.4 or 2.4.1

Let me know if you got more information on this issue.

Regards,
David.