Re: How to restrict ephemral ports used by Yarn App Master

[David <da...@gmail.com>]

Susheel Kumar Gadalay <sk...@...> writes:

> 
> Hi,
> 
> I have a question.
> 
> How do I selectively open port range for Hadoop Yarn App Master on a cluster.
> 
> I have seen the jira issue in
>
http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-issues/201204.mbox/%3C74835698.75.1335357881103.JavaMail.tomcat-RumystHAhCSyP0Er04PKA2D2FQJk+8+b
<at> public.gmane.org%3E
> fixed in version 0.23.2 and accordingly downloaded the latest version
> 2.4.1.
> 
> I have added the following property in mapred-site.xml
>         <property>
>                 <name>yarn.app.mapreduce.am.job.client.port-range</name>
>                 <value>50100-50200</value>
>         </property>
> 
> and accordingly opened the port range 50100-50200 in iptables.
> 
> After submitting the hadoop job it is using the port 50100 but it is
> also using one ephemeral port for IPC server.
> I have noticed different ephemeral ports used for IPC server for each
> job submission.
> 
> Pl let me know what property can be set to restrict this random port range.
> 
> Thanks
> Susheel Kumar
> 
> 


Hello,

I have the same kind of problems.
I added theses values into yarn-site.xml also the error is not exactly the
same when adding this property in this file but the map/reduce job is
failing because the connection is done on other ports (not in the range) and
closed by a firewall rule.
I think there is an issue on the map/reduce hadoop truck 2.4 or 2.4.1

Let me know if you got more information on this issue.

Regards,
David.