You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/03/05 10:22:38 UTC

[jira] [Resolved] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

     [ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-6135.
------------------------------
    Resolution: Not a Problem

We use pull requests rather than patches to propose a change. That said, this is definitely not a valid change. This is not an exhaustive list of TLDs, nor is it true that host names in this context even have a TLD.

> No checks of illegal hostname when runing spark on yarn.
> --------------------------------------------------------
>
>                 Key: SPARK-6135
>                 URL: https://issues.apache.org/jira/browse/SPARK-6135
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.2.1
>         Environment: spark on yarn cluster. 
>            Reporter: Xia Hu
>              Labels: spark-submit, yarn-client
>         Attachments: check_hostname.patch
>
>
> I submit spark application to yarn cluster from a spark client machine, then I find this problem. When using spark on yarn-client model, driver is running in client, then the yarn cluster need to remotely connect to driver. But spark use InetAddress to read out the hostname of client machine,without checking if the hostname is legal or useful. So in my condition, some client machine have hostnames like "sjs_1_2", then this application fail because of cannot connect to driver on "sjs_1_2".
> I suppose there should be a check for if a hostname is legal, and if not, using the IP instead. 
> And for this problem, I found an env. configuration  "SPARK_LOCAL_HOSTNAME" can be used. if I set "SPARK_LOCAL_HOSTNAME" to be  IP address in spark-env.sh, then this problem is solved. But it seems this configuration isn't introduced in any introductions or references  and I found it when reading codes. 
> But I still think a check of if hostname is illegal is needed, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org