[jira] [Issue Comment Deleted] (SPARK-6135) No checks of illegal hostname when runing spark on yarn.

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen updated SPARK-6135:
-----------------------------
    Comment: was deleted

(was: Sorry I didn't make it clearly. 
SPARK_LOCAL_HOSTNAME can solve this problem, so I think it need to be documented in official introduction.
And for my problem, what I want to say is in some companies, people may use ip address more than hostname, so a machine's hostname(especially for vm machines) may be like this "sjs_1_2". I said this hostname is invalid because this hostname cannot be used to remotely connect(eg. "ping sjs_1_2" ). But spark get this hostname and use it. So we get a error like this "Failed to connect to driver at sjs_1_2:xxxx". So I think a check of hostname is needed, and a valid hostname may end up with "org","com","edu",etc. )

> No checks of illegal hostname when runing spark on yarn.
> --------------------------------------------------------
>
>                 Key: SPARK-6135
>                 URL: https://issues.apache.org/jira/browse/SPARK-6135
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.2.1
>         Environment: spark on yarn cluster. 
>            Reporter: Xia Hu
>              Labels: spark-submit, yarn-client
>         Attachments: check_hostname.patch
>
>
> I submit spark application to yarn cluster from a spark client machine, then I find this problem. When using spark on yarn-client model, driver is running in client, then the yarn cluster need to remotely connect to driver. But spark use InetAddress to read out the hostname of client machine,without checking if the hostname is legal or useful. So in my condition, some client machine have hostnames like "sjs_1_2", then this application fail because of cannot connect to driver on "sjs_1_2".
> I suppose there should be a check for if a hostname is legal, and if not, using the IP instead. 
> And for this problem, I found an env. configuration  "SPARK_LOCAL_HOSTNAME" can be used. if I set "SPARK_LOCAL_HOSTNAME" to be  IP address in spark-env.sh, then this problem is solved. But it seems this configuration isn't introduced in any introductions or references  and I found it when reading codes. 
> But I still think a check of if hostname is illegal is needed, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org