You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Patrick Wendell (JIRA)" <ji...@apache.org> on 2015/01/06 21:14:35 UTC
[jira] [Created] (SPARK-5113) Audit and document use of hostnames
and IP addresses in Spark
Patrick Wendell created SPARK-5113:
--------------------------------------
Summary: Audit and document use of hostnames and IP addresses in Spark
Key: SPARK-5113
URL: https://issues.apache.org/jira/browse/SPARK-5113
Project: Spark
Issue Type: Bug
Reporter: Patrick Wendell
Priority: Critical
Spark has multiple network components that start servers and advertise their network addresses to other processes.
We should go through each of these components and make sure they have consistent and/or documented behavior wrt (a) what interface(s) they bind to and (b) what hostname they use to advertise themselves to other processes. We should document this clearly and explain to people what to do in different cases (e.g. EC2, dockerized containers, etc).
When Spark initializes, it will search for a network interface until it finds one that is not a loopback address. Then it will do a reverse DNS lookup for a hostname associated with that interface. Then the network components will use that hostname to advertise the component to other processes. In some cases, that hostname is used as the bind interface also (e.g. I think this happens in the connection manager and possibly akka). In other cases (the web UI and netty shuffle) we seem to bind to all interfaces.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org