You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2015/04/01 13:01:05 UTC
[jira] [Resolved] (SPARK-4799) Spark should not rely on local host
being resolvable on every node
[ https://issues.apache.org/jira/browse/SPARK-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-4799.
------------------------------
Resolution: Duplicate
Target Version/s: (was: 1.2.1)
Looks like this was subsumed by SPARK-5078 and SPARK_LOCAL_HOSTNAME
> Spark should not rely on local host being resolvable on every node
> ------------------------------------------------------------------
>
> Key: SPARK-4799
> URL: https://issues.apache.org/jira/browse/SPARK-4799
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.1.0
> Environment: Tested a Spark+Mesos cluster on top of Docker to reproduce the issue.
> Reporter: Santiago M. Mola
>
> Spark fails when a node hostname is not resolvable by other nodes.
> See an example trace:
> {code}
> 14/12/09 17:02:41 ERROR SendingConnection: Error connecting to 27e434cf36ac:35093
> java.nio.channels.UnresolvedAddressException
> at sun.nio.ch.Net.checkAddress(Net.java:127)
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:644)
> at org.apache.spark.network.SendingConnection.connect(Connection.scala:299)
> at org.apache.spark.network.ConnectionManager.run(ConnectionManager.scala:278)
> at org.apache.spark.network.ConnectionManager$$anon$4.run(ConnectionManager.scala:139)
> {code}
> The relevant code is here:
> https://github.com/apache/spark/blob/bcb5cdad614d4fce43725dfec3ce88172d2f8c11/core/src/main/scala/org/apache/spark/network/nio/ConnectionManager.scala#L170
> {code}
> val id = new ConnectionManagerId(Utils.localHostName, serverChannel.socket.getLocalPort)
> {code}
> This piece of code should use the host IP with Utils.localIpAddress or a method that acknowleges user settings (e.g. SPARK_LOCAL_IP). Since I cannot think about a use case for using hostname here, I'm creating a PR with the former solution, but if you think the later is better, I'm willing to create a new PR with a more elaborate fix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org