You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:04:47 UTC
[jira] [Updated] (SPARK-22906) External shuffle IP different from
Host ip
[ https://issues.apache.org/jira/browse/SPARK-22906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-22906:
---------------------------------
Labels: bulk-closed (was: )
> External shuffle IP different from Host ip
> ------------------------------------------
>
> Key: SPARK-22906
> URL: https://issues.apache.org/jira/browse/SPARK-22906
> Project: Spark
> Issue Type: Improvement
> Components: Block Manager
> Affects Versions: 2.2.1
> Reporter: Unai Sarasola
> Priority: Minor
> Labels: bulk-closed
>
> Now is possible to configure the spark.shuffle.service.port, but there aren't an equivalent for host. Imagine that you are using an external shuffle deployed in Docker.
> If you aren't using Host mode for the docker, you may want to use the internal ip address of the docker to connect to the external shuffle service.
> Also you could use Calico, or just being attached the spark shuffle to a different IP that is used in the Spark executor (example hosts with multiple network interfaces).
> So this is why I implemented the spark.shuffle.service.host configuration
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org