[jira] [Resolved] (SPARK-10149) Locality Level is ANY on "Details for Stage" WebUI page

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-10149.
----------------------------------
    Resolution: Incomplete

> Locality Level is ANY on "Details for Stage" WebUI page
> -------------------------------------------------------
>
>                 Key: SPARK-10149
>                 URL: https://issues.apache.org/jira/browse/SPARK-10149
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core, Web UI
>    Affects Versions: 1.4.1
>            Reporter: Yun Zhao
>            Priority: Major
>              Labels: bulk-closed
>
> Locality Level is ANY on "Details for Stage" WebUI page
> When a sc.textFile(XX) program is running, Locality Level is ANY which should be NODE_LOCAL on stage 0. 
> org.apache.spark.scheduler.TaskSetManager
> {quote}
>       // Check for node-local tasks
>       if (TaskLocality.isAllowed(locality, TaskLocality.NODE_LOCAL)) {
>         for (index <- speculatableTasks if canRunOnHost(index)) {
>           val locations = tasks(index).preferredLocations.map(_.host)
>           if (locations.contains(host)) {
>             speculatableTasks -= index
>             return Some((index, TaskLocality.NODE_LOCAL))
>           }
>         }
>       }
> {quote}	  
> The variable "locations" is hostname of HDFS split, which is from InetAddress.getHostName.
> The variable "host" is ip of Executor, which is from InetAddress.getLocalHost.getHostAddress.
> org.apache.spark.deploy.worker.WorkerArguments
> {quote}
> var host = Utils.localHostName()
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org