[jira] [Created] (SPARK-10149) Locality Level is ANY on "Details for Stage" WebUI page

["Yun Zhao (JIRA)" <ji...@apache.org>]

Yun Zhao created SPARK-10149:
--------------------------------

             Summary: Locality Level is ANY on "Details for Stage" WebUI page
                 Key: SPARK-10149
                 URL: https://issues.apache.org/jira/browse/SPARK-10149
             Project: Spark
          Issue Type: Bug
          Components: Spark Core, Web UI
    Affects Versions: 1.4.1
            Reporter: Yun Zhao


Locality Level is ANY on "Details for Stage" WebUI page


When a sc.textFile(XX) program is running, Locality Level is ANY which should be NODE_LOCAL on stage 0. 


org.apache.spark.scheduler.TaskSetManager

      // Check for node-local tasks
      if (TaskLocality.isAllowed(locality, TaskLocality.NODE_LOCAL)) {
        for (index <- speculatableTasks if canRunOnHost(index)) {
          val locations = tasks(index).preferredLocations.map(_.host)
          if (locations.contains(host)) {
            speculatableTasks -= index
            return Some((index, TaskLocality.NODE_LOCAL))
          }
        }
      }
	  

The variable "locations" is hostname of HDFS split, which is from InetAddress.getHostName.
The variable "host" is ip of Executor, which is from InetAddress.getLocalHost.getHostAddress.


org.apache.spark.deploy.worker.WorkerArguments
var host = Utils.localHostName()



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org