[jira] [Created] (SPARK-27220) Remove Yarn specific leftover from CoarseGrainedSchedulerBackend

["Jacek Lewandowski (JIRA)" <ji...@apache.org>]

Jacek Lewandowski created SPARK-27220:
-----------------------------------------

             Summary: Remove Yarn specific leftover from CoarseGrainedSchedulerBackend
                 Key: SPARK-27220
                 URL: https://issues.apache.org/jira/browse/SPARK-27220
             Project: Spark
          Issue Type: Task
          Components: Spark Core, YARN
    Affects Versions: 2.4.0, 2.3.3, 2.2.3, 2.1.3, 2.0.2
            Reporter: Jacek Lewandowski


{{CoarseGrainedSchedulerBackend}} has the following field:

{code:scala}
  // The num of current max ExecutorId used to re-register appMaster
  @volatile protected var currentExecutorIdCounter = 0
{code}

which is then updated:

{code:scala}
      case RegisterExecutor(executorId, executorRef, hostname, cores, logUrls) =>
...
          // This must be synchronized because variables mutated
          // in this block are read when requesting executors
          CoarseGrainedSchedulerBackend.this.synchronized {
            executorDataMap.put(executorId, data)
            if (currentExecutorIdCounter < executorId.toInt) {
              currentExecutorIdCounter = executorId.toInt
            }
...
{code}

However it is never really used in {{CoarseGrainedSchedulerBackend}}. Its only usage is in Yarn-specific code. It should be moved to Yarn then because {{executorId}} is a {{String}} and there are really no guarantees that it is always an integer.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org