[jira] [Created] (SPARK-23283) Probable mistake in hasLaunchedTask condition

["Jakub Dubovsky (JIRA)" <ji...@apache.org>]

Jakub Dubovsky created SPARK-23283:
--------------------------------------

             Summary: Probable mistake in hasLaunchedTask condition
                 Key: SPARK-23283
                 URL: https://issues.apache.org/jira/browse/SPARK-23283
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.2.1
            Reporter: Jakub Dubovsky


In class {code}TaskSchedulerImpl{code} there is a [variable|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L97] tracking whether some task was already launched. It is then set to true [here|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L377] whenever some task is launched.

I believe the [condition|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala#L376] should be {code}tasks.map(_.size).sum > 0{code} instead of current {code}tasks.size > 0{code}

Current condition asks whether there are some resource offers while it should ask whether there were some launched tasks. In some cases it might happen that resources are offered but no tasks launched. In such a case no warning is issued.

Can you confirm this is valid?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org