[jira] [Commented] (SPARK-1361) DAGScheduler throws NullPointerException occasionally

["wangfei (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955046#comment-13955046 ] 

wangfei commented on SPARK-1361:
--------------------------------

this is an order issue of construction and starting of the taskScheduler and the dagScheduler in SparkContext.  now taskScheduler start before  dagScheduler, so executor backend may register executor even before dagScheduler(or the eventProcessActor) has not initialized. in this situation it will throw NPE

> DAGScheduler throws NullPointerException occasionally
> -----------------------------------------------------
>
>                 Key: SPARK-1361
>                 URL: https://issues.apache.org/jira/browse/SPARK-1361
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 0.9.0
>            Reporter: wangfei
>             Fix For: 1.0.0
>
>
> DAGScheduler throws this NullPointerException below occasionally when running spark apps.
> java.lang.NullPointerException
> at org.apache.spark.scheduler.DAGScheduler.executorAdded(DAGScheduler.scala:186)
> at org.apache.spark.scheduler.TaskSchedulerImpl.executorAdded(TaskSchedulerImpl.scala:409)
> at org.apache.spark.scheduler.TaskSchedulerImpl$$anonfun$resourceOffers$1.apply(TaskSchedulerImpl.scala:210)
> at org.apache.spark.scheduler.TaskSchedulerImpl$$anonfun$resourceOffers$1.apply(TaskSchedulerImpl.scala:206)
> at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:73)
> at org.apache.spark.scheduler.TaskSchedulerImpl.resourceOffers(TaskSchedulerImpl.scala:206)
> at org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverActor.makeOffers(CoarseGrainedSchedulerBackend.scala:130)
> at org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverActor$$anonfun$receive$1.applyOrEls



--
This message was sent by Atlassian JIRA
(v6.2#6252)