[GitHub] [spark] ajithme commented on a change in pull request #24438: [SPARK-23626][CORE] DAGScheduler blocked due to JobSubmitted event

[GitBox <gi...@apache.org>]

ajithme commented on a change in pull request #24438: [SPARK-23626][CORE] DAGScheduler blocked due to JobSubmitted event
URL: https://github.com/apache/spark/pull/24438#discussion_r277928327
 
 

 ##########
 File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
 ##########
 @@ -1309,4 +1309,11 @@ package object config {
     .bytesConf(ByteUnit.BYTE)
     .createOptional
 
+  private[spark] val EAGER_RDD_PARTITION_EVALUATE =
+    ConfigBuilder("spark.rdd.eager.partition")
 
 Review comment:
   Thanks @srowen I did some testing and these seems to solve the problem but i was skeptic of the change so kept it configurable so that user(admin) can change back if this introduces unwanted behavior. I did not intend to make this configuration public. DAGScheduler is still a grey area for me, would like to know any feedback for this change. If it checks all the boxes, i am too +1 for not making this configurable.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org