[GitHub] printstacktrace commented on issue #23402: [SPARK-26500][CORE] Add conf to support ingoring data locality

[GitBox <gi...@apache.org>]

printstacktrace commented on issue #23402: [SPARK-26500][CORE] Add conf to support ingoring data locality
URL: https://github.com/apache/spark/pull/23402#issuecomment-450477662
 
 
   > I don't understand this propose add a configuration used for disable data locality.There are exists a configuration `spark.locality.wait` is charge of the function.You can set zero for `spark.locality.wait`.
   
   When submit a job, the driver will go into DAGScheduler.handleJobSubmitted, it will call **getPreferredLocs** for each partition of a rdd.  It's single thread and time-consuming when there are huge partitions, and this code path is not controlled by spark.locality.wait.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org