[GitHub] [spark] maryannxue commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location

[GitBox <gi...@apache.org>]

maryannxue commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location
URL: https://github.com/apache/spark/pull/26633#issuecomment-561345400
 
 
   @tgravescs Our benchmark comparing AQE w/ LSR (local shuffle reader) with AQE w/o LSR showed that before locality wait fix, there were 2 queries with over 10% regressions, and after the fix, there was no regression and one query had over 27% improvement.
   
   @attilapiros Thank you again for the careful review! I changed a code in a way that `WildcardLocation` is only recognized and handled by DAGScheduler and TasksetManager internally and calling `TaskLocation.apply` will not instantiate a `WildcardLocation`, so the new location will have no effect or impact over all other usages of `TaskLocation`. I also refined `addPendingTask` so that the code change is minimized.
   
   I also added javadoc stating the limited application and the experimental nature of the `WildcardLocation` class.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org