[GitHub] [spark] squito commented on issue #24462: [SPARK-26268][CORE] Do not resubmit tasks when executors are lost

[GitBox <gi...@apache.org>]

squito commented on issue #24462: [SPARK-26268][CORE] Do not resubmit tasks when executors are lost
URL: https://github.com/apache/spark/pull/24462#issuecomment-487145924
 
 
   I understand the motivation here, but I think this should be handled as part of the new shuffle storage plugin mechanism under https://issues.apache.org/jira/browse/SPARK-25299.  We've actually been discussing the right way to expose this kind of knob to a plugin, if its necessary.  I don't think we've gotten around to writing up docs about this yet, sorry
   
   So while I don't think this we should make the change in this PR, I'd be really interested in hearing more about the system you are using for storing shuffle data.
   
   fyi @mccheah @yifeih 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org