[GitHub] [spark] tgravescs commented on issue #24208: [SPARK-27272][CORE] Enable blacklisting of node/executor on fetch failures by default

[GitBox <gi...@apache.org>]

tgravescs commented on issue #24208: [SPARK-27272][CORE] Enable blacklisting of node/executor on fetch failures by default
URL: https://github.com/apache/spark/pull/24208#issuecomment-476716287
 
 
   I guess I'm not sure I agree with tying the fetch failure count to the same config as MAX_FAILED_EXEC_PER_NODE.  I can see someone wanting that to stay the same but be more aggressive on the fetch failures. 
   
    I used to run with the config on all the time at Yahoo where it did it immediately, never saw any issues with it that people reported but that doesn't mean it wasn't to aggressive since generally we had very large clusters.  @abellina have you seen any issues with this being on?
   
   If we don't have any data with needing it perhaps we leave it as is now, the customers out is they just turn it back off and if people report it to aggressive then at that point we add a new config, what do you think? 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org