[GitHub] [accumulo] alerman opened a new issue #1456: Prevent catastrophic tserver shutdown by restricting the status shutdown

[GitBox <gi...@apache.org>]

alerman opened a new issue #1456: Prevent catastrophic tserver shutdown by restricting the status shutdown
URL: https://github.com/apache/accumulo/issues/1456
 
 
   We had a situation where the master had a bad hosts file for a 2 minute period. During that time, the master marked ~70% of the tservers as bad. When the good hosts file was then put in place, the master issued shutdowns to that 70% due to the master having failed the max number of status requests.
   
   There should be a threshold percentage of tservers where the master refuses to issue that shutdown. If the set of tservers to be shutdown is greater than or equal to 10% as a default, do not issue the shutdowns. Clearly log an error level message so that users can monitor for this situation, but don't self destruct.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services