[GitHub] keith-turner commented on issue #823: Handle empty write ahead logs more gracefully.

[GitBox <gi...@apache.org>]

keith-turner commented on issue #823: Handle empty write ahead logs more gracefully.
URL: https://github.com/apache/accumulo/issues/823#issuecomment-448699403
 
 
   I did some research and found the tablet server code that is causing this.  Every time a write ahead log is created it is [immediately added to zookeeper](https://github.com/apache/accumulo/blob/rel/1.9.2/server/tserver/src/main/java/org/apache/accumulo/tserver/log/TabletServerLogger.java#L286).  When writing to a walog if any error occurs then the [WAL is closed](https://github.com/apache/accumulo/blob/rel/1.9.2/server/tserver/src/main/java/org/apache/accumulo/tserver/log/TabletServerLogger.java#L455).  Sometimes the exceptions that occur are unrelated to writing to the walog and occur before anything was ever written to the walog.  If the error keeps happening then this ends up adding lots of empty walogs to ZK which are used during recovery.
   
   One possible solution is that when a walog is closed if nothing was written to it then mark the walog in ZK in such a way that is GCed.  Could possibly immediately remove the file after marking it in ZK.  If the process dies between ZK and delete, then the Accumulo GC would still clean it up.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services