[GitHub] [incubator-celeborn] FMX commented on issue #980: [REFACTOR] Auto format RaftStorage when cluster upgrades

[GitBox <gi...@apache.org>]

FMX commented on issue #980:
URL: https://github.com/apache/incubator-celeborn/issues/980#issuecomment-1318788417

   Hi @SzyWilliam Glad to see your interest.
   I'll provide some details about the problems that we encountered. We recommended our users delete the raft meta storage because that we found the Celeborn master will load stats which was generated by last running. We don't want to format the raft storage whenever a master is restarting because the consensus will be lost.
   
   We are facing two major problems in using ratis. We already enable the snapshot mechanism to reduce raft storage size.
   1. We found that if we restart the master cluster one by one and the consensus stats will be broken. 
   2. We found that if we upgrade a Celeborn master with the old raft storage, the server will not start normally.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@celeborn.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org