[GitHub] [incubator-pegasus] hycdong edited a comment on issue #906: Bug: flush_all_family_columns failed because of Shutdown in progress

[GitBox <gi...@apache.org>]

hycdong edited a comment on issue #906:
URL: https://github.com/apache/incubator-pegasus/issues/906#issuecomment-1035763256


   # Root case
   In current Pegasus implementation, when close a replica, we will call following RocksDB functions in turn:
   ```
   // in replica::close() function
   rocksdb::CancelAllBackgroundWork(_db, true);
   // in pegasus_server::stop() function
   _db->Flush(...);
   ```
   Pegasus now use RocksDB 6.6.4, I have read the source code of RocksDB 6.6.4, and found out that Flush function will always be failed if it called after CancelAllBackgroundWork. 
   According to https://github.com/facebook/rocksdb/blob/v6.6.4/db/db_impl/db_impl.cc#L429, CancelAllBackgourndWork will `shutting_down_ = true`.
   Reference https://github.com/facebook/rocksdb/blob/v6.6.4/db/db_impl/db_impl_compaction_flush.cc#L1766-L1769, Flush will return `Status::ShutdownInProgress()` error if `shutting_down_ = true`.
   
   However, it doesn't mean flush is not triggered in current close process. Function `CancelAllBackgroundWork` will also call flush function if it notice that there are memtables need to be flushed.
   
   # How to fix it
   Adjust the function order in close process, call flush function before CancelAllBackgroundWork.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org