[GitHub] athanatos commented on issue #1949: [bookie-gc] add option for persistent entry-log metadata map

[GitBox <gi...@apache.org>]

athanatos commented on issue #1949: [bookie-gc] add option for persistent entry-log metadata map
URL: https://github.com/apache/bookkeeper/pull/1949#issuecomment-465381437
 
 
   I'd like to suggest an alternate approach.  We already in fact have the ledger map for each entry log compiled at the end of each file.  Suppose instead of adding rocksdb, you use some bounded auto-loading cache (evict newest entry rather than oldest to avoid thrashing the cache during scan) which falls back to simply re-reading the index for the given EntryLog?  It would smoothly transition between fully in-memory and spilling to disk behaviors as needed.  By introducing rocksdb, you introduce a new way for bookie state to be corrupted -- inconsistency between entry log and cached ledger map.  You also introduce rocksdb itself as a source of overhead -- it's got (among other things) its own writer thread and journal.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services