[GitHub] ivankelly opened a new pull request #913: [WIP] Refactor FileInfo locking and refcounting out IndexPersistenceMgr

[GitBox <gi...@apache.org>]

ivankelly opened a new pull request #913: [WIP] Refactor FileInfo locking and refcounting out IndexPersistenceMgr
URL: https://github.com/apache/bookkeeper/pull/913
 
 
   There is a number of bugs in index persistence mgr related to how
   refcounts are handled and how FileInfos are freed, due to the locking
   mechanism. These can result in the fencing bit being lost in ledgers,
   which is a serious issue. They also cause flakes in
   IndexPersistenceMgrTest#testEvictFileInfoWhenUnderlyingFileExists.
   
   For details see the discussion at:
   https://github.com/apache/bookkeeper/pull/513/files/8075b0#r156676238
   
   There are two key problems.
   
   1. FileInfos are flushed asynchronously on eviction. If another thread
   tries to read the FileInfo after the flush has been scheduled, but
   before it runs, it will read stale date (and we'll possibly lose the
   fence bit).
   2. FileInfos can be closed while it is still references in the
   IndexPersistenceMgr. This means that it can be fenced, but this will
   very be flushed to disk because the FileInfo is already closed.
   
   The patch solves this by moving FileInfo locking and refcount into a
   separate class, FileInfoBackingCache. When either the write cache or
   read cache load a file info, it tries to load it from this backing
   cache. All writes to the the backing cache are done under a write
   lock. This class also takes care of reference counting. It hands out
   CachedFileInfo objects, which will be flushed to disk when all
   references are released.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services