[jira] [Created] (HBASE-8412) consider a notion of set of storefiles (for atomic region file set changes, for (or reused from) snapshots

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

Sergey Shelukhin created HBASE-8412:
---------------------------------------

             Summary: consider a notion of set of storefiles (for atomic region file set changes, for (or reused from) snapshots
                 Key: HBASE-8412
                 URL: https://issues.apache.org/jira/browse/HBASE-8412
             Project: HBase
          Issue Type: Brainstorming
            Reporter: Sergey Shelukhin


HBASE-2231 proposes adding compaction events to WAL for I/O fencing.
HBASE-7329 removed flush entries from WAL and there's a concern that entries written with WAL off will not be fenced out and can be flushed into the region that is already opened on a different server.
There are references, HFileLinks etc. 
Snapshots use sets of files from regions, and use some form of ref counting on the files.

There's a common pattern here of having a logical set of StoreFiles, obtained atomically.
Perhaps we should have a notion of a set of store files, sort of a manifest. It would probably be stored on FS by default. 
We can make it part of every store, and update it atomically after compactions, flushes and such (master will probably also have to fence it, same as WAL).
Snapshots, splits, etc. might also use it. For that, it would need refcounting. I am not familiar with snapshot refcounting, but I hear it involves links and backlinks... we can reuse it or we can add new one.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira