[GitHub] merlimat opened a new issue #753: Allow option to disable data sync on journal

[GitBox <gi...@apache.org>]

merlimat opened a new issue #753: Allow option to disable data sync on journal
URL: https://github.com/apache/bookkeeper/issues/753
 
 
   For deployments where a RAID battery-backed cache is not available and where durability is not a requirements, we should allow bookies to rely on page cache and relax durability.
   
   This change adds a system-wide options. All bookies are expected to have the same setting. 
   With relaxed durability, a user should expect that some amount of data can be loss in case of power failures or kernel panics. 
   
   This change is kind of similar to the proposal for BP-14, though not in contrast with it. In BP-14, there is a fine grained approach to durability with the option to flush the ledger. Here we're adding the option to turn it off cluster-wide.
   
   The rational around this setting is that: 
    * Fsync performance is thighly related to the hardware choice. BookKeeper can perform very well with strong durability on commodity hardware, provided the hardware setup is good. 
    * Typically, choosing the right hardware boils down to either:
       * Use a RAID controller with battery-backed write cache (this is very common in most RAID controllers). In this case the write cache is able to group together multiple IOps and radically increase the throughput and lower the latency, while maintaining the same degree of durability (in case of power loss, the write cache gets flushed on disk using the battery power). 
       * Use a "fast" SSD for bookie journal. There's a huge variance on SSDs performance. Some guy from Ceph community put together a list of drives with perf numbers (http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/). Even for "enterprise" models, performance on fsync heavy loads is all over the place.
    * In some cases, a user might not have the luxury of choosing the right hardware setup. This is typically the case in cloud environments where the hardware options are fixed.
    * If a user doesn't require strong durability for its use case, and can afford to lose data in some scenario, we should allow him to configure the bookies in this way.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services