[jira] [Created] (IGNITE-5821) Implement fuzzy checkpoints

["Alexey Goncharuk (JIRA)" <ji...@apache.org>]

Alexey Goncharuk created IGNITE-5821:
----------------------------------------

             Summary: Implement fuzzy checkpoints
                 Key: IGNITE-5821
                 URL: https://issues.apache.org/jira/browse/IGNITE-5821
             Project: Ignite
          Issue Type: New Feature
          Components: persistence
    Affects Versions: 2.1
            Reporter: Alexey Goncharuk
             Fix For: 2.2


Currently, we are able to run only sharp checkpoints (all committed transactions are in the checkpoint, all non-committed are not included, all data structures are fully consistent).
This has the following disadvantages:

1) All transactions are blocked for the markCheckpointBegin call
2) We have an additional overhead for checkpoint COW buffer
3) If checkpoint buffer is exhausted, we block all transactions and synchronously wait for the checkpoint to be finished.

There is a technique called fuzzy checkpoints:
1) We keep a WAL LSN in every dirty page
2) When a page is being flushed to disk, we sync WAL up to the LSN
3) We maintain checkpoint low watermark so that WAL does not grow indefinitely
4) WAL logging is changed in a way that does not allow data structures updates to be mixed in WAL
5) The recovery procedure is changed to apply all physical deltas up to the end of WAL and have consistent memory state, then logical records revert all non-committed transactions



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)