[jira] [Created] (IOTDB-1003) Split WAL buffer to better absorb ingestion

["Tian Jiang (Jira)" <ji...@apache.org>]

Tian Jiang created IOTDB-1003:
---------------------------------

             Summary: Split WAL buffer to better absorb ingestion
                 Key: IOTDB-1003
                 URL: https://issues.apache.org/jira/browse/IOTDB-1003
             Project: Apache IoTDB
          Issue Type: Improvement
          Components: Core/Engine
            Reporter: Tian Jiang


In the current implementation, we provide only one WAL buffer for each MemTable, which means if the buffer is full, we will have to wait until the buffer is flushed before we can write the next log, thus creating spikes in ingestions.

So, it is benefitial to split the WAL buffer into a working one and a flusing one, and if last flush has completed before we ran out of the current working one, we can directly swap them, and continue writes.

There is no point to split the buffer into more than 2 buffers, as the total capacity of the WALs are strictly controlled to bound data loss.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)