[jira] [Commented] (IGNITE-8017) Disable WAL during initial preloading

["Ilya Lantukh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409493#comment-16409493 ] 

Ilya Lantukh commented on IGNITE-8017:
--------------------------------------

More global approach

> Disable WAL during initial preloading
> -------------------------------------
>
>                 Key: IGNITE-8017
>                 URL: https://issues.apache.org/jira/browse/IGNITE-8017
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Ilya Lantukh
>            Assignee: Ilya Lantukh
>            Priority: Major
>
> While handling SupplyMessage, node handles each supplied data entry separately, which causes a WAL record for each entry to be written. It significantly limits preloading speed, especially with WALMode == FSYNC - it will perform fsync for every entry!
> We can improve rebalancing speed and reduce pressure on disk by disabling WAL until all data is loaded. The disadvantage of this approach is that data might get corrupted if node crashes - but node that crashed during preloading has to clear all it's data anyway. However, it is important to distinguish situations when new node joined cluster or added to baseline topology (and doesn't hold any data) and when additional partitions got assigned to node after baseline topology changed (in this case node has to keep all data in consistent state).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)