[jira] [Created] (IGNITE-18721) Improve WAL duing index rebuild

["Ilya Shishkov (Jira)" <ji...@apache.org>]

Ilya Shishkov created IGNITE-18721:
--------------------------------------

             Summary: Improve WAL duing index rebuild 
                 Key: IGNITE-18721
                 URL: https://issues.apache.org/jira/browse/IGNITE-18721
             Project: Ignite
          Issue Type: Improvement
            Reporter: Ilya Shishkov
         Attachments: WalDuringIndexRebuildTest.patch

Currently, Ignite writes WAL Segments during index rebuilding. For large index files this process can last for a several hours. It seems, that we could optimize or even turn off WAL segments writing during index rebuild process in order to minimize index rebuilding time.

There are 2 possible ways now to force index rebuilding:
# Force index rebuild by a {{control.sh}}.
# Manually delete index.bin files.

Both rebuilding processes generates different WALRecords in different proportions, as you can see in a tables below (results derived from [^WalDuringIndexRebuildTest.patch]).

*Summary for index rebuild task:*
||Record type                    ||Cnt||
|BTREE_PAGE_REPLACE             |5045341|
|PAGE_RECORD                    |389204|
|CHECKPOINT_RECORD              |4|                             
|INIT_NEW_PAGE_RECORD           |1|                             
|DATA_PAGE_SET_FREE_LIST_PAGE   |2|                             
|PAGES_LIST_ADD_PAGE            |1|                             
|METASTORE_DATA_RECORD          |1|

*Summary for index.bin remove:*
||Record type                      ||Count||
|PAGE_RECORD                        |386487|
|BTREE_PAGE_INSERT                  |5075253| 
|BTREE_FIX_COUNT                    |20216|                                                  
|BTREE_EXISTING_PAGE_SPLIT          |351907|
|CHECKPOINT_RECORD                  |5|                             
|INIT_NEW_PAGE_RECORD               |133|                           
|DATA_PAGE_INSERT_FRAGMENT_RECORD   |2|                             
|DATA_PAGE_SET_FREE_LIST_PAGE       |3|                             
|BTREE_META_PAGE_ADD_ROOT           |103|                           
|BTREE_INIT_NEW_ROOT                |154|                           
|BTREE_PAGE_RECYCLE                 |2|           
|PAGES_LIST_ADD_PAGE                |4|                             
|PAGES_LIST_REMOVE_PAGE             |1|                             
|METASTORE_DATA_RECORD              |3|                             
|BTREE_META_PAGE_INIT_ROOT_V3       |52|   



--
This message was sent by Atlassian Jira
(v8.20.10#820010)