[jira] [Updated] (IGNITE-17230) Support split-file page store

["Kirill Tkalenko (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-17230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kirill Tkalenko updated IGNITE-17230:
-------------------------------------
    Summary: Support split-file page store  (was: Support splt-file page store)

> Support split-file page store
> -----------------------------
>
>                 Key: IGNITE-17230
>                 URL: https://issues.apache.org/jira/browse/IGNITE-17230
>             Project: Ignite
>          Issue Type: Task
>            Reporter: Kirill Tkalenko
>            Assignee: Kirill Tkalenko
>            Priority: Major
>              Labels: ignite-3
>             Fix For: 3.0.0-alpha6
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> *Notes*
> Description may not be complete.
> *Goal*
> To implement a new checkpoint (described in IGNITE-15818), we will introduce a new entity *DeltaFilePageStore*, which will be created for each partition at each checkpoint and removed after merging with the *FilePageStore* (the main partition file) using the compacter.
> *DeltaFilePageStore* will consist of:
> * Header (maybe updated in the course of implementation):
> ** Allocation *pageIdx* - *pageIdx* of the last created page;
> * Sorted list of *pageIdx* - allows a binary search to find the file offset for an *pageId -> pageIdx*;
> * Page content - sorted by *pageIdx*.
> What will change for *FilePageStore*:
> * List of class *DeltaFilePageStore* will be added (from the newest to the oldest by the time of creation);
> * Allocation index (pageIdx of the last created page) - it will be logical and contained in the header of *FilePageStore*. At node start, it will be read from the header of *FilePageStore* or obtained from the first *DeltaFilePageStore* (the newest one).
> How pages will be read by *pageId -> pageIdx*:
> * Interrogates the class *DeltaFilePageStore* in order from the newest to the oldest;
> * If not found, then we read page from the *FilePageStore* itself.
> *Some implementation notes*
> * Format of the file name for the *DeltaFilePageStore*  is *part-%d-delta-%d.bin* for example *part-1-delta-3.bin* where the first digit is the partition identifier, and the second is the serial number of the delta file for this partition;
> * Before creating *part-1-delta-3.bin*, a temporary file *part-1-delta-3.bin.tmp* will be created at the checkpoint first, then filled, then renamed to *part-1-delta-3.bin*;
> * In each partition and delta file we will store the *org.apache.ignite.internal.storage.pagememory.io.PartitionMetaIo*, which will be the first page in this file, and it will be special;
> ** We will not store it in *PageMemory* (we will not let it be dirty) for optimization and will write it immediately to the delta file at the checkpoint and at the merge with the partition file by the compactor;
> ** Instead of the allocation index, we will store the logical size of the partition, which we will store in the *PartitionMetaIo*;
> * We will store the list of *pageIdx* in the header of the delta file.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)