You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Filipp Shergalis (Jira)" <ji...@apache.org> on 2025/11/24 11:07:00 UTC

[jira] [Created] (IGNITE-27147) Optimise compaction of multiple delta files

Filipp Shergalis created IGNITE-27147:
-----------------------------------------

             Summary: Optimise compaction of multiple delta files
                 Key: IGNITE-27147
                 URL: https://issues.apache.org/jira/browse/IGNITE-27147
             Project: Ignite
          Issue Type: Improvement
            Reporter: Filipp Shergalis
            Assignee: Filipp Shergalis
             Fix For: 3.2


We process delta files from the oldest to the newest, and pages can be written to main file multiple times.

For each delta file we already have in memory sorted  arrays with page indexes. We can use "two pointer technique" to skip duplicated pages in linear time (n - pages in all other delta files).

In the beginning of partition compaction create an int array containing indexes for each delta file. Processing each page iterate over all other delta files until we get page >= current one and skip the page if found. Next page continues from the same index, so it's efficient



--
This message was sent by Atlassian Jira
(v8.20.10#820010)