You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Pavel Pereslegin (Jira)" <ji...@apache.org> on 2025/09/01 12:10:00 UTC

[jira] [Assigned] (IGNITE-26067) Add modification counter to partition

     [ https://issues.apache.org/jira/browse/IGNITE-26067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pavel Pereslegin reassigned IGNITE-26067:
-----------------------------------------

    Assignee: Pavel Pereslegin

> Add modification counter to partition
> -------------------------------------
>
>                 Key: IGNITE-26067
>                 URL: https://issues.apache.org/jira/browse/IGNITE-26067
>             Project: Ignite
>          Issue Type: Improvement
>          Components: tables ai3
>            Reporter: Konstantin Orlov
>            Assignee: Pavel Pereslegin
>            Priority: Major
>              Labels: ignite-3
>
> To implement staleness check for query plan auto-reoptimization, it's proposed to introduce modification counter on per-partition basis. We don’t really need a fair modification counter, but rather a timestamp when we detected staleness for the last time. Hence, for every partition we need to maintain a triplet of volatile attributes *{{(modificationCounter, nextMilestone, lastMilestoneReachedTs)}}*. This triplet should be initialized at the start of the partition with values *{{(0, computeNextMilestone(currentPartitionSize), HybridClock.now())}}*. To implement *{{computeNextMilestone}}*, we need to introduce 2 new configuration parameters:
> {code:java}
> sql.statistics.staleRowsFraction = 0.2 (default) // fraction of table required to be modified to trigger statistics recollection, i.e. currentSize * staleRowsFraction to compute actual number of rows required
> sql.statistics.minStaleRowsCount = 500 (default) // minimal number of updates required to trigger statistics recollection
>  {code}
> These are cluster wide parameters, later we will introduce overrides on a per-table basis.
> A *{{modificationCounter}}* should be incremented by 1 per any RowId from update command, i.e. it should be incremented by 1 for *{{UpdateCommand}}*, and by *{{size(batch)}}* for *{{UpdateAllCommand}}*.
> When *{{modificationCounter}}* exceeds *{{nextMilestone}}*, a new milestone should be computed, and *{{lastMilestoneReachedTs}}* is set with *{{commitTs}}*.
> The counters should be exposed as metrics



--
This message was sent by Atlassian Jira
(v8.20.10#820010)