[jira] [Commented] (IGNITE-26281) Implement in-memory metadata storage for log index files

["Aleksandr Polovtsev (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-26281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18023789#comment-18023789 ] 

Aleksandr Polovtsev commented on IGNITE-26281:
----------------------------------------------

I didn't implement GroupMetadata as a queue in this ticket, because it's not needed at this stage of development. It's going to be converted to a queue in one of the log truncation-related tickets.

> Implement in-memory metadata storage for log index files
> --------------------------------------------------------
>
>                 Key: IGNITE-26281
>                 URL: https://issues.apache.org/jira/browse/IGNITE-26281
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Ivan Bessonov
>            Assignee: Aleksandr Polovtsev
>            Priority: Major
>              Labels: ignite-3
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> What is it: it's an in-memory representation of already persisted metadata for index files described here: https://issues.apache.org/jira/browse/IGNITE-26279.
> Schematically it should look like this:
> {code:java}
> InMemoryMetadata = Map<
>   ReplicationGroupId,
>   GroupMetadata
> >
> GroupMetadata {
>   Queue<BlockOfIndexMetas>
> }
> BlockOfIndexMetas = {
>   startRaftIndex
>   endRaftIndex
>   startIndexId
>   IndexMeta[]
> }
> IndexMeta = {
>   startRaftIndex
>   generation
>   offsetInIndexFile
> } {code}
> {{InMemoryMetadata}} will also be spread across individual log storages of individual raft groups for fast access.
> {{GroupMetadata}} represents a queue of index files, so to speak, from newest to oldest. It reflects the lookup order in the queue when we're executing the {{getEntry}} operation.
> {{IndexMeta[]}} groups together all index files for which the {{End}} of a previous meta matches {{Start}} of next meta. This means that a particularly large {{truncateSuffix}} might lead to adding a new {{BlockOfIndexMetas}} instance to the queue.
> In this Jira we should maintain this structure from the checkpoint thread. We should try to make it block-free, so that other threads can read from it while we're writing.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)