[jira] [Comment Edited] (IGNITE-17613) Create incremental snapshot

["Maksim Timonin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-17613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17607736#comment-17607736 ] 

Maksim Timonin edited comment on IGNITE-17613 at 9/21/22 1:08 PM:
------------------------------------------------------------------

There is a draft PR: [https://github.com/apache/ignite/pull/10261/files]

It contains:
 # A commit for IGNITE-17645 (add last wal segment to SnapshotMetadata)
[https://github.com/apache/ignite/pull/10261/commits/582305821c197cda17e0bbb7a386f2c457fd93a5]
 # Flag for create incremental snapshot
 # Сheck that there is no gaps in WAL segments (IgniteSnapshotManager#validateIncrementalSnapshotSegments, IgniteSnapshotManager#validateCurrentSegments)
 # Checks cache data (IgniteSnapshotManger#checkCacheData), binary meta (IgniteSnapshotManager#checkBinaryMeta)
 # draft of IncrementalSnapshotMetadata
 # Entrypoint for creating incremental snapshot: IgniteSnapshotManager#createIncrementalSnapshot


was (Author: timonin.maksim):
There is a draft PR: [https://github.com/apache/ignite/pull/10261/files]

It contains:
 # A commit for IGNITE-17645 (add last wal segment to SnapshotMetadata)
[https://github.com/apache/ignite/pull/10261/commits/582305821c197cda17e0bbb7a386f2c457fd93a5]
 # Flag for create incremental snapshot
 # Some checks for snapshot: checking that there is no gaps in WAL segments (IgniteSnapshotManager#validateIncrementalSnapshotSegments, IgniteSnapshotManager#validateCurrentSegments)
 # draft of IncrementalSnapshotMetadata
 # Entrypoint for creating incremental snapshot: IgniteSnapshotManager#createIncrementalSnapshot

> Create incremental snapshot
> ---------------------------
>
>                 Key: IGNITE-17613
>                 URL: https://issues.apache.org/jira/browse/IGNITE-17613
>             Project: Ignite
>          Issue Type: Sub-task
>            Reporter: Maksim Timonin
>            Assignee: Maksim Timonin
>            Priority: Major
>              Labels: IEP-89, ise
>
> Incremental snapshot is a lightweight alternative to full snapshot. It bases on the non-blocking Consistent Cut algorithm and provides a collection of WAL segments that hold logical changes since previous snapshot (full or incremental).
> Incremental snapshot should contain:
>  * compacted WAL segments
>  * meta file with Consistent Cut to restore on
>  * binary_meta if it has changed since previous snapshot.
> Incremental snapshot is stored within full snapshot directory.
> Incremental snapshot before creation checks:
>  * Base snapshot (at least its metafile) exists. Exists metafile for all incremental snapshots.
>  * Validate that no misses in WAL segments since previous snapshot.
>  * Check that new _ConsistentCutVersion_ is greater than version of previous snapshots.
>  * Check that baseline topology and cacheGroups are the same (relatively to base snapshot).
> More info in IEP: [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211884314]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)