You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Ignite TC Bot (Jira)" <ji...@apache.org> on 2022/10/06 15:13:00 UTC
[jira] [Commented] (IGNITE-17613) Create incremental snapshot
[ https://issues.apache.org/jira/browse/IGNITE-17613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17613598#comment-17613598 ]
Ignite TC Bot commented on IGNITE-17613:
----------------------------------------
{panel:title=Branch: [pull/10263/head] Base: [master] : Possible Blockers (1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Control Utility 2{color} [[tests 1|https://ci2.ignite.apache.org/viewLog.html?buildId=6809616]]
* IgniteControlUtilityTestSuite2: GridCommandHandlerDefragmentationTest.testDefragmentationStatus - History for base branch is absent.
{panel}
{panel:title=Branch: [pull/10263/head] Base: [master] : New Tests (450)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#00008b}Control Utility 2{color} [[tests 429|https://ci2.ignite.apache.org/viewLog.html?buildId=6809616]]
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testAtomicAndTxValueAndVersion[strategy=REMOVE, explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testAtomicAndTxVersionOnly[strategy=RELATIVE_MAJORITY, explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testCacheFilter[strategy=RELATIVE_MAJORITY, explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testRepairNonExistentCache[strategy=RELATIVE_MAJORITY, explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testAtomicAndTxValueAndVersion[strategy=REMOVE, explicitGrp=true, callByGrp=true] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testAtomicAndTxVersionOnly[strategy=REMOVE, explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testCacheFilter[strategy=REMOVE, explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testRepairNonExistentCache[strategy=REMOVE, explicitGrp=false, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testAtomicAndTxValueAndVersion[strategy=REMOVE, explicitGrp=true, callByGrp=false] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testAtomicAndTxVersionOnly[strategy=REMOVE, explicitGrp=true, callByGrp=true] - PASSED{color}
* {color:#013220}IgniteControlUtilityTestSuite2: GridCommandHandlerConsistencyTest.testCacheFilter[strategy=REMOVE, explicitGrp=true, callByGrp=true] - PASSED{color}
... and 418 new tests
{color:#00008b}Platform .NET (Windows){color} [[tests 2|https://ci2.ignite.apache.org/viewLog.html?buildId=6809582]]
* {color:#013220}exe: ClientConnectionTest.TestExceptionInRetryPolicyPropagatesToCaller(True) - PASSED{color}
* {color:#013220}exe: ClientConnectionTest.TestExceptionInRetryPolicyPropagatesToCaller(False) - PASSED{color}
{color:#00008b}Basic 1{color} [[tests 1|https://ci2.ignite.apache.org/viewLog.html?buildId=6809504]]
* {color:#013220}IgniteBasicTestSuite: GridFuncSelfTest.testMapEqNotOrdered - PASSED{color}
{color:#00008b}Thin Client: Java{color} [[tests 4|https://ci2.ignite.apache.org/viewLog.html?buildId=6809603]]
* {color:#013220}ClientTestSuite: ReliabilityTestPartitionAwareAsync.testExceptionInRetryPolicyPropagatesToCaller - PASSED{color}
* {color:#013220}ClientTestSuite: ReliabilityTestPartitionAware.testExceptionInRetryPolicyPropagatesToCaller - PASSED{color}
* {color:#013220}ClientTestSuite: ReliabilityTestAsync.testExceptionInRetryPolicyPropagatesToCaller - PASSED{color}
* {color:#013220}ClientTestSuite: ReliabilityTest.testExceptionInRetryPolicyPropagatesToCaller - PASSED{color}
{color:#00008b}Snapshots{color} [[tests 14|https://ci2.ignite.apache.org/viewLog.html?buildId=6809595]]
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testCreation[Encryption=false] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IgniteClusterSnapshotWalRecordTest.testClusterSnapshotRecordIsWrittenToSnapshotMetadata[Encryption=true] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IgniteClusterSnapshotWalRecordTest.testClusterSnapshotRecordIsWrittenToSnapshotMetadata[Encryption=false] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testIncrementalSnapshotFailsOnCacheDestroy[Encryption=false] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testIncrementalSnapshotFailsOnCacheChange[Encryption=false] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testIncrementalSnapshotFailOnDirtyDir[Encryption=false] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testFailForUnknownBaseSnapshot[Encryption=false] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testIncrementalSnapshotFailOnDirtyDir[Encryption=true] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testFailForUnknownBaseSnapshot[Encryption=true] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testCreation[Encryption=true] - PASSED{color}
* {color:#013220}IgniteSnapshotTestSuite: IncrementalSnapshotTest.testIncrementalSnapshotFailsOnTopologyChange[Encryption=false] - PASSED{color}
... and 3 new tests
{panel}
[TeamCity *--> Run :: All* Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6809614&buildTypeId=IgniteTests24Java8_RunAll]
> Create incremental snapshot
> ---------------------------
>
> Key: IGNITE-17613
> URL: https://issues.apache.org/jira/browse/IGNITE-17613
> Project: Ignite
> Issue Type: Sub-task
> Reporter: Maksim Timonin
> Assignee: Maksim Timonin
> Priority: Major
> Labels: IEP-89, ise
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Incremental snapshot is a lightweight alternative to full snapshot. It bases on the non-blocking Consistent Cut algorithm and provides a collection of WAL segments that hold logical changes since previous snapshot (full or incremental).
> Incremental snapshot should contain:
> * compacted WAL segments
> * meta file with Consistent Cut to restore on
> * binary_meta if it has changed since previous snapshot.
> Incremental snapshot is stored within full snapshot directory.
> Incremental snapshot before creation checks:
> * Base snapshot (at least its metafile) exists. Exists metafile for all incremental snapshots.
> * Validate that no misses in WAL segments since previous snapshot.
> * Check that new _ConsistentCutVersion_ is greater than version of previous snapshots.
> * Check that baseline topology and cacheGroups are the same (relatively to base snapshot).
> More info in IEP: [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211884314]
> ----
> Highlevel process of creation of incremental snapshot(without consistent cut algorithm):
> * Incremental snapshot started with create snapshot command with --incremental flag.
> * Incremental snapshot consists of:
> ** WAL segments from previous increment snapshot (or full snapshot in case first incremental snapshot).
> ** Changed binary meta and marshaller files.
> ** Snapshot metafile
> * Incremental snapshot are placed to {{work/snapshots/mybackup/increments/node01/0001}} folder where
> ** mybackup - name of the full snapshot.
> ** node01 - consistent id of the node.
> ** 0001, 0002, etc - number of incremental snapshot.
> * Incremental snapshot creation consists of the following actions executed on each node. The whole process orchestrated by the {{DistributedProcess}} in the same manner as the full snapshot creation:
> ** creation of the snapshot folder
> ** awaits while required WAL segments will be archived
> ** copy (hard-linked) required WAL segments to the incremental snapshot folder.
> ** creates snapshot metafile.
> * Failover guarantees (remove partially created snapshot, etc) should be the same as for the full snapshot.
> * Removal of the full snapshot must also removes all the incremental snapshot based on the full one.
> * Removal of the incremental snapshot may be done only for the last one. If there are next incremental snapshot (0003, for example) then removal of any previous (0001 or 0002) must be restricted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)