[jira] [Updated] (IGNITE-10048) Bounded iteration in standalone WAL iterator with compaction enabled may skip records

["Maxim Muzafarov (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Maxim Muzafarov updated IGNITE-10048:
-------------------------------------
    Issue Type: Bug  (was: Improvement)

> Bounded iteration in standalone WAL iterator with compaction enabled may skip records
> -------------------------------------------------------------------------------------
>
>                 Key: IGNITE-10048
>                 URL: https://issues.apache.org/jira/browse/IGNITE-10048
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Ivan Rakov
>            Assignee: Ivan Rakov
>            Priority: Major
>             Fix For: 2.8
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Bounded iteration with non-zero start/end offsets may skip some records in intermediate segments. Reproducer (wal compaction should be enabled):
> {noformat}
>     /**
>      *
>      */
>     public void testBoundedIterationOverSeveralSegments() throws Exception {
>         walCompactionEnabled = true;
>         IgniteEx ig = (IgniteEx)startGrid();
>         String archiveWalDir = getArchiveWalDirPath(ig);
>         ig.cluster().active(true);
>         IgniteCache<Object, Object> cache = ig.getOrCreateCache(
>             new CacheConfiguration<>().setName("c-n").setAffinity(new RendezvousAffinityFunction(false, 32)));
>         IgniteCacheDatabaseSharedManager sharedMgr = ig.context().cache().context().database();
>         IgniteWriteAheadLogManager walMgr = ig.context().cache().context().wal();
>         WALPointer fromPtr = null;
>         int recordsCnt = WAL_SEGMENT_SIZE / 8 /* record size */ * 5;
>         for (int i = 0; i < recordsCnt; i++) {
>             WALPointer ptr = walMgr.log(new PartitionDestroyRecord(i, i));
>             if (i == 100)
>                 fromPtr = ptr;
>         }
>         assertNotNull(fromPtr);
>         cache.put(1, 1);
>         forceCheckpoint();
>         // Generate WAL segments for filling WAL archive folder.
>         for (int i = 0; i < 2 * ig.configuration().getDataStorageConfiguration().getWalSegments(); i++) {
>             sharedMgr.checkpointReadLock();
>             try {
>                 walMgr.log(new SnapshotRecord(i, false), RolloverType.NEXT_SEGMENT);
>             }
>             finally {
>                 sharedMgr.checkpointReadUnlock();
>             }
>         }
>         cache.put(2, 2);
>         forceCheckpoint();
>         U.sleep(5000);
>         stopGrid();
>         WALIterator it = new IgniteWalIteratorFactory(log)
>             .iterator(new IteratorParametersBuilder().from((FileWALPointer)fromPtr).filesOrDirs(archiveWalDir));
>         TreeSet<Integer> foundCounters = new TreeSet<>();
>         it.forEach(x -> {
>             WALRecord rec = x.get2();
>             if (rec instanceof PartitionDestroyRecord)
>                 foundCounters.add(((WalRecordCacheGroupAware)rec).groupId());
>         });
>         assertEquals(new Integer(100), foundCounters.first());
>         assertEquals(new Integer(recordsCnt - 1), foundCounters.last());
>         assertEquals(recordsCnt - 100, foundCounters.size());
>     }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)