[jira] [Commented] (IGNITE-10048) Bounded iteration in standalone WAL iterator with compaction enabled may skip records

["Maxim Muzafarov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948495#comment-16948495 ] 

Maxim Muzafarov commented on IGNITE-10048:
------------------------------------------

Moved to the next release due to inactivity. Please, feel free to move it back if you will be able to complete the ticket by 2.8 code freeze date, December 2, 2019.


> Bounded iteration in standalone WAL iterator with compaction enabled may skip records
> -------------------------------------------------------------------------------------
>
>                 Key: IGNITE-10048
>                 URL: https://issues.apache.org/jira/browse/IGNITE-10048
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Ivan Rakov
>            Assignee: Ivan Rakov
>            Priority: Major
>             Fix For: 2.9
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Bounded iteration with non-zero start/end offsets may skip some records in intermediate segments. Reproducer (wal compaction should be enabled):
> {noformat}
>     /**
>      *
>      */
>     public void testBoundedIterationOverSeveralSegments() throws Exception {
>         walCompactionEnabled = true;
>         IgniteEx ig = (IgniteEx)startGrid();
>         String archiveWalDir = getArchiveWalDirPath(ig);
>         ig.cluster().active(true);
>         IgniteCache<Object, Object> cache = ig.getOrCreateCache(
>             new CacheConfiguration<>().setName("c-n").setAffinity(new RendezvousAffinityFunction(false, 32)));
>         IgniteCacheDatabaseSharedManager sharedMgr = ig.context().cache().context().database();
>         IgniteWriteAheadLogManager walMgr = ig.context().cache().context().wal();
>         WALPointer fromPtr = null;
>         int recordsCnt = WAL_SEGMENT_SIZE / 8 /* record size */ * 5;
>         for (int i = 0; i < recordsCnt; i++) {
>             WALPointer ptr = walMgr.log(new PartitionDestroyRecord(i, i));
>             if (i == 100)
>                 fromPtr = ptr;
>         }
>         assertNotNull(fromPtr);
>         cache.put(1, 1);
>         forceCheckpoint();
>         // Generate WAL segments for filling WAL archive folder.
>         for (int i = 0; i < 2 * ig.configuration().getDataStorageConfiguration().getWalSegments(); i++) {
>             sharedMgr.checkpointReadLock();
>             try {
>                 walMgr.log(new SnapshotRecord(i, false), RolloverType.NEXT_SEGMENT);
>             }
>             finally {
>                 sharedMgr.checkpointReadUnlock();
>             }
>         }
>         cache.put(2, 2);
>         forceCheckpoint();
>         U.sleep(5000);
>         stopGrid();
>         WALIterator it = new IgniteWalIteratorFactory(log)
>             .iterator(new IteratorParametersBuilder().from((FileWALPointer)fromPtr).filesOrDirs(archiveWalDir));
>         TreeSet<Integer> foundCounters = new TreeSet<>();
>         it.forEach(x -> {
>             WALRecord rec = x.get2();
>             if (rec instanceof PartitionDestroyRecord)
>                 foundCounters.add(((WalRecordCacheGroupAware)rec).groupId());
>         });
>         assertEquals(new Integer(100), foundCounters.first());
>         assertEquals(new Integer(recordsCnt - 1), foundCounters.last());
>         assertEquals(recordsCnt - 100, foundCounters.size());
>     }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)