[jira] [Created] (IGNITE-11937) Fix MVCC PDS flaky suites timeout

["Alexei Scherbakov (JIRA)" <ji...@apache.org>]

Alexei Scherbakov created IGNITE-11937:
------------------------------------------

             Summary: Fix MVCC PDS flaky suites timeout
                 Key: IGNITE-11937
                 URL: https://issues.apache.org/jira/browse/IGNITE-11937
             Project: Ignite
          Issue Type: Bug
            Reporter: Alexei Scherbakov


Currently we have non-zero failure rate for some MVCC PDS suites in master.

Seems this is due to failure [1] in testRebalancingDuringLoad* tests group, which leads to dumping WAL and lock states at the time proportional to current WAL length increasing test duration for random time depending on WAL length.

Worse thing the test remains green despite throwing a critical exception.

[1]  Stacktrace
{noformat}
[2019-06-19 15:56:53,386][ERROR][sys-stripe-6-#134%persistence.IgnitePdsContinuousRestartTestWithSharedGroupAndIndexes3%][IgniteTestResources] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailure
Handler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [pages(groupId, page
Id)=[IgniteBiTuple [val1=81227264, val2=844420635164676]], msg=Runtime failure on search row: TxKey [major=1560948946388, minor=17286]]]]
class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=81227264, val2=844420635164676]], msg=Runtime failure on search row: TxKey [major=1560948946388, minor=17286]]
        at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5909)
        at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1859)
        at org.apache.ignite.internal.processors.cache.mvcc.txlog.TxLog.put(TxLog.java:293)
        at org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl.updateState(MvccProcessorImpl.java:699)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.setMvccState(IgniteTxManager.java:2570)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1228)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1070)
        at org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.prepareRemoteTx(GridDistributedTxRemoteAdapter.java:421)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.startRemoteTx(IgniteTxHandler.java:1837)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processDhtTxPrepareRequest(IgniteTxHandler.java:1198)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$400(IgniteTxHandler.java:118)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:224)
        at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:222)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1141)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
        at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
        at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1558)
        at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1186)
        at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
        at org.apache.ignite.internal.managers.communication.GridIoManager$8.run(GridIoManager.java:1083)
        at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:559)
        at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Unexpected new transaction state. [currState=2, newState=1, cntr=17286]
        at org.apache.ignite.internal.processors.cache.mvcc.txlog.TxLog$TxLogUpdateClosure.invalid(TxLog.java:629)
        at org.apache.ignite.internal.processors.cache.mvcc.txlog.TxLog$TxLogUpdateClosure.checkAborted(TxLog.java:605)
        at org.apache.ignite.internal.processors.cache.mvcc.txlog.TxLog$TxLogUpdateClosure.call(TxLog.java:499)
        at org.apache.ignite.internal.processors.cache.mvcc.txlog.TxLog$TxLogUpdateClosure.call(TxLog.java:440)
        at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.invokeClosure(BPlusTree.java:3800)
        at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.access$5700(BPlusTree.java:3694)
        at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1948)
        at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1826)
        ... 24 more
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)