You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Sergey Antonov (JIRA)" <ji...@apache.org> on 2018/10/23 08:37:00 UTC

[jira] [Created] (IGNITE-9969) Race on accessing to WAL segment on node startup.

Sergey Antonov created IGNITE-9969:
--------------------------------------

             Summary: Race on accessing to WAL segment on node startup.
                 Key: IGNITE-9969
                 URL: https://issues.apache.org/jira/browse/IGNITE-9969
             Project: Ignite
          Issue Type: Bug
            Reporter: Sergey Antonov


We could get race on WAL segment on node startup. In same time, one thread will be archive WAL segment and another thread will be apply last updates after last checkpoint from this WAL segment.
{noformat}
[2018-10-22 17:08:30,763][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=0, segIdx=0, origFile=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dstFile=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]
[2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null]
[2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2018-10-22 17:08:30,807][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Failed to reinitialize local partitions (rebalancing will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage [id=4bc2a1c9661-142c551e-2962-403a-b864-0281692574ee, reqId=4ca70fa9-7a54-411a-8ad5-f4633e40edd4, initiatingNodeId=3f392b21-9ce6-4ce5-811e-c66796800000, activate=true, baselineTopology=BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]], forceChangeBaselineTopology=false, timestamp=1540217310624], affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT]
class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	at java.lang.Thread.run(Thread.java:748)
[2018-10-22 17:08:30,926][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Copied file [src=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dst=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]{noformat}
h4. How to reproduce:

Run IgniteCompatibilityBasicTestSuite and test PersistenceBasicCompatibilityTest#testNodeStartByOldVersionPersistenceData_2_3() could fail with following exception:

 
{noformat}
[2018-10-22 17:08:30,819][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCachePartitionExchangeManager] Failed to wait for completion of partition map exchange (preloading will not start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], crd=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=false, hash=364933128], init=false, lastVer=null, partReleaseFut=AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]PartitionReleaseFuture [topVer=, futures=[ExplicitLockReleaseFuture [topVer=, futures=[]], AtomicUpdateReleaseFuture [topVer=, futures=[]], DataStreamerReleaseFuture [topVer=, futures=[]], LocalTxReleaseFuture [topVer=, futures=[]], AllTxReleaseFuture [topVer=, futures=[RemoteTxReleaseFuture [topVer=, futures=[]]]]]], exchActions=null, affChangeMsg=null, initTs=1540217310618, centralizedAff=false, forceAffReassignment=true, exchangeLocE=null, cacheChangeFailureMsgSent=false, done=true, state=CRD, registerCachesFuture=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1024540699], partitionsSent=false, partitionsReceived=false, delayedLatestMsg=null, afterLsnrCompleteFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1881071890], evtLatch=0, remaining=HashSet [], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, hash=1188696354]]
class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process.

	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:335)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378)
	... 15 more{noformat}
 

Below is a full log of test:
{noformat}
[2018-10-22 17:08:08,194][INFO ][main][root] >>> Starting test: PersistenceBasicCompatibilityTest#testNodeStartByOldVersionPersistenceData_2_3 <<<
[2018-10-22 17:08:08,582][INFO ][Thread-16][jvm-fcbe9773#2_3_0] Starting Ignite Node... Args=[C:\Users\santonov\AppData\Local\Temp\\igniteCompatibilityClosure.tmp_507120272, persistence.PersistenceBasicCompatibilityTest1, fcbe9773-6b4b-495d-bd36-f0fe336ff54e, fcbe9773-6b4b-495d-bd36-f0fe336ff54e, C:\Users\santonov\AppData\Local\Temp\\igniteCompatibilityClosure.tmp_348283360]
[2018-10-22 17:08:09,079][INFO ][Thread-16][jvm-fcbe9773#2_3_0] Configured log4j from: C:\IdeaProjects\apache-ignite\modules\core\src\test\config\log4j-test.xml
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,473][INFO ][main][PersistenceBasicCompatibilityTest1] 
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] 
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>>    __________  ________________  
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>>   /  _/ ___/ |/ /  _/_  __/ __/  
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>>  _/ // (7 7    // /  / / / _/    
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> /___/\___/_/|_/___/ /_/ /___/   
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> 
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> ver. 2.3.0#20171028-sha1:8add7fd5
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> 2017 Copyright(C) Apache Software Foundation
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> 
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Ignite documentation: http://ignite.apache.org
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] 
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] Config URL: n/a
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] Daemon mode: off
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] OS: Windows 10 10.0 amd64
[2018-10-22 17:08:09,474][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] OS user: santonov
[2018-10-22 17:08:09,475][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,474][INFO ][main][PersistenceBasicCompatibilityTest1] PID: 15928
[2018-10-22 17:08:09,475][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,475][INFO ][main][PersistenceBasicCompatibilityTest1] Language runtime: Java Platform API Specification ver. 1.8
[2018-10-22 17:08:09,475][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,475][INFO ][main][PersistenceBasicCompatibilityTest1] VM information: Java(TM) SE Runtime Environment 1.8.0_181-b13 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.181-b13
[2018-10-22 17:08:09,476][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,476][INFO ][main][PersistenceBasicCompatibilityTest1] VM total memory: 3.5GB
[2018-10-22 17:08:09,476][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,476][INFO ][main][PersistenceBasicCompatibilityTest1] Remote Management [restart: off, REST: off, JMX (remote: off)]
[2018-10-22 17:08:09,478][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,478][INFO ][main][PersistenceBasicCompatibilityTest1] IGNITE_HOME=C:\IdeaProjects\apache-ignite
[2018-10-22 17:08:09,479][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,479][INFO ][main][PersistenceBasicCompatibilityTest1] VM arguments: [-ea]
[2018-10-22 17:08:09,479][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,479][INFO ][main][PersistenceBasicCompatibilityTest1] System cache's DataRegion size is configured to 40 MB. Use DataStorageConfiguration.systemCacheMemorySize property to change the setting.
[2018-10-22 17:08:09,487][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,487][INFO ][main][PersistenceBasicCompatibilityTest1] Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']]
[2018-10-22 17:08:09,490][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,490][WARN ][pub-#19%persistence.PersistenceBasicCompatibilityTest1%][GridDiagnostic] This operating system has been tested less rigorously: Windows 10 10.0 amd64. Our team will appreciate the feedback if you experience any problems running ignite in this environment.
[2018-10-22 17:08:09,491][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,491][INFO ][main][PersistenceBasicCompatibilityTest1] 3-rd party licenses can be found at: C:\IdeaProjects\apache-ignite\libs\licenses
[2018-10-22 17:08:09,493][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,493][WARN ][pub-#22%persistence.PersistenceBasicCompatibilityTest1%][GridDiagnostic] Initial heap size is 254MB (should be no less than 512MB, use -Xms512m -Xmx512m).
[2018-10-22 17:08:09,509][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor] Configured plugins:
[2018-10-22 17:08:09,509][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor]   ^-- TestCompatibilityPlugin 1.0
[2018-10-22 17:08:09,509][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor]   ^-- 
[2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,509][INFO ][main][IgnitePluginProcessor] 
[2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor]   ^-- TestReconnectPlugin 1.0
[2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor]   ^-- 
[2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] 
[2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor]   ^-- StanByClusterTestProvider 1.0
[2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor]   ^-- null
[2018-10-22 17:08:09,510][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,510][INFO ][main][IgnitePluginProcessor] 
[2018-10-22 17:08:09,614][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,614][INFO ][main][TcpCommunicationSpi] Successfully bound communication NIO server to TCP port [port=45010, locHost=/127.0.0.1, selectorsCnt=4, selectorSpins=0, pairedConn=false]
[2018-10-22 17:08:09,615][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,615][WARN ][main][TcpCommunicationSpi] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.
[2018-10-22 17:08:09,748][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,747][WARN ][main][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival).
[2018-10-22 17:08:09,749][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,749][INFO ][main][PersistenceBasicCompatibilityTest1] Security status [authentication=off, tls/ssl=off]
[2018-10-22 17:08:09,789][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,789][INFO ][main][TcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=127.0.0.1/127.0.0.1, locNodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e]
[2018-10-22 17:08:09,808][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,808][INFO ][main][PdsFoldersResolver] Successfully created new persistent storage folder [C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]
[2018-10-22 17:08:09,809][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,809][INFO ][main][PdsFoldersResolver] Consistent ID used for local node is [81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332] according to persistence data storage folders
[2018-10-22 17:08:09,810][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,810][INFO ][main][CacheObjectBinaryProcessorImpl] Resolved directory for serialized binary metadata: C:\IdeaProjects\apache-ignite\work\binary_meta\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:09,932][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,931][INFO ][main][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:09,932][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,932][INFO ][main][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:09,933][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:09,933][INFO ][main][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:10,064][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,063][INFO ][main][FileWriteAheadLogManager] Started write-ahead log manager [mode=DEFAULT]
[2018-10-22 17:08:10,146][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,146][INFO ][main][ClientListenerProcessor] Client connector processor has started on TCP port 10800
[2018-10-22 17:08:10,559][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,559][INFO ][main][PersistenceBasicCompatibilityTest1] Non-loopback local IPs: 10.0.0.100, fe80:0:0:0:41c9:b4fa:d846:61ac%eth5, fe80:0:0:0:5422:f228:82d7:f577%eth0, fe80:0:0:0:9c7:33de:6920:739e%wlan2, fe80:0:0:0:a99b:6b5:1b8c:71d6%wlan0, fe80:0:0:0:c1f7:1e90:f607:de9d%wlan1, fe80:0:0:0:cd36:5279:6ba0:90aa%eth2, fe80:0:0:0:f0d5:95b5:b835:21dc%eth1
[2018-10-22 17:08:10,559][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:10,559][INFO ][main][PersistenceBasicCompatibilityTest1] Enabled local MACs: 00A0C6000000, 30243242E663, 30243242E664, 30243242E667, 32243242E663, 482AE30AD0AE, E04F439828DF
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] Performance suggestions for grid 'persistence.PersistenceBasicCompatibilityTest1' (fix if possible)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1]   ^-- Disable checkpoints (remove 'checkpointSpi' from configuration)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1]   ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1]   ^-- Specify JVM heap max size (add '-Xmx<size>[g|G|m|M|k|K]' to JVM options)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1]   ^-- Set max direct memory size if getting 'OOME: Direct buffer memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1]   ^-- Disable processing of calls to System.gc() (add '-XX:+DisableExplicitGC' to JVM options)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1]   ^-- Disable assertions (remove '-ea' from JVM options)
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] Refer to this page for more performance suggestions: https://apacheignite.readme.io/docs/jvm-and-system-tuning
[2018-10-22 17:08:19,706][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] 
[2018-10-22 17:08:19,707][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,706][INFO ][main][PersistenceBasicCompatibilityTest1] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[2018-10-22 17:08:19,707][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,707][INFO ][main][PersistenceBasicCompatibilityTest1] 
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,708][INFO ][main][PersistenceBasicCompatibilityTest1] 
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] 
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> +----------------------------------------------------------------------+
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Ignite ver. 2.3.0#20171028-sha1:8add7fd5b501b40658096cdde48af9e948aa8150
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> +----------------------------------------------------------------------+
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> OS name: Windows 10 10.0 amd64
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> CPU(s): 8
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Heap: 3.5GB
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> VM name: 15928@LAPTOP-IGKOMEL9
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Ignite instance name: persistence.PersistenceBasicCompatibilityTest1
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Local node [ID=FCBE9773-6B4B-495D-BD36-F0FE336FF54E, order=1, clientMode=false]
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Local node addresses: [127.0.0.1]
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] >>> Local ports: TCP:10800 TCP:45010 TCP:47500 
[2018-10-22 17:08:19,708][INFO ][Thread-16][jvm-fcbe9773#2_3_0] 
[2018-10-22 17:08:19,709][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,709][INFO ][main][GridDiscoveryManager] Topology snapshot [ver=1, servers=1, clients=0, CPUs=8, heap=3.5GB]
[2018-10-22 17:08:19,721][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,720][INFO ][tcp-disco-msg-worker-#4%persistence.PersistenceBasicCompatibilityTest1%][GridClusterStateProcessor] Started state transition: true
[2018-10-22 17:08:19,733][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, customEvt=ChangeGlobalStateMessage [id=feaba1c9661-d034c9b8-c91f-4b97-91e2-99320879a2f4, reqId=6578bb8f-b31c-45a0-955c-5360ae2589d6, initiatingNodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, activate=true], allowMerge=false]
[2018-10-22 17:08:19,733][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Start activation process [nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
[2018-10-22 17:08:19,733][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:19,734][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,733][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:19,734][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,734][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:19,735][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,735][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Started write-ahead log manager [mode=DEFAULT]
[2018-10-22 17:08:19,794][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,793][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][PageMemoryImpl] Started page memory [memoryAllocated=3.2 GiB, pages=808272, tableSize=46.9 MiB, checkpointBuffer=810.3 MiB]
[2018-10-22 17:08:19,818][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,818][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null]
[2018-10-22 17:08:19,819][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,818][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false], lastMarked=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false], lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2018-10-22 17:08:19,829][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,829][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][FileWriteAheadLogManager] Resuming logging to WAL segment [file=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, offset=0, ver=1]
[2018-10-22 17:08:19,963][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,963][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheProcessor] Started cache [name=ignite-sys-cache, memoryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL]
[2018-10-22 17:08:19,966][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,966][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Successfully activated caches [nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
[2018-10-22 17:08:19,971][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:19,971][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], waitTime=0ms, futInfo=NA]
[2018-10-22 17:08:20,018][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,018][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null]
[2018-10-22 17:08:20,018][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,018][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOffset=0, len=0, forceFlush=false], lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2018-10-22 17:08:20,086][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,085][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheDatabaseSharedManager] Finished applying WAL changes [updatesApplied=0, time=73ms]
[2018-10-22 17:08:20,088][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,088][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
[2018-10-22 17:08:20,103][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,103][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], err=null]
[2018-10-22 17:08:20,103][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,103][INFO ][tcp-disco-msg-worker-#4%persistence.PersistenceBasicCompatibilityTest1%][GridClusterStateProcessor] Received state change finish message: true
[2018-10-22 17:08:20,113][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,112][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], crd=true]
[2018-10-22 17:08:20,117][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,117][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=1, minorTopVer=1], evt=DISCOVERY_CUSTOM_EVT, node=fcbe9773-6b4b-495d-bd36-f0fe336ff54e]
[2018-10-22 17:08:20,138][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,138][INFO ][sys-#39%persistence.PersistenceBasicCompatibilityTest1%][GridClusterStateProcessor] Successfully performed final activation steps [nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
[2018-10-22 17:08:20,156][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,156][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, customEvt=DynamicCacheChangeBatch [id=9faba1c9661-d034c9b8-c91f-4b97-91e2-99320879a2f4, reqs=[DynamicCacheChangeRequest [cacheName=PersistenceBasicCompatibilityTest, hasCfg=true, nodeId=fcbe9773-6b4b-495d-bd36-f0fe336ff54e, clientStartOnly=false, stop=false, destroy=false]], exchangeActions=ExchangeActions [startCaches=[PersistenceBasicCompatibilityTest], stopCaches=null, startGrps=[PersistenceBasicCompatibilityTest], stopGrps=[], resetParts=null, stateChangeRequest=null], startCaches=false], allowMerge=false]
[2018-10-22 17:08:20,242][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,242][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCacheProcessor] Started cache [name=PersistenceBasicCompatibilityTest, memoryPolicyName=default, mode=PARTITIONED, atomicity=TRANSACTIONAL]
[2018-10-22 17:08:20,253][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,252][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], waitTime=0ms, futInfo=NA]
[2018-10-22 17:08:20,316][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,316][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] finishExchangeOnCoordinator [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=2]]
[2018-10-22 17:08:20,321][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,321][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], resVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], err=null]
[2018-10-22 17:08:20,322][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,322][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][time] Finished exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=2], crd=true]
[2018-10-22 17:08:20,328][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [2018-10-22 17:08:20,328][INFO ][exchange-worker-#37%persistence.PersistenceBasicCompatibilityTest1%][GridCachePartitionExchangeManager] Skipping rebalancing (nothing scheduled) [top=AffinityTopologyVersion [topVer=1, minorTopVer=2], evt=DISCOVERY_CUSTOM_EVT, node=fcbe9773-6b4b-495d-bd36-f0fe336ff54e]
[2018-10-22 17:08:20,633][INFO ][Thread-16][jvm-fcbe9773#2_3_0] [Compatibility] Node has been started, id=fcbe9773-6b4b-495d-bd36-f0fe336ff54e
[2018-10-22 17:08:20,815][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] 

>>>    __________  ________________  
>>>   /  _/ ___/ |/ /  _/_  __/ __/  
>>>  _/ // (7 7    // /  / / / _/    
>>> /___/\___/_/|_/___/ /_/ /___/   
>>> 
>>> ver. 2.7.0-SNAPSHOT#19700101-sha1:DEV
>>> 2018 Copyright(C) Apache Software Foundation
>>> 
>>> Ignite documentation: http://ignite.apache.org

[2018-10-22 17:08:20,815][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Config URL: n/a
[2018-10-22 17:08:20,817][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] IgniteConfiguration [igniteInstanceName=persistence.PersistenceBasicCompatibilityTest0, pubPoolSize=8, svcPoolSize=8, callbackPoolSize=8, stripedPoolSize=8, sysPoolSize=8, mgmtPoolSize=4, igfsPoolSize=8, dataStreamerPoolSize=8, utilityCachePoolSize=8, utilityCacheKeepAliveTime=60000, p2pPoolSize=2, qryPoolSize=8, igniteHome=C:\IdeaProjects\apache-ignite, igniteWorkDir=C:\IdeaProjects\apache-ignite\work, mbeanSrv=com.sun.jmx.mbeanserver.JmxMBeanServer@38cccef, nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, marsh=BinaryMarshaller [], marshLocJobs=false, daemon=false, p2pEnabled=false, netTimeout=10000, sndRetryDelay=1000, sndRetryCnt=3, metricsHistSize=10000, metricsUpdateFreq=1000, metricsExpTime=9223372036854775807, discoSpi=TcpDiscoverySpi [addrRslvr=null, sockTimeout=0, ackTimeout=0, marsh=null, reconCnt=10, reconDelay=2000, maxAckTimeout=600000, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false], segPlc=STOP, segResolveAttempts=2, waitForSegOnStart=true, allResolversPassReq=true, segChkFreq=10000, commSpi=TcpCommunicationSpi [connectGate=null, connPlc=org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$FirstConnectionPolicy@2eceb355, enableForcibleNodeKill=false, enableTroubleshootingLog=false, locAddr=null, locHost=null, locPort=45010, locPortRange=100, shmemPort=-1, directBuf=true, directSndBuf=false, idleConnTimeout=600000, connTimeout=5000, maxConnTimeout=600000, reconCnt=10, sockSndBuf=32768, sockRcvBuf=32768, msgQueueLimit=0, slowClientQueueLimit=0, nioSrvr=null, shmemSrv=null, usePairedConnections=false, connectionsPerNode=1, tcpNoDelay=true, filterReachableAddresses=false, ackSndThreshold=32, unackedMsgsBufSize=0, sockWriteTimeout=2000, boundTcpPort=-1, boundTcpShmemPort=-1, selectorsCnt=4, selectorSpins=0, addrRslvr=null, ctxInitLatch=java.util.concurrent.CountDownLatch@112888fe[Count = 1], stopping=false], evtSpi=MemoryEventStorageSpi [expireAgeMs=9223372036854775807, expireCnt=10000, filter=null], colSpi=NoopCollisionSpi [], deploySpi=LocalDeploymentSpi [], indexingSpi=org.apache.ignite.spi.indexing.noop.NoopIndexingSpi@6cd8ff3f, addrRslvr=null, encryptionSpi=org.apache.ignite.spi.encryption.noop.NoopEncryptionSpi@7ebddfb2, clientMode=false, rebalanceThreadPoolSize=1, txCfg=TransactionConfiguration [txSerEnabled=false, dfltIsolation=REPEATABLE_READ, dfltConcurrency=PESSIMISTIC, dfltTxTimeout=0, txTimeoutOnPartitionMapExchange=0, pessimisticTxLogSize=0, pessimisticTxLogLinger=10000, tmLookupClsName=null, txManagerFactory=null, useJtaSync=false], cacheSanityCheckEnabled=true, discoStartupDelay=60000, deployMode=SHARED, p2pMissedCacheSize=100, locHost=127.0.0.1, timeSrvPortBase=31100, timeSrvPortRange=100, failureDetectionTimeout=10000, sysWorkerBlockedTimeout=null, clientFailureDetectionTimeout=10000, metricsLogFreq=0, hadoopCfg=null, connectorCfg=null, odbcCfg=null, warmupClos=null, atomicCfg=AtomicConfiguration [seqReserveSize=1000, cacheMode=PARTITIONED, backups=1, aff=null, grpName=null], classLdr=null, sslCtxFactory=null, platformCfg=null, binaryCfg=BinaryConfiguration [idMapper=null, nameMapper=null, serializer=null, compactFooter=true], memCfg=null, pstCfg=null, dsCfg=DataStorageConfiguration [sysRegionInitSize=41943040, sysRegionMaxSize=104857600, pageSize=0, concLvl=0, dfltDataRegConf=DataRegionConfiguration [name=default, maxSize=268435456, initSize=268435456, swapPath=null, pageEvictionMode=DISABLED, evictionThreshold=0.9, emptyPagesPoolSize=100, metricsEnabled=false, metricsSubIntervalCount=5, metricsRateTimeInterval=60000, persistenceEnabled=true, checkpointPageBufSize=0], dataRegions=null, storagePath=null, checkpointFreq=180000, lockWaitTime=10000, checkpointThreads=4, checkpointWriteOrder=SEQUENTIAL, walHistSize=20, maxWalArchiveSize=1073741824, walSegments=10, walSegmentSize=67108864, walPath=db/wal, walArchivePath=db/wal/archive, metricsEnabled=false, walMode=LOG_ONLY, walTlbSize=131072, walBuffSize=0, walFlushFreq=2000, walFsyncDelay=1000, walRecordIterBuffSize=67108864, alwaysWriteFullPages=false, fileIOFactory=org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory@5c3972f7, metricsSubIntervalCnt=5, metricsRateTimeInterval=60000, walAutoArchiveAfterInactivity=-1, writeThrottlingEnabled=false, walCompactionEnabled=false, walCompactionLevel=1, checkpointReadLockTimeout=null], activeOnStart=true, autoActivation=true, longQryWarnTimeout=3000, sqlConnCfg=null, cliConnCfg=ClientConnectorConfiguration [host=null, port=10800, portRange=100, sockSndBufSize=0, sockRcvBufSize=0, tcpNoDelay=true, maxOpenCursorsPerConn=128, threadPoolSize=8, idleTimeout=0, jdbcEnabled=true, odbcEnabled=true, thinCliEnabled=true, sslEnabled=false, useIgniteSslCtxFactory=true, sslClientAuth=false, sslCtxFactory=null], mvccVacuumThreadCnt=2, mvccVacuumFreq=5000, authEnabled=false, failureHnd=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]NoOpFailureHandler [super=], commFailureRslvr=null]
[2018-10-22 17:08:20,818][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Daemon mode: off
[2018-10-22 17:08:20,818][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] OS: Windows 10 10.0 amd64
[2018-10-22 17:08:20,818][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] OS user: santonov
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] PID: 17528
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Language runtime: Java Platform API Specification ver. 1.8
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] VM information: Java(TM) SE Runtime Environment 1.8.0_181-b13 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.181-b13
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] VM total memory: 3.5GB
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Remote Management [restart: off, REST: off, JMX (remote: off)]
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Logger: GridTestLog4jLogger [quiet=false, config=null]
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] IGNITE_HOME=C:\IdeaProjects\apache-ignite
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] VM arguments: [-ea, -Didea.test.cyclic.buffer.size=67108864, -javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.2.2\lib\idea_rt.jar=53501:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2018.2.2\bin, -Dfile.encoding=UTF-8]
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] System cache's DataRegion size is configured to 40 MB. Use DataStorageConfiguration.systemRegionInitialSize property to change the setting.
[2018-10-22 17:08:20,819][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Configured caches [in 'sysMemPlc' dataRegion: ['ignite-sys-cache']]
[2018-10-22 17:08:20,820][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] 3-rd party licenses can be found at: C:\IdeaProjects\apache-ignite\libs\licenses
[2018-10-22 17:08:20,819][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Please set system property '-Djava.net.preferIPv4Stack=true' to avoid possible problems in mixed environments.
[2018-10-22 17:08:20,820][WARN ][pub-#132%persistence.PersistenceBasicCompatibilityTest0%][GridDiagnostic] This operating system has been tested less rigorously: Windows 10 10.0 amd64. Our team will appreciate the feedback if you experience any problems running ignite in this environment.
[2018-10-22 17:08:20,820][WARN ][pub-#135%persistence.PersistenceBasicCompatibilityTest0%][GridDiagnostic] Initial heap size is 254MB (should be no less than 512MB, use -Xms512m -Xmx512m).
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] Configured plugins:
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- TestCompatibilityPlugin 1.0
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- 
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] 
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- TestReconnectPlugin 1.0
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- 
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] 
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- StanByClusterTestProvider 1.0
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- null
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] 
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- PageMemory tracker plugin 1.0
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor]   ^-- 
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][IgnitePluginProcessor] 
[2018-10-22 17:08:20,825][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FailureProcessor] Configured failure handler: [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]]]
[2018-10-22 17:08:20,830][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TcpCommunicationSpi] Message queue limit is set to 0 which may lead to potential OOMEs when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message queues growth on sender and receiver sides.
[2018-10-22 17:08:20,829][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TcpCommunicationSpi] Successfully bound communication NIO server to TCP port [port=45010, locHost=/127.0.0.1, selectorsCnt=4, selectorSpins=0, pairedConn=false]

[2018-10-22 17:08:21,019][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Security status [authentication=off, tls/ssl=off][2018-10-22 17:08:21,019][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCollisionManager] Collision resolution is disabled (all jobs will be activated upon arrival).
[2018-10-22 17:08:21,022][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TestTcpDiscoverySpi] Successfully bound to TCP port [port=47500, localHost=127.0.0.1/127.0.0.1, locNodeId=3f392b21-9ce6-4ce5-811e-c66796800000]
[2018-10-22 17:08:21,029][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PdsFoldersResolver] Successfully locked persistence storage folder [C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]
[2018-10-22 17:08:21,029][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PdsFoldersResolver] Consistent ID used for local node is [81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332] according to persistence data storage folders
[2018-10-22 17:08:21,030][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][CacheObjectBinaryProcessorImpl] Resolved directory for serialized binary metadata: C:\IdeaProjects\apache-ignite\work\binary_meta\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:21,034][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:21,035][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:21,035][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:21,046][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][FileWriteAheadLogManager] Started write-ahead log manager [mode=LOG_ONLY]
[2018-10-22 17:08:21,048][WARN ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] DataRegionConfiguration.maxWalArchiveSize instead DataRegionConfiguration.walHistorySize would be used for removing old archive wal files
[2018-10-22 17:08:21,049][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PartitionsEvictManager] Evict partition permits=2
[2018-10-22 17:08:21,053][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][ClientListenerProcessor] Client connector processor has started on TCP port 10800
[2018-10-22 17:08:21,413][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Non-loopback local IPs: 10.0.0.100, fe80:0:0:0:41c9:b4fa:d846:61ac%eth5, fe80:0:0:0:5422:f228:82d7:f577%eth0, fe80:0:0:0:9c7:33de:6920:739e%wlan2, fe80:0:0:0:a99b:6b5:1b8c:71d6%wlan0, fe80:0:0:0:c1f7:1e90:f607:de9d%wlan1, fe80:0:0:0:cd36:5279:6ba0:90aa%eth2, fe80:0:0:0:f0d5:95b5:b835:21dc%eth1
[2018-10-22 17:08:21,413][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Enabled local MACs: 00A0C6000000, 30243242E663, 30243242E664, 30243242E667, 32243242E663, 482AE30AD0AE, E04F439828DF
[2018-10-22 17:08:21,414][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null]
[2018-10-22 17:08:21,416][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB]
[2018-10-22 17:08:21,416][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=0, fileOff=0, len=0], lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2018-10-22 17:08:21,470][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2018-10-22 17:08:21,522][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridCacheDatabaseSharedManager] Finished applying WAL changes [updatesApplied=0, time=52ms]
[2018-10-22 17:08:21,531][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][TestTcpDiscoverySpi] Connection check threshold is calculated: 10000
[2018-10-22 17:08:30,612][INFO ][disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%][MvccProcessorImpl] Assigned mvcc coordinator [crd=MvccCoordinator [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, crdVer=1540217310609, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=0]], crdNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false]]
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Performance suggestions for grid 'persistence.PersistenceBasicCompatibilityTest0' (fix if possible)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- Disable checkpoints (remove 'checkpointSpi' from configuration)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- Specify JVM heap max size (add '-Xmx<size>[g|G|m|M|k|K]' to JVM options)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- Set max direct memory size if getting 'OOME: Direct buffer memory' (add '-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]' to JVM options)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- Disable processing of calls to System.gc() (add '-XX:+DisableExplicitGC' to JVM options)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- Disable assertions (remove '-ea' from JVM options)
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Refer to this page for more performance suggestions: https://apacheignite.readme.io/docs/jvm-and-system-tuning
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] 
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] Data Regions Configured:
[2018-10-22 17:08:30,617][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0]   ^-- default [initSize=256.0 MiB, maxSize=256.0 MiB, persistence=true]
[2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] 

>>> +------------------------------------------+
>>> Ignite ver. 2.7.0-SNAPSHOT#19700101-sha1:DEV
>>> +------------------------------------------+
>>> OS name: Windows 10 10.0 amd64
>>> CPU(s): 8
>>> Heap: 3.5GB
>>> VM name: 17528@LAPTOP-IGKOMEL9
>>> Ignite instance name: persistence.PersistenceBasicCompatibilityTest0
>>> Local node [ID=3F392B21-9CE6-4CE5-811E-C66796800000, order=1, clientMode=false]
>>> Local node addresses: [127.0.0.1]
>>> Local ports: TCP:10800 TCP:45010 TCP:47500 

[2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][PersistenceBasicCompatibilityTest0] >>> Ignite cluster is not active (limited functionality available). Use control.(sh|bat) script or IgniteCluster interface to activate.
[2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridDiscoveryManager] Topology snapshot [ver=1, locNode=3f392b21, servers=1, clients=0, state=INACTIVE, CPUs=8, offheap=0.25GB, heap=3.5GB]
[2018-10-22 17:08:30,618][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][root] Node started with the following configuration [id=3f392b21-9ce6-4ce5-811e-c66796800000, marshaller=BinaryMarshaller [], discovery=TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@3c9eddf4], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false], binaryCfg=BinaryConfiguration [idMapper=null, nameMapper=null, serializer=null, compactFooter=true], lateAff=true]
[2018-10-22 17:08:30,624][INFO ][test-runner-#113%persistence.PersistenceBasicCompatibilityTest%][GridClusterStateProcessor] Sending activate request with BaselineTopology BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]]
[2018-10-22 17:08:30,625][INFO ][disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%][GridClusterStateProcessor] Received activate request with BaselineTopology[id=0]
[2018-10-22 17:08:30,626][INFO ][disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%][GridClusterStateProcessor] Started state transition: true
[2018-10-22 17:08:30,627][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][time] Started exchange init [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], mvccCrd=MvccCoordinator [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, crdVer=1540217310609, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=0]], mvccCrdChange=true, crd=true, evt=DISCOVERY_CUSTOM_EVT, evtNode=3f392b21-9ce6-4ce5-811e-c66796800000, customEvt=ChangeGlobalStateMessage [id=4bc2a1c9661-142c551e-2962-403a-b864-0281692574ee, reqId=4ca70fa9-7a54-411a-8ad5-f4633e40edd4, initiatingNodeId=3f392b21-9ce6-4ce5-811e-c66796800000, activate=true, baselineTopology=BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]], forceChangeBaselineTopology=false, timestamp=1540217310624], allowMerge=false]
[2018-10-22 17:08:30,627][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Start activation process [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
[2018-10-22 17:08:30,627][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FilePageStoreManager] Resolved page store work directory: C:\IdeaProjects\apache-ignite\work\db\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:30,628][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Resolved write ahead log work directory: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:30,628][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Resolved write ahead log archive directory: C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332
[2018-10-22 17:08:30,630][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Started write-ahead log manager [mode=LOG_ONLY]
[2018-10-22 17:08:30,633][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB]
[2018-10-22 17:08:30,638][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=256.0 MiB, pages=63520, tableSize=4.9 MiB, checkpointBuffer=256.0 MiB]
[2018-10-22 17:08:30,640][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB]
[2018-10-22 17:08:30,641][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][PageMemoryImpl] Started page memory [memoryAllocated=100.0 MiB, pages=24808, tableSize=1.9 MiB, checkpointBuffer=100.0 MiB]
[2018-10-22 17:08:30,641][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][msg] Components activation performed in 14 ms.
[2018-10-22 17:08:30,691][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null]
[2018-10-22 17:08:30,691][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Checking memory state [lastValidPos=FileWALPointer [idx=0, fileOff=0, len=0], lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2018-10-22 17:08:30,756][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Resuming logging to WAL segment [file=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, offset=140187, ver=1]
[2018-10-22 17:08:30,756][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Record serializer version change detected, will start logging with a new WAL record serializer to a new WAL segment [curFile=org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle@7191bd14, newVer=2, oldVer=1]
[2018-10-22 17:08:30,763][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Starting to copy WAL segment [absIdx=0, segIdx=0, origFile=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dstFile=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]
[2018-10-22 17:08:30,777][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridClusterStateProcessor] Writing BaselineTopology[id=0]
[2018-10-22 17:08:30,779][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Binary recovery performed in 137 ms.
[2018-10-22 17:08:30,781][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][CacheAffinitySharedManager] Updating caches registry performed in 2 ms.
[2018-10-22 17:08:30,788][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheProcessor] Started cache [name=ignite-sys-cache, id=-2100569601, memoryPolicyName=sysMemPlc, mode=REPLICATED, atomicity=TRANSACTIONAL, backups=2147483647, mvcc=false], encryptionEnabled=false]
[2018-10-22 17:08:30,795][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheProcessor] Started cache [name=PersistenceBasicCompatibilityTest, id=2041049331, memoryPolicyName=default, mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=1, mvcc=false], encryptionEnabled=false]
[2018-10-22 17:08:30,795][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][CacheAffinitySharedManager] Caches starting performed in 14 ms.
[2018-10-22 17:08:30,796][INFO ][sys-#157%persistence.PersistenceBasicCompatibilityTest0%][GridAffinityAssignmentCache] Local node affinity assignment distribution is not ideal [cache=PersistenceBasicCompatibilityTest, expectedPrimary=1024.00, actualPrimary=1024, expectedBackups=1024.00, actualBackups=0, warningThreshold=50.00%]
[2018-10-22 17:08:30,797][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][CacheAffinitySharedManager] Affinity initialization for started caches performed in 2 ms.
[2018-10-22 17:08:30,797][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Successfully activated caches [nodeId=3f392b21-9ce6-4ce5-811e-c66796800000, client=false, topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
[2018-10-22 17:08:30,798][INFO ][sys-#159%persistence.PersistenceBasicCompatibilityTest0%][GridAffinityAssignmentCache] Local node affinity assignment distribution is not ideal [cache=PersistenceBasicCompatibilityTest, expectedPrimary=1024.00, actualPrimary=1024, expectedBackups=1024.00, actualBackups=0, warningThreshold=50.00%]
[2018-10-22 17:08:30,799][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Finished waiting for partition release future [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL]
[2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Read checkpoint status [startMarker=null, endMarker=null]
[2018-10-22 17:08:30,800][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCacheDatabaseSharedManager] Applying lost cache updates since last checkpoint record [lastMarked=FileWALPointer [idx=0, fileOff=0, len=0], lastCheckpointId=00000000-0000-0000-0000-000000000000]
[2018-10-22 17:08:30,807][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Failed to reinitialize local partitions (rebalancing will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=ChangeGlobalStateMessage [id=4bc2a1c9661-142c551e-2962-403a-b864-0281692574ee, reqId=4ca70fa9-7a54-411a-8ad5-f4633e40edd4, initiatingNodeId=3f392b21-9ce6-4ce5-811e-c66796800000, activate=true, baselineTopology=BaselineTopology [id=0, branchingHash=-416176595, branchingType='New BaselineTopology', baselineNodes=[81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332]], forceChangeBaselineTopology=false, timestamp=1540217310624], affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT]
class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process.

	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:335)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378)
	... 15 more
[2018-10-22 17:08:30,814][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Finish exchange future [startVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], resVer=null, err=class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]
[2018-10-22 17:08:30,815][INFO ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridDhtPartitionsExchangeFuture] Completed partition exchange [localNode=3f392b21-9ce6-4ce5-811e-c66796800000, exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], done=true], topVer=null, durationFromInit=187]
[2018-10-22 17:08:30,819][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][GridCachePartitionExchangeManager] Failed to wait for completion of partition map exchange (preloading will not start): GridDhtPartitionsExchangeFuture [firstDiscoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], crd=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], exchId=GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode [id=3f392b21-9ce6-4ce5-811e-c66796800000, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1540217301530, loc=true, ver=2.7.0#19700101-sha1:00000000, isClient=false], topVer=1, nodeId8=3f392b21, msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1540217310618]DiscoveryCustomEvent [customMsg=null, affTopVer=AffinityTopologyVersion [topVer=1, minorTopVer=1], super=], nodeId=3f392b21, evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=false, hash=364933128], init=false, lastVer=null, partReleaseFut=AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]AffinityTopologyVersion [topVer=1, minorTopVer=1]PartitionReleaseFuture [topVer=, futures=[ExplicitLockReleaseFuture [topVer=, futures=[]], AtomicUpdateReleaseFuture [topVer=, futures=[]], DataStreamerReleaseFuture [topVer=, futures=[]], LocalTxReleaseFuture [topVer=, futures=[]], AllTxReleaseFuture [topVer=, futures=[RemoteTxReleaseFuture [topVer=, futures=[]]]]]], exchActions=null, affChangeMsg=null, initTs=1540217310618, centralizedAff=false, forceAffReassignment=true, exchangeLocE=null, cacheChangeFailureMsgSent=false, done=true, state=CRD, registerCachesFuture=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1024540699], partitionsSent=false, partitionsReceived=false, delayedLatestMsg=null, afterLsnrCompleteFut=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=null, hash=1881071890], evtLatch=0, remaining=HashSet [], super=GridFutureAdapter [ignoreInterrupts=false, state=DONE, res=class o.a.i.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, hash=1188696354]]
class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process.

	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:335)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378)
	... 15 more
[2018-10-22 17:08:30,823][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][IgniteTestResources] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]]
class org.apache.ignite.IgniteCheckedException: Failed to initialize WAL segment: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:412)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:3113)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3195)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3087)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.<init>(FileWriteAheadLogManager.java:3003)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.replay(FileWriteAheadLogManager.java:925)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.applyLastUpdates(GridCacheDatabaseSharedManager.java:2289)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreState(GridCacheDatabaseSharedManager.java:1632)
	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.beforeExchange(GridCacheDatabaseSharedManager.java:1298)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.distributedExchange(GridDhtPartitionsExchangeFuture.java:1455)
	at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:808)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2746)
	at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2618)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal: The process cannot access the file because it is being used by another process.

	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:115)
	at java.nio.channels.FileChannel.open(FileChannel.java:287)
	at java.nio.channels.FileChannel.open(FileChannel.java:335)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.<init>(RandomAccessFileIO.java:43)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:42)
	at org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIOFactory.create(RandomAccessFileIOFactory.java:37)
	at org.apache.ignite.internal.processors.cache.persistence.wal.FileDescriptor.toIO(FileDescriptor.java:142)
	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:378)
	... 15 more
[2018-10-22 17:08:30,829][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] No deadlocked threads detected.
[2018-10-22 17:08:30,891][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] Thread dump at 2018/10/22 17:08:30 MSK
Thread [name="sys-#161%persistence.PersistenceBasicCompatibilityTest0%", id=208, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#160%persistence.PersistenceBasicCompatibilityTest0%", id=207, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#159%persistence.PersistenceBasicCompatibilityTest0%", id=206, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#158%persistence.PersistenceBasicCompatibilityTest0%", id=205, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#157%persistence.PersistenceBasicCompatibilityTest0%", id=204, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#156%persistence.PersistenceBasicCompatibilityTest0%", id=203, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="ttl-cleanup-worker-#155%persistence.PersistenceBasicCompatibilityTest0%", id=202, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778)
        at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:149)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#154%persistence.PersistenceBasicCompatibilityTest0%", id=201, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="wal-segment-syncer-#153%persistence.PersistenceBasicCompatibilityTest0%", id=200, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778)
        at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$WalSegmentSyncer.body(FileWriteAheadLogManager.java:3614)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%", id=199, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
        at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:89)
        at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
        at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:88)
        at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:113)
        at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.archiveSegment(FileWriteAheadLogManager.java:1913)
        at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.java:1783)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%", id=197, state=RUNNABLE, blockCnt=0, waitCnt=38]
        at sun.management.ThreadImpl.dumpThreads0(Native Method)
        at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454)
        at o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1368)
        at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:140)
        - locked o.a.i.i.processors.failure.FailureProcessor@3792ce1
        at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:116)
        at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2634)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="disco-event-worker-#150%persistence.PersistenceBasicCompatibilityTest0%", id=196, state=WAITING, blockCnt=0, waitCnt=4]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3a2d0702, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2856)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2825)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="tcp-disco-srvr-#12%persistence.PersistenceBasicCompatibilityTest0%", id=195, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at java.net.DualStackPlainSocketImpl.accept0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
        - locked java.net.SocksSocketImpl@42ba2f76
        at java.net.ServerSocket.implAccept(ServerSocket.java:545)
        at java.net.ServerSocket.accept(ServerSocket.java:513)
        at o.a.i.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5845)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at o.a.i.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5763)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="tcp-disco-msg-worker-#11%persistence.PersistenceBasicCompatibilityTest0%", id=194, state=TIMED_WAITING, blockCnt=0, waitCnt=940]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@27e68407, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
        at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
        at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7183)
        at o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%", id=193, state=WAITING, blockCnt=0, waitCnt=5]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6db2ba3e, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2660)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2705)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-data-loader-flusher-#148%persistence.PersistenceBasicCompatibilityTest0%", id=192, state=WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3ff5b59d, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:211)
        at o.a.i.i.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:109)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-3-#145%persistence.PersistenceBasicCompatibilityTest0%", id=189, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2ca7ec8a
        - locked java.util.Collections$UnmodifiableSet@3f7283a5
        - locked sun.nio.ch.WindowsSelectorImpl@47e9bd3f
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-2-#144%persistence.PersistenceBasicCompatibilityTest0%", id=188, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@5522b102
        - locked java.util.Collections$UnmodifiableSet@4672d490
        - locked sun.nio.ch.WindowsSelectorImpl@6d361850
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-1-#143%persistence.PersistenceBasicCompatibilityTest0%", id=187, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@673c508c
        - locked java.util.Collections$UnmodifiableSet@275657c8
        - locked sun.nio.ch.WindowsSelectorImpl@7750c1e6
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-0-#142%persistence.PersistenceBasicCompatibilityTest0%", id=186, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2a79a3c4
        - locked java.util.Collections$UnmodifiableSet@2f443e37
        - locked sun.nio.ch.WindowsSelectorImpl@3ab9c400
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="nio-acceptor-client-listener-#146%persistence.PersistenceBasicCompatibilityTest0%", id=190, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked sun.nio.ch.Util$3@4b176d40
        - locked java.util.Collections$UnmodifiableSet@2c179065
        - locked sun.nio.ch.WindowsSelectorImpl@22acb656
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-sharedfs-timeout-worker-#10%persistence.PersistenceBasicCompatibilityTest0%", id=185, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.lang.Object@56c1ffc7, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at o.a.i.spi.checkpoint.sharedfs.SharedFsTimeoutTask.body(SharedFsTimeoutTask.java:99)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="tcp-comm-worker-#9%persistence.PersistenceBasicCompatibilityTest0%", id=184, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@26cce532, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4287)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="grid-nio-worker-tcp-comm-3-#140%persistence.PersistenceBasicCompatibilityTest0%", id=182, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@3648cd36
        - locked java.util.Collections$UnmodifiableSet@7191d3af
        - locked sun.nio.ch.WindowsSelectorImpl@28f7359d
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-tcp-comm-2-#139%persistence.PersistenceBasicCompatibilityTest0%", id=181, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@6953b0c4
        - locked java.util.Collections$UnmodifiableSet@1902f1a5
        - locked sun.nio.ch.WindowsSelectorImpl@66714581
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-tcp-comm-1-#138%persistence.PersistenceBasicCompatibilityTest0%", id=180, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@773bdf0f
        - locked java.util.Collections$UnmodifiableSet@39a84bc
        - locked sun.nio.ch.WindowsSelectorImpl@d9bc2b3
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-tcp-comm-0-#137%persistence.PersistenceBasicCompatibilityTest0%", id=179, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@61aad3cc
        - locked java.util.Collections$UnmodifiableSet@4840bb51
        - locked sun.nio.ch.WindowsSelectorImpl@1fa86981
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="nio-acceptor-tcp-comm-#141%persistence.PersistenceBasicCompatibilityTest0%", id=183, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked sun.nio.ch.Util$3@4cf5be9d
        - locked java.util.Collections$UnmodifiableSet@7bf6207c
        - locked sun.nio.ch.WindowsSelectorImpl@36219660
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-timeout-worker-#136%persistence.PersistenceBasicCompatibilityTest0%", id=178, state=TIMED_WAITING, blockCnt=1, waitCnt=14]
    Lock [object=java.lang.Object@6a990544, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at o.a.i.i.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:269)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#135%persistence.PersistenceBasicCompatibilityTest0%", id=177, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#134%persistence.PersistenceBasicCompatibilityTest0%", id=176, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#133%persistence.PersistenceBasicCompatibilityTest0%", id=175, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#132%persistence.PersistenceBasicCompatibilityTest0%", id=174, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#131%persistence.PersistenceBasicCompatibilityTest0%", id=173, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#130%persistence.PersistenceBasicCompatibilityTest0%", id=172, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="jvm-pause-detector-worker", id=171, state=TIMED_WAITING, blockCnt=0, waitCnt=200]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.LongJVMPauseDetector$1.run(LongJVMPauseDetector.java:100)

Thread [name="data-streamer-stripe-7-#129%persistence.PersistenceBasicCompatibilityTest0%", id=170, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-6-#128%persistence.PersistenceBasicCompatibilityTest0%", id=169, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-5-#127%persistence.PersistenceBasicCompatibilityTest0%", id=168, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-4-#126%persistence.PersistenceBasicCompatibilityTest0%", id=167, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-3-#125%persistence.PersistenceBasicCompatibilityTest0%", id=166, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-2-#124%persistence.PersistenceBasicCompatibilityTest0%", id=165, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-1-#123%persistence.PersistenceBasicCompatibilityTest0%", id=164, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-0-#122%persistence.PersistenceBasicCompatibilityTest0%", id=163, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-7-#121%persistence.PersistenceBasicCompatibilityTest0%", id=162, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-6-#120%persistence.PersistenceBasicCompatibilityTest0%", id=161, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-5-#119%persistence.PersistenceBasicCompatibilityTest0%", id=160, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-4-#118%persistence.PersistenceBasicCompatibilityTest0%", id=159, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-3-#117%persistence.PersistenceBasicCompatibilityTest0%", id=158, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-2-#116%persistence.PersistenceBasicCompatibilityTest0%", id=157, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-1-#115%persistence.PersistenceBasicCompatibilityTest0%", id=156, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-0-#114%persistence.PersistenceBasicCompatibilityTest0%", id=155, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="test-runner-#113%persistence.PersistenceBasicCompatibilityTest%", id=152, state=WAITING, blockCnt=2, waitCnt=7]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
        at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
        at o.a.i.i.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:312)
        at o.a.i.i.IgniteKernal.active(IgniteKernal.java:3597)
        at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:121)
        at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:139)
        at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.testNodeStartByOldVersionPersistenceData_2_3(PersistenceBasicCompatibilityTest.java:98)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at junit.framework.TestCase.runTest(TestCase.java:176)
        at o.a.i.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2176)
        at o.a.i.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:142)
        at o.a.i.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2091)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="Thread-11", id=57, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@4fa859e3
Thread [name="Thread-10", id=56, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@7ffb57da
Thread [name="Thread-9", id=55, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@5c2099e1
Thread [name="Thread-8", id=54, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@45efd3fb
Thread [name="Thread-7", id=53, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@27642a4
Thread [name="Thread-6", id=52, state=RUNNABLE, blockCnt=2, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@2942d82
Thread [name="Thread-5", id=51, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@1b31ca3
Thread [name="Thread-4", id=50, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@66a42f4e
Thread [name="Thread-3", id=49, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="ignite-clock-for-tests", id=12, state=TIMED_WAITING, blockCnt=0, waitCnt=7288]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.util.GridTestClockTimer.run(GridTestClockTimer.java:50)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="Monitor Ctrl-Break", id=6, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        - locked java.io.InputStreamReader@41466b1f
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        - locked java.io.InputStreamReader@41466b1f
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:64)

Thread [name="Attach Listener", id=5, state=RUNNABLE, blockCnt=0, waitCnt=0]

Thread [name="Signal Dispatcher", id=4, state=RUNNABLE, blockCnt=0, waitCnt=0]

Thread [name="Finalizer", id=3, state=WAITING, blockCnt=60, waitCnt=5]
    Lock [object=java.lang.ref.ReferenceQueue$Lock@7b6e77c1, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

Thread [name="Reference Handler", id=2, state=WAITING, blockCnt=5, waitCnt=4]
    Lock [object=java.lang.ref.Reference$Lock@59ffeecd, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

Thread [name="main", id=1, state=TIMED_WAITING, blockCnt=2, waitCnt=3]
    Lock [object=o.a.i.thread.IgniteThread@1839cd81, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at java.lang.Thread.join(Thread.java:1260)
        at o.a.i.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2103)
        at junit.framework.TestCase.runBare(TestCase.java:141)
        at junit.framework.TestResult$1.protect(TestResult.java:122)
        at junit.framework.TestResult.runProtected(TestResult.java:142)
        at junit.framework.TestResult.run(TestResult.java:125)
        at junit.framework.TestCase.run(TestCase.java:129)
        at junit.framework.TestSuite.runTest(TestSuite.java:255)
        at junit.framework.TestSuite.run(TestSuite.java:250)
        at junit.framework.TestSuite.runTest(TestSuite.java:255)
        at junit.framework.TestSuite.run(TestSuite.java:250)
        at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
        at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
        at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
        at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
        at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)



[2018-10-22 17:08:30,902][ERROR][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][IgniteTestResources] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=EmptySet []]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: GridWorker [name=partition-exchanger, igniteInstanceName=persistence.PersistenceBasicCompatibilityTest0, finished=true, heartbeatTs=1540217310805]]]
class org.apache.ignite.IgniteException: GridWorker [name=partition-exchanger, igniteInstanceName=persistence.PersistenceBasicCompatibilityTest0, finished=true, heartbeatTs=1540217310805]
	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
	at org.apache.ignite.internal.worker.WorkersRegistry.onStopped(WorkersRegistry.java:169)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:153)
	at java.lang.Thread.run(Thread.java:748)
[2018-10-22 17:08:30,903][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] No deadlocked threads detected.
[2018-10-22 17:08:30,916][WARN ][exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%][FailureProcessor] Thread dump at 2018/10/22 17:08:30 MSK
Thread [name="sys-#161%persistence.PersistenceBasicCompatibilityTest0%", id=208, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#160%persistence.PersistenceBasicCompatibilityTest0%", id=207, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#159%persistence.PersistenceBasicCompatibilityTest0%", id=206, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#158%persistence.PersistenceBasicCompatibilityTest0%", id=205, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#157%persistence.PersistenceBasicCompatibilityTest0%", id=204, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#156%persistence.PersistenceBasicCompatibilityTest0%", id=203, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="ttl-cleanup-worker-#155%persistence.PersistenceBasicCompatibilityTest0%", id=202, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778)
        at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:149)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-#154%persistence.PersistenceBasicCompatibilityTest0%", id=201, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ec7abc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="wal-segment-syncer-#153%persistence.PersistenceBasicCompatibilityTest0%", id=200, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.util.IgniteUtils.sleep(IgniteUtils.java:7778)
        at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$WalSegmentSyncer.body(FileWriteAheadLogManager.java:3614)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%", id=199, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.FileDispatcherImpl.force0(Native Method)
        at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:89)
        at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:388)
        at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:88)
        at o.a.i.i.processors.cache.persistence.file.RandomAccessFileIO.force(RandomAccessFileIO.java:113)
        at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.archiveSegment(FileWriteAheadLogManager.java:1913)
        at o.a.i.i.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.java:1783)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="exchange-worker-#151%persistence.PersistenceBasicCompatibilityTest0%", id=197, state=RUNNABLE, blockCnt=0, waitCnt=38]
        at sun.management.ThreadImpl.dumpThreads0(Native Method)
        at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454)
        at o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1368)
        at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:140)
        - locked o.a.i.i.processors.failure.FailureProcessor@3792ce1
        at o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:116)
        at o.a.i.i.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1829)
        at o.a.i.i.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
        at o.a.i.i.worker.WorkersRegistry.onStopped(WorkersRegistry.java:169)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:153)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="disco-event-worker-#150%persistence.PersistenceBasicCompatibilityTest0%", id=196, state=WAITING, blockCnt=0, waitCnt=4]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3a2d0702, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body0(GridDiscoveryManager.java:2856)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryWorker.body(GridDiscoveryManager.java:2825)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="tcp-disco-srvr-#12%persistence.PersistenceBasicCompatibilityTest0%", id=195, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at java.net.DualStackPlainSocketImpl.accept0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketAccept(DualStackPlainSocketImpl.java:131)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:199)
        - locked java.net.SocksSocketImpl@42ba2f76
        at java.net.ServerSocket.implAccept(ServerSocket.java:545)
        at java.net.ServerSocket.accept(ServerSocket.java:513)
        at o.a.i.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:5845)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at o.a.i.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:5763)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="tcp-disco-msg-worker-#11%persistence.PersistenceBasicCompatibilityTest0%", id=194, state=TIMED_WAITING, blockCnt=0, waitCnt=944]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@27e68407, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingDeque.pollFirst(LinkedBlockingDeque.java:522)
        at java.util.concurrent.LinkedBlockingDeque.poll(LinkedBlockingDeque.java:684)
        at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7183)
        at o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="disco-notifier-worker-#149%persistence.PersistenceBasicCompatibilityTest0%", id=193, state=WAITING, blockCnt=0, waitCnt=5]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6db2ba3e, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body0(GridDiscoveryManager.java:2660)
        at o.a.i.i.managers.discovery.GridDiscoveryManager$DiscoveryMessageNotifierWorker.body(GridDiscoveryManager.java:2705)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-data-loader-flusher-#148%persistence.PersistenceBasicCompatibilityTest0%", id=192, state=WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3ff5b59d, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:211)
        at o.a.i.i.processors.datastreamer.DataStreamProcessor$2.body(DataStreamProcessor.java:109)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-3-#145%persistence.PersistenceBasicCompatibilityTest0%", id=189, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2ca7ec8a
        - locked java.util.Collections$UnmodifiableSet@3f7283a5
        - locked sun.nio.ch.WindowsSelectorImpl@47e9bd3f
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-2-#144%persistence.PersistenceBasicCompatibilityTest0%", id=188, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@5522b102
        - locked java.util.Collections$UnmodifiableSet@4672d490
        - locked sun.nio.ch.WindowsSelectorImpl@6d361850
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-1-#143%persistence.PersistenceBasicCompatibilityTest0%", id=187, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@673c508c
        - locked java.util.Collections$UnmodifiableSet@275657c8
        - locked sun.nio.ch.WindowsSelectorImpl@7750c1e6
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-client-listener-0-#142%persistence.PersistenceBasicCompatibilityTest0%", id=186, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@2a79a3c4
        - locked java.util.Collections$UnmodifiableSet@2f443e37
        - locked sun.nio.ch.WindowsSelectorImpl@3ab9c400
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="nio-acceptor-client-listener-#146%persistence.PersistenceBasicCompatibilityTest0%", id=190, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked sun.nio.ch.Util$3@4b176d40
        - locked java.util.Collections$UnmodifiableSet@2c179065
        - locked sun.nio.ch.WindowsSelectorImpl@22acb656
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-sharedfs-timeout-worker-#10%persistence.PersistenceBasicCompatibilityTest0%", id=185, state=TIMED_WAITING, blockCnt=0, waitCnt=3]
    Lock [object=java.lang.Object@56c1ffc7, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at o.a.i.spi.checkpoint.sharedfs.SharedFsTimeoutTask.body(SharedFsTimeoutTask.java:99)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="tcp-comm-worker-#9%persistence.PersistenceBasicCompatibilityTest0%", id=184, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@26cce532, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4287)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237)
        at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)

Thread [name="grid-nio-worker-tcp-comm-3-#140%persistence.PersistenceBasicCompatibilityTest0%", id=182, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@3648cd36
        - locked java.util.Collections$UnmodifiableSet@7191d3af
        - locked sun.nio.ch.WindowsSelectorImpl@28f7359d
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-tcp-comm-2-#139%persistence.PersistenceBasicCompatibilityTest0%", id=181, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@6953b0c4
        - locked java.util.Collections$UnmodifiableSet@1902f1a5
        - locked sun.nio.ch.WindowsSelectorImpl@66714581
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-tcp-comm-1-#138%persistence.PersistenceBasicCompatibilityTest0%", id=180, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@773bdf0f
        - locked java.util.Collections$UnmodifiableSet@39a84bc
        - locked sun.nio.ch.WindowsSelectorImpl@d9bc2b3
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-nio-worker-tcp-comm-0-#137%persistence.PersistenceBasicCompatibilityTest0%", id=179, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked o.a.i.i.util.nio.SelectedSelectionKeySet@61aad3cc
        - locked java.util.Collections$UnmodifiableSet@4840bb51
        - locked sun.nio.ch.WindowsSelectorImpl@1fa86981
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2151)
        at o.a.i.i.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="nio-acceptor-tcp-comm-#141%persistence.PersistenceBasicCompatibilityTest0%", id=183, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked sun.nio.ch.Util$3@4cf5be9d
        - locked java.util.Collections$UnmodifiableSet@7bf6207c
        - locked sun.nio.ch.WindowsSelectorImpl@36219660
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.accept(GridNioServer.java:2929)
        at o.a.i.i.util.nio.GridNioServer$GridNioAcceptWorker.body(GridNioServer.java:2877)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="grid-timeout-worker-#136%persistence.PersistenceBasicCompatibilityTest0%", id=178, state=TIMED_WAITING, blockCnt=1, waitCnt=14]
    Lock [object=java.lang.Object@6a990544, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at o.a.i.i.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:269)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#135%persistence.PersistenceBasicCompatibilityTest0%", id=177, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#134%persistence.PersistenceBasicCompatibilityTest0%", id=176, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#133%persistence.PersistenceBasicCompatibilityTest0%", id=175, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#132%persistence.PersistenceBasicCompatibilityTest0%", id=174, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#131%persistence.PersistenceBasicCompatibilityTest0%", id=173, state=TIMED_WAITING, blockCnt=0, waitCnt=2]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="pub-#130%persistence.PersistenceBasicCompatibilityTest0%", id=172, state=TIMED_WAITING, blockCnt=0, waitCnt=1]
    Lock [object=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@bf94fdc, ownerName=null, ownerId=-1]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="jvm-pause-detector-worker", id=171, state=TIMED_WAITING, blockCnt=0, waitCnt=201]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.LongJVMPauseDetector$1.run(LongJVMPauseDetector.java:100)

Thread [name="data-streamer-stripe-7-#129%persistence.PersistenceBasicCompatibilityTest0%", id=170, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-6-#128%persistence.PersistenceBasicCompatibilityTest0%", id=169, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-5-#127%persistence.PersistenceBasicCompatibilityTest0%", id=168, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-4-#126%persistence.PersistenceBasicCompatibilityTest0%", id=167, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-3-#125%persistence.PersistenceBasicCompatibilityTest0%", id=166, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-2-#124%persistence.PersistenceBasicCompatibilityTest0%", id=165, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-1-#123%persistence.PersistenceBasicCompatibilityTest0%", id=164, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="data-streamer-stripe-0-#122%persistence.PersistenceBasicCompatibilityTest0%", id=163, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-7-#121%persistence.PersistenceBasicCompatibilityTest0%", id=162, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-6-#120%persistence.PersistenceBasicCompatibilityTest0%", id=161, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-5-#119%persistence.PersistenceBasicCompatibilityTest0%", id=160, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-4-#118%persistence.PersistenceBasicCompatibilityTest0%", id=159, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-3-#117%persistence.PersistenceBasicCompatibilityTest0%", id=158, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-2-#116%persistence.PersistenceBasicCompatibilityTest0%", id=157, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-1-#115%persistence.PersistenceBasicCompatibilityTest0%", id=156, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="sys-stripe-0-#114%persistence.PersistenceBasicCompatibilityTest0%", id=155, state=WAITING, blockCnt=0, waitCnt=2]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:669)
        at o.a.i.i.util.StripedExecutor$Stripe.body(StripedExecutor.java:493)
        at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="test-runner-#113%persistence.PersistenceBasicCompatibilityTest%", id=152, state=WAITING, blockCnt=2, waitCnt=7]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
        at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
        at o.a.i.i.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:312)
        at o.a.i.i.IgniteKernal.active(IgniteKernal.java:3597)
        at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:121)
        at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.doTestStartupWithOldVersion(PersistenceBasicCompatibilityTest.java:139)
        at o.a.i.compatibility.persistence.PersistenceBasicCompatibilityTest.testNodeStartByOldVersionPersistenceData_2_3(PersistenceBasicCompatibilityTest.java:98)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at junit.framework.TestCase.runTest(TestCase.java:176)
        at o.a.i.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2176)
        at o.a.i.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:142)
        at o.a.i.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2091)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="Thread-11", id=57, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@4fa859e3
Thread [name="Thread-10", id=56, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@7ffb57da
Thread [name="Thread-9", id=55, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@5c2099e1
Thread [name="Thread-8", id=54, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@45efd3fb
Thread [name="Thread-7", id=53, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@27642a4
Thread [name="Thread-6", id=52, state=RUNNABLE, blockCnt=2, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@2942d82
Thread [name="Thread-5", id=51, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@1b31ca3
Thread [name="Thread-4", id=50, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    Locked synchronizers:
        java.util.concurrent.ThreadPoolExecutor$Worker@66a42f4e
Thread [name="Thread-3", id=49, state=RUNNABLE, blockCnt=0, waitCnt=0]
        at sun.nio.ch.Iocp.getQueuedCompletionStatus(Native Method)
        at sun.nio.ch.Iocp.access$300(Iocp.java:46)
        at sun.nio.ch.Iocp$EventHandlerTask.run(Iocp.java:333)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="ignite-clock-for-tests", id=12, state=TIMED_WAITING, blockCnt=0, waitCnt=7292]
        at java.lang.Thread.sleep(Native Method)
        at o.a.i.i.util.GridTestClockTimer.run(GridTestClockTimer.java:50)
        at java.lang.Thread.run(Thread.java:748)

Thread [name="Monitor Ctrl-Break", id=6, state=RUNNABLE, blockCnt=1, waitCnt=0]
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        - locked java.io.InputStreamReader@41466b1f
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        - locked java.io.InputStreamReader@41466b1f
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at com.intellij.rt.execution.application.AppMainV2$1.run(AppMainV2.java:64)

Thread [name="Attach Listener", id=5, state=RUNNABLE, blockCnt=0, waitCnt=0]

Thread [name="Signal Dispatcher", id=4, state=RUNNABLE, blockCnt=0, waitCnt=0]

Thread [name="Finalizer", id=3, state=WAITING, blockCnt=62, waitCnt=6]
    Lock [object=java.lang.ref.ReferenceQueue$Lock@7b6e77c1, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

Thread [name="Reference Handler", id=2, state=WAITING, blockCnt=6, waitCnt=5]
    Lock [object=java.lang.ref.Reference$Lock@59ffeecd, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

Thread [name="main", id=1, state=TIMED_WAITING, blockCnt=2, waitCnt=3]
    Lock [object=o.a.i.thread.IgniteThread@1839cd81, ownerName=null, ownerId=-1]
        at java.lang.Object.wait(Native Method)
        at java.lang.Thread.join(Thread.java:1260)
        at o.a.i.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2103)
        at junit.framework.TestCase.runBare(TestCase.java:141)
        at junit.framework.TestResult$1.protect(TestResult.java:122)
        at junit.framework.TestResult.runProtected(TestResult.java:142)
        at junit.framework.TestResult.run(TestResult.java:125)
        at junit.framework.TestCase.run(TestCase.java:129)
        at junit.framework.TestSuite.runTest(TestSuite.java:255)
        at junit.framework.TestSuite.run(TestSuite.java:250)
        at junit.framework.TestSuite.runTest(TestSuite.java:255)
        at junit.framework.TestSuite.run(TestSuite.java:250)
        at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
        at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
        at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
        at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
        at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)



[2018-10-22 17:08:30,926][INFO ][wal-file-archiver%persistence.PersistenceBasicCompatibilityTest0-#152%persistence.PersistenceBasicCompatibilityTest0%][FileWriteAheadLogManager] Copied file [src=C:\IdeaProjects\apache-ignite\work\db\wal\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal, dst=C:\IdeaProjects\apache-ignite\work\db\wal\archive\node00-81e7a57f-1360-4cc1-ba2b-1ca1cf9d5332\0000000000000000.wal]

Process finished with exit code -1{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)