You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ignite.apache.org by av...@apache.org on 2019/05/18 15:17:34 UTC
[ignite] branch ignite-10078-jepsen2 created (now 7078928)
This is an automated email from the ASF dual-hosted git repository.
av pushed a change to branch ignite-10078-jepsen2
in repository https://gitbox.apache.org/repos/asf/ignite.git.
at 7078928 Squashed commit of the following:
This branch includes the following new commits:
new 7078928 Squashed commit of the following:
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[ignite] 01/01: Squashed commit of the following:
Posted by av...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
av pushed a commit to branch ignite-10078-jepsen2
in repository https://gitbox.apache.org/repos/asf/ignite.git
commit 70789285f35968a7c7cb3893439e38b11e13fd8f
Author: Anton Vinogradov <av...@apache.org>
AuthorDate: Sat May 18 18:17:10 2019 +0300
Squashed commit of the following:
commit 96cc41bf789c6f643e366bda4cc429aa2965cd5e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri May 17 19:06:45 2019 +0300
IGNITE-10078 Fix exception text.
commit 847206b9ff197a6f0df630968d0ea8ff71f1a238
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri May 17 17:39:47 2019 +0300
IGNITE-10078 Fix issue with backup updates for mixed tx-atomic groups.
commit 4261124f8f2602c6131ba49b0e29d9ea73b5bc38
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri May 17 11:50:30 2019 +0300
IGNITE-10078 Fix race in IgnitePdsPartitionFileDestroyTest.
commit f902a9464275fe92ce8615f618f779f7cc16c0f0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri May 17 11:04:27 2019 +0300
IGNITE-10078 Restore suite.
commit 028f6b544ab44c99b80430b24715456b0072946a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 19:59:01 2019 +0300
IGNITE-10078 tc run.
commit e75ac967735ada8fb86b6e1ad0f440023112374d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 19:24:18 2019 +0300
IGNITE-10078 tc run.
commit caf9b20d67b9cac0ad0b455d8530b38ad153079d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 16:01:17 2019 +0300
IGNITE-10078 Temporary disable failing test.
commit fcf4168bb0233f3a2ba170b6c6411e4be0046690
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 15:52:56 2019 +0300
IGNITE-10078 Fix evicting partition to owning assignment clearing issue.
commit e9144456d0c84693e78cd5c76be07e8bd7656eee
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 12:41:57 2019 +0300
IGNITE-10078 Allow update reording during logical recovery.
commit fd2b7873fa4cfdf3dd6a615403f38ac052fce74e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 12:26:56 2019 +0300
IGNITE-10078 Fix javadoc and codestyle issues.
commit 4a593bf33aee4fad818d14966100c9f66cfb5d54
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 12:18:34 2019 +0300
IGNITE-10078 Minor refactoring.
commit 52497ddc576e79b400d3b5f8bca6455eef979ddd
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu May 16 12:16:01 2019 +0300
IGNITE-10078 Minor refactoring.
commit ca67cc531b1cdfd47f22e949716b43ebd3ae1ca2
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed May 15 15:51:54 2019 +0300
IGNITE-10078 Fix several issues.
commit e1f5f4e33b25a6ac4fd1082749d8aae079759799
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed May 8 10:24:27 2019 +0300
IGNITE-10078 Minor codestyle fix.
commit 2c0d7a3b837cc5ce7e2bbe5d712b156dc41a431f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue May 7 10:28:38 2019 +0300
IGNITE-10078 Fix codestyle issues.
commit cda9e5090f5750b78a069b3848049560c56148b2
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon May 6 18:51:22 2019 +0300
IGNITE-10078 Disabled RollbackRecord for MVCC caches.
commit 296abfbd886b9ad4fe488a6749803ee5fc9fcf47
Merge: 70b02dec4d 7a42592843
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon May 6 18:30:03 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java
commit 70b02dec4dbb41851a976c0d7c41ab2ad065829c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon May 6 16:58:39 2019 +0300
IGNITE-10078 Fix in-memory partition clearing issue, reverted a revert of IGNITE-10799.
commit 3689f9593a7b264617e7c95a309163c9e7ef8c6c
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun May 5 17:41:49 2019 +0300
Revert "IGNITE-10799 Optimize affinity recalculation in case of node join or leave - Fixes #6242."
This reverts commit afe7933b156d51691997fefd251b76de5ea15e1a.
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/GridAffinityAssignmentCache.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheAffinitySharedManager.java
commit 83685aa98a1d67334c6d80f025e5495e6d8d9e61
Author: Alexey Scherbakov <al...@gmail.com>
Date: Tue Apr 30 13:05:02 2019 +0300
IGNITE-10078 TC check.
commit d7eeb7fc9fb273eac41b7c297688d65f3c582bbb
Author: Alexey Scherbakov <al...@gmail.com>
Date: Tue Apr 30 11:39:47 2019 +0300
IGNITE-10078 Revert to old initAffinityOnNodeJoin
commit 04114b944d82409d2dcd13f89d4dd79297b87456
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 29 18:44:57 2019 +0300
IGNITE-10078 Fix concurrent isolated updater and tx mode.
commit ec11a9e91b83beddf3a79e6de7484e04cbd1fea2
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 29 16:29:26 2019 +0300
IGNITE-10078 Cleanup.
commit 4741ad3267d76e3a7a496926fff534b508f39a5c
Merge: 4a44047cbe 2ad6c85a67
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 29 14:06:29 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV2Serializer.java
commit 4a44047cbe1f1e44d19e02a401d319799ac452ce
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Apr 28 17:46:13 2019 +0300
IGNITE-10078 Fix issue with cache group name (not properly handled in test code).
commit 15ea64f792f29db3dd88da66b1974df025e1bf2f
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Apr 28 14:55:22 2019 +0300
IGNITE-10078 Fix issue with isolated updater.
commit d05fb3f0971a62097d28ef52de1ecd41309d61d4
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sat Apr 27 15:30:50 2019 +0300
IGNITE-10078 Partial revert of affinity calculation optimization on node left.
commit 1c4a20a7276eefc5f2da6decc580bef6eb1bbfd7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 26 19:12:39 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
commit 349e8d6d6603fb6b82e3ba14b8178a7be0b70ece
Merge: 19ca96fa9c a998b25828
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 26 19:11:43 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
commit 19ca96fa9ce8c25eed0d047b8d0f8a069e58a96e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 26 18:51:43 2019 +0300
IGNITE-10078 Minor.
commit e2a44a3c989abe1db59759a0ed742f022f569a7f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 26 18:49:36 2019 +0300
IGNITE-10078 Final codestyle fix.
commit 25dc10a2b87b65fb80df80ff331893a89cd177e9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 26 17:39:42 2019 +0300
IGNITE-10078 Fix wrong late affinity calculation.
commit a31aaa18abaed5f5e590ed4c7fc2a3d205a9f4f9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 25 19:59:18 2019 +0300
IGNITE-10078 Cleanup.
commit 7d41dba297ac11cf9ced030e634e05cd3aed641d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 25 19:16:37 2019 +0300
IGNITE-10078 Cleanup.
commit cf11881fb1a5f22ca6624389ad661a3f7b9a4aa7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 25 18:21:01 2019 +0300
IGNITE-10078 Cleanup.
commit 3b265806d61a5d5b25f50c4920eeb858a196578a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 25 16:35:06 2019 +0300
IGNITE-10078 Cleanup.
commit 03dbe7cd0fdbb969b38647b43fbbca9476b2c869
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 25 14:41:53 2019 +0300
IGNITE-10078 Fix CacheGroupTest scenarios with mixed tx-atomic caches in groups.
commit 6980832cbf5dd00b964a9e2f6a87229f542a53e9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 24 18:47:32 2019 +0300
IGNITE-10078 Fix issue with isolated updater in test.
commit 6860fa3b3df0e1179ec63b55ba1ded4c79fcf61a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 24 18:01:35 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/IgnitePdsContinuousRestartTest.java
# modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java
commit be160d5edf77853de47a323b5519bc283aebc1dd
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 24 15:05:44 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/IgnitePdsContinuousRestartTest.java
# modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java
commit 05e1df5d507bd0a6af302cd463cf14de00d09a98
Merge: 4ccae674c3 c2c0b148ae
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 24 15:05:32 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/IgnitePdsContinuousRestartTest.java
# modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java
commit 4ccae674c3ad14e994d4c3600e2f9bed3224f34a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 24 14:50:00 2019 +0300
IGNITE-10078 Cleanup TODOs.
commit 6223cb2518b00defd6f47ebfef1feae7ded16fbb
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 23 20:35:40 2019 +0300
IGNITE-10078 Cleanup TODOs.
commit 3e7da5ab7867f08052676f1939c903dde35ef71e
Merge: e48ce33e4d db285122a7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 23 15:54:29 2019 +0300
Merge branch 'ignite-10078' of https://github.com/gridgain/apache-ignite into ignite-10078
commit e48ce33e4d000221aebc78ec0e053f3c096f00c6
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 23 15:53:41 2019 +0300
IGNITE-10078 Implemented "old mode" for atomic or mixed cache groups.
commit db285122a76c86f9f0a171ad64728473ead69468
Author: Alexey Scherbakov <al...@gmail.com>
Date: Mon Apr 22 23:34:03 2019 +0300
IGNITE-10078 Fix NPE in exchange future.
commit 3cc00ab8f6b77982c3da8c82f20f014250a0929c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 22 17:58:01 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgnitePdsTestSuite3.java
commit 9062305f0ed4860904d5c4a6a24dd181e8af8dc1
Merge: 60fc9b26d3 d6827886a4
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 22 17:57:48 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgnitePdsTestSuite3.java
commit 60fc9b26d3dfd798611094bc67d2047a6aa5da23
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Apr 21 18:56:16 2019 +0300
IGNITE-10078 Fix issue with re-rebalance of moving partition on coordinator.
commit 7b2b7dbf074841090bebb9dc48f5b02121172fad
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 19 19:30:05 2019 +0300
IGNITE-10078 Proper fix for clearing partition + full rebalance.
commit 165a4d17ec97b92f90ea4a19a89108bc1589b1e3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 19:50:14 2019 +0300
IGNITE-10078 Cleanup - wip.
commit b8a97b78a2ab81eeb8a5136b58bb31d99d0b91e8
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 18:48:57 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheAffinitySharedManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/DefaultFreeList.java
commit 030b4606a938d41151303878a71396ef0f5eb38c
Merge: 204202826e 280894315f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 18:44:15 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheAffinitySharedManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/DefaultFreeList.java
commit 204202826ee5a7cc261d8b8107b51f26dc0e7ea3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 17:51:14 2019 +0300
IGNITE-10078 Minor style.
commit 13a89dd7aa3c472cef427e84bc2c63ceb5401865
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 17:23:58 2019 +0300
IGNITE-10078 More logging for flaky test CacheRentingStateRepair.
commit 60da22ae7ca84b0e83fc7d235d2a8bd07cb92292
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 17:21:26 2019 +0300
IGNITE-10078 More logging for flaky test CacheRentingStateRepair.
commit 069842550d674e7fa28938b4f3f61389b174ffd2
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 13:17:41 2019 +0300
IGNITE-10078 Extending expected exceptions list in continuous restarts test.
commit f779e293dc687f50e0db4645a7c7f9ba6d234907
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 18 12:01:44 2019 +0300
IGNITE-10078 Extending expected exceptions list in continuous restarts test.
commit a5609b1585fa6efec058736b1c489776532ad229
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 17 20:20:30 2019 +0300
IGNITE-10078 Ignore flaky test.
commit 88429164db2405d920418390fe88b358654bdbc1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 17 20:15:55 2019 +0300
IGNITE-10078 Fix data store init on activation.
commit 41c0ef30f54d7378a1dccef5b94ff2b89f847715
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 15 20:02:12 2019 +0300
IGNITE-10078 In-place update fix, improved logging for rebalance.
commit d7686fee1b8285b89cacbd561c53721679568984
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Apr 14 21:54:32 2019 +0300
IGNITE-10078 Disabled debugging counter for TC run.
commit 31df1cbc6890a94e82796dc3558693caef0ab665
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Apr 14 19:31:03 2019 +0300
IGNITE-10078 Valid late assignment switch.
commit cbd4e0cfc0b00858412f97962998e3a22e64d6e6
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 12 19:07:22 2019 +0300
IGNITE-10078 minor renaming.
commit 27c5c8e4904430a8733fa0e0e750ed23b8c9f691
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 12 18:14:17 2019 +0300
IGNITE-10078 Fix for force clearing partition if was in moving state before rebalance.
commit 40e88354d64aad4f44e76faf5d3d0be4278efd14
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 12 15:30:50 2019 +0300
IGNITE-10078 Desync on full rebalance reproducer.
commit b7a9dd115711b15fb04a8cf2e6859aeab246d105
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 11 15:07:33 2019 +0300
IGNITE-10078 Additional logging.
commit 4a4979c57e2ac876e455c1742ed86e2aa1512bf5
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 9 19:20:13 2019 +0300
IGNITE-10078 Additional logging.
commit 9ca272222ebc80f87b08c9527db42a2b6352813b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 9 18:43:40 2019 +0300
IGNITE-10078 Desync fix test 3.
commit a21cce2b2922b71c8cc559ee7be31c3b82ea9636
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 9 18:21:42 2019 +0300
IGNITE-10078 Desync fix test 2.
commit aea2d5cdf02adc11ce7870b7d2515e8a10330e5a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 9 15:49:49 2019 +0300
IGNITE-10078 Desync fix test.
commit a27761d262f840890cd01b57ff16e537596fd114
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 8 19:34:51 2019 +0300
IGNITE-10078 Reproducer for desync + hist rebalance issue.
commit 048b0a68583ccc1de94afa6f7961bcbd9ddb14b6
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 8 18:32:22 2019 +0300
IGNITE-10078 Fixed concurrency issue.
commit 109ef03f46b5075612b789afe4ab3a1e6ee675f8
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 5 18:01:58 2019 +0300
IGNITE-10078 Cleaned up tests.
commit d7a48d6873ab3075d1bafcd1699512a9aabd1cb3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 5 16:30:29 2019 +0300
IGNITE-10078 Disable removal from deferred queue on rebalance by ttl.
commit ee3306749d993d7534ba2abb2f29a06cf48ca219
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 5 14:18:26 2019 +0300
IGNITE-10078 Next fix try.
commit 29565048837a952928e7a3e4796161677dda2cdc
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 5 14:16:16 2019 +0300
IGNITE-10078 Next fix try.
commit 8ee04285948e3d3f1b9e913aa41c7cb4b78e3ada
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 5 13:56:45 2019 +0300
IGNITE-10078 Reproduced desync issue.
commit 560341ec7fc665e3bc08eebe817eb6b1e20d41df
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Apr 5 11:47:20 2019 +0300
IGNITE-10078 Test deferred queue fix.
commit b9ab5bdde2babd2e52c0d06f36d0900feda0049e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 4 19:04:18 2019 +0300
IGNITE-10078 WAL reader.
commit a55854aa6bda5bbd4c0ae5bd3303c8808ae2a984
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 4 17:51:28 2019 +0300
IGNITE-10078 Added test for scenario when missed update is received after checkpoint.
commit 5bc28b8e392b7202da1cfa96fd53830a96b404c7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Apr 4 17:51:09 2019 +0300
IGNITE-10078 Added test for scenario when missed update is received after checkpoint.
commit 8019f91cf0d4619225dbeb5a20d54d29d4a74d1b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 3 20:15:52 2019 +0300
IGNITE-10078 Fix logical recovery issue with zero counter.
commit 63eff8ed8160b9498cb38e3f64125a237e3de219
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Apr 3 18:47:14 2019 +0300
IGNITE-10078 Use rollbackrecord on logical recovery to avoid unnecessary rebalancing.
commit c0e7d577149fdb65e918af6d3a33423ce7edcb9a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 2 11:54:53 2019 +0300
IGNITE-10078 Fix naming for out-of-order updates.
commit 2f85c2c91ea3726dbcfc73294189db1bd5a28742
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 2 11:44:20 2019 +0300
IGNITE-10078 Pre-create partition only if having valid state.
commit eb911681d38e9ef437428c31e6e304ea22276376
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Apr 2 10:50:24 2019 +0300
IGNITE-10078 Minor.
commit 9eeb2dcc0eb6e6e27139935bcb9e240cfb3b110d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 1 20:18:33 2019 +0300
IGNITE-10078 Added assertion for sequential partition.
commit 46a89b1dd65443d9f9f180e2438cb458a52d3577
Merge: ab14a8ec9f 963a40b7aa
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 1 14:55:09 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit ab14a8ec9f56d1b97f274f54d0dbbfae66fe7ccd
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 1 14:53:58 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
commit 196b9f2bf83b44e55cfc9f599aedf7b7cd1da25e
Merge: 4b9301c439 9a9c817f8f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 1 14:53:38 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
commit 4b9301c43919c50472f96a1b529eef66b96daeeb
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Apr 1 12:08:11 2019 +0300
IGNITE-10078 Minor code cleanup.
commit 237ffd0b1b9084ec73af395d8a072c1173fec2bf
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Mar 31 19:32:49 2019 +0300
IGNITE-10078 Partition update counter debugging.
commit c832ab48533547cb3179d3b537c7c9061b8b6642
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Mar 24 15:41:46 2019 +0300
IGNITE-10078 Fix update counter overlap issue.
commit 2a8f9dd5366efb2b6a49ef8b066e3971eb3a6d5f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 22 19:22:56 2019 +0300
IGNITE-10078 Cleanup TxPartitionCouterState* tests.
commit 473f7b7a1ea7b0a2ae19272bf482354f5f1309c5
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 22 17:56:30 2019 +0300
IGNITE-10078 Cleanup TxPartitionCouterState* tests.
commit 7021afe4655cc9458401e90475fc6920134f4f4e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 22 16:22:46 2019 +0300
IGNITE-10078 Cleanup TxPartitionCouterState* tests.
commit 6184535280a7365466691cd8831980f7c03f5df5
Author: Alexey Scherbakov <al...@gmail.com>
Date: Thu Mar 21 23:02:03 2019 +0300
IGNITE-10078 Fix wrong test.
commit 20ab7b99ff73c811a7e354e3cc63a910bdf17eb0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 21 19:09:33 2019 +0300
IGNITE-10078 Fixed test.
commit ebe1e51e286dfb71683fbbf92d7d7733a6df8b29
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 21 16:25:24 2019 +0300
IGNITE-10078 Fixed new record type and gaps shring issues.
commit 556fe9070d7f4a74cd7c08fd6dcb69e0d7787ecf
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 21 11:38:13 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java
# modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java
commit 2c876b7a0063ea3fcff80cbf16dd66b63c825713
Merge: f802c53c0c b221ab8dd1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 21 11:37:24 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java
# modules/core/src/test/java/org/apache/ignite/testframework/junits/GridAbstractTest.java
commit f802c53c0c15face1035d3616ca4ec3231dfa171
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Mar 20 19:42:08 2019 +0300
IGNITE-10078 Fixed issues in partition metastore. Removed copypaste.
commit 2400675fe8698dfc364a7efba515a599c07f3351
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Mar 19 20:52:05 2019 +0300
IGNITE-10078 Add partition metastore.
commit 9b1a06b2e24881a34052b73c6266dc61bf8f2bbb
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Mar 19 19:29:39 2019 +0300
IGNITE-10078 Add partition metastore.
commit 59aa188edf14dc51186b457afb309bf032f272b6
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Mar 19 19:13:38 2019 +0300
IGNITE-10078 Add partition metastore.
commit 55c6f7cbba1df258704772264302cd2c6da5b0b0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 18 20:03:50 2019 +0300
IGNITE-10078 Fix wrong (partially) assertion.
commit ac2cff4c742cccd0287434fc06cf02daa7ff418b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 18 14:38:30 2019 +0300
IGNITE-10078 Fix metastore default partition.
commit 725c0ea33a31b8cc87b510dc7f0ecf0e63247f68
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 18 11:27:18 2019 +0300
IGNITE-10078 Minor.
commit 7af2cef3dbae5dd7da355986c321019a836f88e4
Author: Alexey Scherbakov <al...@gmail.com>
Date: Mon Mar 18 00:32:24 2019 +0300
IGNITE-10078 Fix freelist issue.
commit 0dc46d6296a603c2bba93e6d055f60457cb750e4
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Mar 17 20:50:46 2019 +0300
IGNITE-10078 Fix freelist issue.
commit 73880630a11733cecf80b49b661576bac9e49aff
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 15 19:15:31 2019 +0300
IGNITE-10078 Fix concurrent modification issue for gaps.
commit 7880d721d8543c01cc40ff766c268133fcb02741
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 15 16:11:48 2019 +0300
IGNITE-10078 Code style.
commit 304ea22552a8f545ffbcdea1618a42f56bec9814
Merge: cff5b4e4d0 ee172d2604
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 15 15:53:13 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
commit cff5b4e4d05bbfcaff84533c6798460c9be12878
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 15 15:43:30 2019 +0300
IGNITE-10078 FreeList refactoring.
commit 70024029c8f27c76ed70d72867c3220ab9c21112
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Mar 15 15:25:47 2019 +0300
IGNITE-10078 FreeList refactoring.
commit 843c84b5994697f3f01d1eb2039680a0c0384139
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 14 20:31:54 2019 +0300
IGNITE-10078 FreeList refactoring.
commit eb6e9a303947d831419fc8cb5750b3e4a38102c6
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Mar 12 20:05:34 2019 +0300
IGNITE-10078 FreeList refactoring.
commit 1530626b8f4836a5f2b6b80a467d25f85a5c468d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Mar 12 14:19:56 2019 +0300
IGNITE-10078 FreeList refactoring.
commit 7feed34f26b467225beb4515d71d0ec97decba0b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Mar 12 14:18:05 2019 +0300
IGNITE-10078 FreeList refactoring.
commit 76d367098ec8854b2c9d0121c1d798bd8f2e3724
Author: Alexey Scherbakov <al...@gmail.com>
Date: Mon Mar 11 21:52:57 2019 +0300
IGNITE-10078 Refactoring freelist to work with any Storable.
commit b747416898445e5f502f404ba2973867872f1186
Author: Alexey Scherbakov <al...@gmail.com>
Date: Mon Mar 11 16:51:48 2019 +0300
IGNITE-10078 Added local cache tx test to ignored for MVCC.
commit 20634d470e922fa2a5268cec4be49ddf3e4bf8a9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 11 13:31:58 2019 +0300
IGNITE-10078 Javadoc cleanup.
commit 50394ae2d2b535a4f0757133e4c7508293cd956d
Merge: 7ace0b5be5 3e80ca44e4
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 11 12:11:46 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit 7ace0b5be5ee5d1026b2fe9fe7267de6ecd38a97
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 11 12:04:08 2019 +0300
IGNITE-10078 Removed local cache tx test from MVCC suite.
commit 96c492bc642fb0ff8bf2efd7fde56c68a3f152a8
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Mar 10 20:58:57 2019 +0300
IGNITE-10078 Fixed mixed local and dht tx.
commit 1e5796cfb4a8718d5d7fca3f7b050dbe99535615
Author: Alexey Scherbakov <al...@gmail.com>
Date: Fri Mar 8 20:00:10 2019 +0300
IGNITE-10078 Cleaned up tests with all owners restart in the middle of tx.
commit 29b87e76545f9a07b1ea53b01edd3e0f5bb9bf89
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 7 17:18:19 2019 +0300
IGNITE-10078 Added fail all test for history rebalance.
commit 57ff05af9f00cc53ed829329e2b02e4e09c58b38
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 7 17:02:30 2019 +0300
IGNITE-10078 Cleaned up restart all on partial commit test. Property for failing nodes on counter inconsistency.
commit 2b30ec1aaae0b6328a802d801af656ca700afc33
Merge: 8d39aab13b cfa12b19fa
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Mar 7 10:50:11 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit 8d39aab13b8ac604ea30addd8cb9b13376699f2e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Mar 5 13:17:52 2019 +0300
IGNITE-10078 Change counter generation for mvcc mode to not use reservation mode.
commit 4a57bbdf6037df2c46f8f77cb67d3d42ad4dc76e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 4 19:48:23 2019 +0300
IGNITE-10078 Cleanup.
commit 63ba1379e2edd8bc6f87b3116295326c4674dec3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 4 17:23:05 2019 +0300
IGNITE-10078 Removed unneeded code.
commit 0cd54d08722b12f9d25559c5f54d593a2434dbb0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 4 17:19:11 2019 +0300
IGNITE-10078 Removed unneeded code.
commit f21879a780f9320b4db4a75170473c84122bfee2
Merge: ed202a97cc 6008a0af5f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Mar 4 16:12:52 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit ed202a97ccc7314ed0f7ab51285ebf2634a93e67
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Mar 3 21:44:20 2019 +0300
IGNITE-10078 wip.
commit 91b5c27ff774e5eba799846fa570bf615c3a68c5
Merge: 949abb2e01 7100b05d96
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Mar 3 13:20:23 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit 949abb2e01f6154adbd56458a3e5e29361a7ea07
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Feb 28 20:37:53 2019 +0300
IGNITE-10078 removed commons-cli dep, uncommented mvcc tests.
commit b416274f1d4bce8586e157d70d2515054b7efbd9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Feb 28 20:01:37 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheSharedTtlCleanupManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/PartitionCountersNeighborcastFuture.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheLocalTxStoreExceptionSelfTest.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/IgnitePdsContinuousRestartTest.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryFailoverAbstractSelfTest.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite1.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite2.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite3.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite4.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite5.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite6.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite7.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite8.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite9.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheTestSuite9.java
commit 4f9f9db63579129c3039a9c77c1982c95b25c3f9
Merge: 82869a6569 efc6b96578
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Feb 28 19:36:44 2019 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheSharedTtlCleanupManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/PartitionCountersNeighborcastFuture.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/GridCacheLocalTxStoreExceptionSelfTest.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/IgnitePdsContinuousRestartTest.java
# modules/core/src/test/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryFailoverAbstractSelfTest.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite1.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite2.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite3.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite4.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite5.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite6.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite7.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite8.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheMvccTestSuite9.java
# modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheTestSuite9.java
commit 82869a65696545370aacb10ce053446694ec809c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Feb 28 17:30:20 2019 +0300
IGNITE-10078 wip.
commit a9a8f60163511a0bc150ee00cff05125b36eb670
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Feb 28 17:27:08 2019 +0300
IGNITE-10078 debugging memleak.
commit 86d1505501b25765cabf50f48b2793317aa181c7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Feb 28 13:39:32 2019 +0300
IGNITE-10078 debugging memleak.
commit 63575fd7e028079058b38c924bbe5baa68663c54
Author: Alexey Scherbakov <al...@gmail.com>
Date: Wed Feb 27 21:23:25 2019 +0300
IGNITE-10078 wip.
commit fd39008ae713fd024ed2f9104865442a36af3144
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Feb 27 19:03:49 2019 +0300
IGNITE-10078 debugging memleak.
commit 9dc944768108dbef274b3e10268b2be701df53c9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Feb 27 18:58:19 2019 +0300
IGNITE-10078 debugging memleak.
commit 5cc20becc77161e0f59a612721f928b909f609ee
Author: Alexey Scherbakov <al...@gmail.com>
Date: Wed Feb 27 09:28:12 2019 +0300
IGNITE-10078 wip.
commit 08f4612f2224cbc7df1b947e54b6bcd153c88759
Author: Alexey Scherbakov <al...@gmail.com>
Date: Wed Feb 27 00:51:29 2019 +0300
IGNITE-10078 wip.
commit 9e8bf896ebbb9079441ab65092c42bd016b164e1
Author: Alexey Scherbakov <al...@gmail.com>
Date: Mon Feb 25 22:52:46 2019 +0300
IGNITE-10078 wip.
commit 8273eafe35b0f93c57cee20a4bd8a36de0434860
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Feb 25 18:00:25 2019 +0300
IGNITE-10078 wip.
commit 8288ba34a8e41a13ed92f66400aaa6fb29a0fcca
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Feb 24 19:28:57 2019 +0300
IGNITE-10078 wip.
commit 20d052268c27da3492e795cfeb38000921b7c83c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Feb 20 19:30:08 2019 +0300
IGNITE-10078 wip.
commit e48cd10c562d780ce54100be26611ace644fbaa6
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Feb 17 21:34:52 2019 +0300
IGNITE-10078 wip.
commit 7f67efd960e2d5333cf0574f38d2154671c67e13
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Feb 17 18:30:44 2019 +0300
IGNITE-10078 wip.
commit 9fa1128e22be21d88cd0e600232b7aa7148b7dca
Author: Alexey Scherbakov <al...@gmail.com>
Date: Sun Feb 17 18:18:32 2019 +0300
IGNITE-10078 wip.
commit 9b6d2e67c2940579f95cd32b08cc0f75131f883d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Feb 15 19:33:43 2019 +0300
IGNITE-10078 wip.
commit 6ef3ddfe7291aa301090f5466ee44a7e1c475769
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Feb 14 16:41:30 2019 +0300
IGNITE-10078 wip.
commit aedadfbddae26807b3c8710c75bab3f7f5c52f1e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Feb 12 16:00:26 2019 +0300
IGNITE-10078 wip.
commit 5b46f2cd69ed32b12f6745eca46ace1940b82436
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Feb 12 15:54:43 2019 +0300
IGNITE-10078 wip.
commit 3d656cb3d7bcd7bd806c100ace10964585505376
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Feb 11 15:58:09 2019 +0300
IGNITE-10078 wip.
commit f3a9ae136df9c3867887e77da2630a7c88b74340
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Jan 30 19:37:34 2019 +0300
IGNITE-10078 wip.
commit 51b420a3493a39ed1cc890e423f61915a404b9a5
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Jan 28 20:07:52 2019 +0300
IGNITE-10078 wip.
commit d35ba43b4dc18cde5bfb5891b59dd55b38814529
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Jan 25 19:35:43 2019 +0300
IGNITE-10078 wip.
commit 2435f3855692c7e359f752ad6ff734fb3a3eb9c3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Jan 25 12:05:08 2019 +0300
IGNITE-10078 Implement read only counters to avoid unnecessary storage init.
commit 1eab30f08e14bc0c6e519dbd9b1efe97173dd2f1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Jan 23 19:27:35 2019 +0300
IGNITE-10078 wip.
commit e4ac96856ae975856acd5a67761ddd2ebc20ab56
Merge: b56a1b4422 15cefeabe1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Jan 23 19:27:22 2019 +0300
IGNITE-10078 wip.
commit b56a1b442232358e1754e5ade92bbf4fc2e7ff22
Author: ascherbakoff <al...@gmail.com>
Date: Sun Jan 20 20:33:03 2019 +0300
IGNITE-10078 wip.
commit 1aaab90c9329910e906eeb3e01eb23381d77adde
Author: ascherbakoff <al...@gmail.com>
Date: Sun Jan 20 17:35:03 2019 +0300
IGNITE-10078 wip.
commit 7bde03c5a98f229277a98bad76a3bcb95fcb0530
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Jan 14 19:56:15 2019 +0300
IGNITE-10078 wip.
commit e5da1e699595089b9a999ecde134a570ecc50637
Author: ascherbakoff <al...@gmail.com>
Date: Sun Jan 13 21:30:03 2019 +0300
IGNITE-10078 wip.
commit 18cb5c844aef1230892a42fc034a09546686f96d
Author: ascherbakoff <al...@gmail.com>
Date: Sun Jan 13 20:11:27 2019 +0300
IGNITE-10078 wip.
commit f2b8509559118fb7ba4d1e9b7fc5d8c069b0e578
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Jan 10 15:12:33 2019 +0300
IGNITE-10078 wip.
commit 6858d29c24610c4d318ce698248ac98aa52034d9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Jan 9 20:11:53 2019 +0300
IGNITE-10078 wip.
commit 06922c86c9d75a85780d3755dacb6a8ebb526c73
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Jan 9 18:19:24 2019 +0300
IGNITE-10078 wip.
commit bd658cdc017d56d0954b90966ccb6577753b7131
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Jan 9 17:58:31 2019 +0300
IGNITE-10078 wip.
commit a6841c4f59a76475b9fafcb75bc5552a0da5e117
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Jan 9 17:41:11 2019 +0300
IGNITE-10078 wip.
commit deb7c1a170c1ed7365523cb42c2c8781d2cf8a4e
Author: ascherbakoff <al...@gmail.com>
Date: Tue Jan 8 21:30:29 2019 +0300
IGNITE-10078 wip.
commit 9375a0c1147b69cf3e5eb959248d9a101a89448e
Author: ascherbakoff <al...@gmail.com>
Date: Mon Jan 7 21:29:29 2019 +0300
IGNITE-10078 wip.
commit dc25d341d50f4dbc49f0afa9f409732d48a96393
Author: ascherbakoff <al...@gmail.com>
Date: Mon Jan 7 21:27:17 2019 +0300
IGNITE-10078 wip.
commit 4aedc9f113e4b1ce81cc22b4812dc343022dd7e8
Author: ascherbakoff <al...@gmail.com>
Date: Sat Jan 5 22:07:32 2019 +0300
IGNITE-10078 wip.
commit 233fa73a5c2dc148433b1c4a62c73daf1643e175
Author: ascherbakoff <al...@gmail.com>
Date: Sat Jan 5 22:07:18 2019 +0300
IGNITE-10078 wip.
commit d349054573203b696f7194fd5d60d20c6136219e
Author: ascherbakoff <al...@gmail.com>
Date: Sat Jan 5 16:45:51 2019 +0300
IGNITE-10078 wip.
commit b4ee593ae7a63b4c2384cca8be6445c402a2c88a
Author: ascherbakoff <al...@gmail.com>
Date: Fri Jan 4 20:32:57 2019 +0300
IGNITE-10078 wip.
commit c0ac8df87163626e73b5b0a9cc7f9bb81ffefd62
Author: ascherbakoff <al...@gmail.com>
Date: Fri Jan 4 20:30:52 2019 +0300
IGNITE-10078 wip.
commit 6e31e7379e9597ace892b37c1e9db06be37369db
Merge: 30393a15f6 bd6bc433bc
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Jan 4 20:20:59 2019 +0300
IGNITE-10078 wip.
commit 30393a15f6e848096490ca50de75ab0ad7f60da8
Author: ascherbakoff <al...@gmail.com>
Date: Fri Jan 4 19:57:04 2019 +0300
IGNITE-10078 wip.
commit 78b976ba29a54322fb77978dd837d1235f9f956c
Author: ascherbakoff <al...@gmail.com>
Date: Fri Jan 4 19:55:34 2019 +0300
IGNITE-10078 wip.
commit b71bb041e634da4f5498b483b6cf0b616a310f16
Author: ascherbakoff <al...@gmail.com>
Date: Wed Jan 2 17:16:37 2019 +0300
IGNITE-10078 wip.
commit 4e395780f2e930832ddb6af256bd9960ba34e56e
Author: ascherbakoff <al...@gmail.com>
Date: Fri Dec 28 16:59:46 2018 +0300
IGNITE-10078 wip.
commit 7f57adc354054d8a176d6abdfa191472d86d23da
Author: ascherbakoff <al...@gmail.com>
Date: Fri Dec 28 13:26:28 2018 +0300
IGNITE-10078 wip.
commit 78b7b21544ae18dffcbd28e11d870fb4e87e8968
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 19:04:07 2018 +0300
IGNITE-10078 wip.
commit 458b9afdc28e318f6f1e2d84a482e6db56f89e0c
Merge: 806f33ea32 ca71fe9ba1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 19:03:22 2018 +0300
IGNITE-10078 wip.
commit 806f33ea327669874bfc55a1c3ab8b2f336df05e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 18:23:46 2018 +0300
IGNITE-10078 wip.
commit 0f4ea2945da15b8f05fd8c552ff49ff54bc49083
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 18:13:50 2018 +0300
IGNITE-10078 wip.
commit f6408cb2335bc7a59237c38f405ea7fa18147ea1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 17:33:41 2018 +0300
IGNITE-10078 wip.
commit 804f15318e3c8843125b1c666559fea217df958b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 17:11:22 2018 +0300
IGNITE-10078 wip.
commit bc90e89f9f02116cb481b18067f77cfb1b667333
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 17:06:57 2018 +0300
IGNITE-10078 wip.
commit 70e055bc445e21ecade2674ff3f6afe0778e42f7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 16:27:54 2018 +0300
IGNITE-10078 wip.
commit 5c43d5b85676b9d2e9773aa660fc026b69b76746
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 27 15:50:08 2018 +0300
IGNITE-10078 wip.
commit 77b244819fcd3d60089dcca1919e79a12ade1e2e
Author: ascherbakoff <al...@gmail.com>
Date: Wed Dec 26 23:18:06 2018 +0300
IGNITE-10078 wip.
commit b7aadbe362ff10e12fa45366a95450eb6dfb5139
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 20:58:34 2018 +0300
IGNITE-10078 wip.
commit 5d348022d338643c3fb083f98fe2a3f3e8dd9902
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 20:47:02 2018 +0300
IGNITE-10078 wip.
commit 37056a266068007833ede53dddaad133a5ceda52
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 20:24:36 2018 +0300
IGNITE-10078 wip.
commit a4f05d7c0fd2380694358d7e74ddb86ccc5dafec
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 18:07:28 2018 +0300
IGNITE-10078 wip.
commit ba12e3eea5eba72ad934b8ea9c29c7d9cd620be1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 17:23:32 2018 +0300
IGNITE-10078 wip.
commit 1c2fa92b3038b151e504d22884d53d54c590dc85
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 17:05:27 2018 +0300
IGNITE-10078 wip.
commit 1f5f06922feaa8719e227df9b7258511e21ddcb8
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 13:38:29 2018 +0300
IGNITE-10078 wip.
commit cad6c2bf2180c528486333f0dc4538b3b4f9ff13
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 26 11:45:58 2018 +0300
IGNITE-10078 wip.
commit ba6332c328ea9fe30781e6a5cc1115380a71857b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 25 20:44:13 2018 +0300
IGNITE-10078 wip.
commit 897a527e79bda8125c777d8fda9a595074c350eb
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 25 20:40:17 2018 +0300
IGNITE-10078 wip.
commit ef371035ba5ff530e74962846731ec7791aa20ec
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 25 19:49:26 2018 +0300
IGNITE-10078 wip.
commit 5b24857533ec3fa4b8f6e767cc148e964f5cfe07
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 25 19:46:40 2018 +0300
IGNITE-10078 wip.
commit 158c1649562469257d7b32c7854d88cfbae46dcc
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 25 17:15:54 2018 +0300
IGNITE-10078 wip.
commit 1960c4b2e88069210fc2acbb8d49abe46c4328a7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 25 17:03:40 2018 +0300
IGNITE-10078 wip.
commit ecabcf35bc124ab66b4027096724a994d3cd6b56
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 25 11:56:50 2018 +0300
IGNITE-10078 wip.
commit 0c3d2c9315b0577a878f94a8934f966ce25c6cc9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 24 18:47:13 2018 +0300
IGNITE-10078 wip.
commit d212202a3911dc0c6a99dbb1d67e437b5109452d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 24 18:42:06 2018 +0300
IGNITE-10078 wip.
commit 987b344942483bd3e82f64f5bc47cba7f8cefb52
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 24 15:08:01 2018 +0300
IGNITE-10078 wip.
commit 79dd016d0ac88ae770150033035533d205aac62b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 24 14:53:19 2018 +0300
IGNITE-10078 wip.
commit dc110f0b8f56b72317b9446b4155529fc7dbf37d
Author: ascherbakoff <al...@gmail.com>
Date: Sun Dec 23 21:25:35 2018 +0300
IGNITE-10078 wip.
commit 39f7e740216e95f4253c0796fa087ee69796748d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 21 20:50:42 2018 +0300
IGNITE-10078 wip.
commit 2b668f0d20828c729c5a3536da806d4c43758792
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 21 20:49:19 2018 +0300
IGNITE-10078 wip.
commit ef0da6871a45e313d63ed7c364ea0f21b071b72e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 21 20:48:53 2018 +0300
IGNITE-10078 wip.
commit f213e4ed5d78202aeace9bb8629595709fa6874a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 21 19:11:01 2018 +0300
IGNITE-10078 wip.
commit aaa280fa7016fb7ca63ae55d80c3857e11f09229
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 19 20:03:30 2018 +0300
IGNITE-10078 wip.
commit 2ce83a2dcc9cde0428211f85d7eaa7eac5329368
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 19 14:30:35 2018 +0300
IGNITE-10078 wip.
commit e1a19d5d830513a007266908bd24859cd0e79d9c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 18 18:53:57 2018 +0300
IGNITE-10078 wip.
commit 56c48d55c555579abc28df3257d42a36806dbe80
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 18 18:38:22 2018 +0300
IGNITE-10078 wip.
commit 8c58d38f3179645b7dc5e00f72212be5f32d3bfb
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 18 17:56:27 2018 +0300
IGNITE-10078 wip.
commit 8e19369343c50cc49826ee909d4a5bdff6de67a9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 18 17:06:03 2018 +0300
IGNITE-10078 wip.
commit 9d708ca48cdff7da4a4e04e3ef80807ae3d3005f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 18 14:34:38 2018 +0300
IGNITE-10078 wip.
commit f3072e0288036c0543dade5909810f0600f669e2
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 18 14:14:52 2018 +0300
IGNITE-10078 Enable one-phase commit.
commit c4b9664a3e6b6ab8229c1c714c16fb8c527e3eb0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 18 14:12:08 2018 +0300
IGNITE-10078 wip.
commit 161ebfa94f91469b1d945f75478661d3defc7831
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 17 19:54:15 2018 +0300
IGNITE-10078 wip.
commit 737b0b31ffff42fd8e33a85ce7b1098f66eae57f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 17 19:43:02 2018 +0300
IGNITE-10078 wip.
commit a7078dffdce93ce40411aa334cdb4c6282b168a4
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 17 18:02:09 2018 +0300
IGNITE-10078 wip.
commit 3f9f79bb7460bbc56f5c6a428d4302e814060f25
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 17 13:43:38 2018 +0300
IGNITE-10078 wip.
commit bdd8eb4ff0358a6359480449536ab6e18dd9d770
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 17 13:35:01 2018 +0300
IGNITE-10078 wip.
commit 2916f2a33772fa9d6bf3cd24b8bab26848b7bfb5
Author: ascherbakoff <al...@gmail.com>
Date: Sun Dec 16 16:17:50 2018 +0300
IGNITE-10078 wip.
commit d3845bce883c741eea9058518e4cef5e6d0e64d9
Author: ascherbakoff <al...@gmail.com>
Date: Sun Dec 16 15:20:24 2018 +0300
IGNITE-10078 wip.
commit e7258b457f1b571b60bee063790cfbd2bb0d66c1
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 19:56:22 2018 +0300
IGNITE-10078 wip.
commit 1fde3b63ea500b10d5aaf2ba7fc2faa047afb034
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 14:49:35 2018 +0300
IGNITE-10078 wip.
commit f46ae88f3326b2f832e76b4e47a3fa2ad302b275
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 14:36:10 2018 +0300
IGNITE-10078 wip.
commit 61b0c6d1c34a55b8e3c8d38ad6b2e33292809937
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 14:22:39 2018 +0300
IGNITE-10078 wip.
commit 2debfa927c8f8b96dc8e31b19e7be1b131b26c5c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 13:41:06 2018 +0300
IGNITE-10078 wip.
commit 909f2a349514ee42694b5b7a9cf1c559c16765b3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 13:11:57 2018 +0300
IGNITE-10078 wip.
commit 7877cb7ff8577ad5fac6361539fa56ef3c82c09b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 13:09:43 2018 +0300
IGNITE-10078 wip.
commit 143339e0d8a985dea90bfe730f1941f3741bfe4b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 14 12:52:10 2018 +0300
IGNITE-10078 wip.
commit 4280181b45cbb19aac66e2011786a299e6af5346
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 20:11:48 2018 +0300
IGNITE-10078 wip.
commit 823eef29dcd530e5afce3f39ec2f74be41ea7b08
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 19:39:32 2018 +0300
IGNITE-10078 wip.
commit 2a3ee55e26b2b556827bc32d4854c104343d2766
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 19:18:39 2018 +0300
IGNITE-10078 wip.
commit 2976329006cd1d8b7cf5b2d6ae2ab9fe96a0e217
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 18:25:10 2018 +0300
IGNITE-10078 wip.
commit fb92f91ece0e9749b4613a413022b4c447a49e89
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 18:11:13 2018 +0300
IGNITE-10078 wip.
commit c02c54bfe64bf5daa33a31576b95a09128400e44
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 18:04:13 2018 +0300
IGNITE-10078 wip.
commit 0ffdba5b270e2fa9c4a1aab475e13223868f2a7c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 16:28:54 2018 +0300
IGNITE-10078 wip.
commit 9b9cb50a2e6d650b34a7df82cfc80837badbaae3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 16:24:02 2018 +0300
IGNITE-10078 wip.
commit 546275e360c200baf1987f06b5724abc67de8c33
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 16:15:17 2018 +0300
IGNITE-10078 wip.
commit 9cf1280d47202a71e155fc400f5e09fc59313020
Merge: 96aabe9b7b 2bf085e877
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 14:14:03 2018 +0300
IGNITE-10078 wip.
commit 96aabe9b7bdd6b04198ec958538b94085aa5739b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 13:54:59 2018 +0300
IGNITE-10078 wip.
commit d3bbeddf6eeb02346c301dfa230795e7341de28a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 13 12:55:52 2018 +0300
IGNITE-10078 wip.
commit bf7337235b8aedf05b195ecf1b39486b407eb426
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 20:22:12 2018 +0300
IGNITE-10078 wip.
commit e624147bbe21b026079a2c4da22de06be0afd226
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 20:09:40 2018 +0300
IGNITE-10078 wip.
commit 261ddd48dc4f408443f56b52df84d075765dc615
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 20:01:42 2018 +0300
IGNITE-10078 wip.
commit 4e3c20e9012e83b34414ae9c3222347ba20febc7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 19:57:09 2018 +0300
IGNITE-10078 wip.
commit 17ae851b34c800c3986ce697da30bdc070206daf
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 19:27:44 2018 +0300
IGNITE-10078 wip.
commit 7016b30f0c42e369d713f18b4bffa101ac09626e
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 19:22:25 2018 +0300
IGNITE-10078 wip.
commit c53346f53f0bc5d8fabbb9e831eaa38725a85e62
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 18:51:16 2018 +0300
IGNITE-10078 wip.
commit 6ed9fba3d42113046a9520ce4b910d645734a3f7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 18:41:10 2018 +0300
IGNITE-10078 wip.
commit 98ce00d16c4e73414a51a4ca9486d0e37969b556
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 14:52:17 2018 +0300
IGNITE-10078 wip.
commit fe9719f7e16d35ffcfd448d2622c39e46bd01503
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 14:49:00 2018 +0300
IGNITE-10078 wip.
commit db071ae2add8d687b1604aec42f1e683fc7608bc
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 14:16:26 2018 +0300
IGNITE-10078 wip.
commit 2206b0d117c9628a6e13c2785a9cc8cd4f44bcb3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 12 14:06:28 2018 +0300
IGNITE-10078 wip.
commit 67dd96faf96048b1f50bf529fc268f1115e09c68
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 11 19:27:33 2018 +0300
IGNITE-10078 wip.
commit 3951f0baefab993882e2c8d8a1f916a23d89af85
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 11 19:24:45 2018 +0300
IGNITE-10078 wip.
commit 8948804cebb8ff409ef4d700f0d52d589cb4bf16
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 11 18:53:41 2018 +0300
IGNITE-10078 wip.
commit 4d182aa68b3b424390b04d7a1b8bcea79cd0070c
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 11 18:50:35 2018 +0300
IGNITE-10078 wip.
commit beb453b7b22aec136b6d79a0acd3cc7c7899d451
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 18:40:42 2018 +0300
IGNITE-10078 wip.
commit dd05b00f0afbe0a2121091c8e20d10222c6b6f0a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 18:34:03 2018 +0300
IGNITE-10078 wip.
commit 8d3842da5e130a361e52ace7b913933e6eb0d383
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 18:29:14 2018 +0300
IGNITE-10078 wip.
commit 0b904406859e2851ec484022044255d32804e3bd
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 18:25:15 2018 +0300
IGNITE-10078 wip.
commit 7e3df8f06124745c8aedd9f5c32535c98840a1bb
Merge: 1fc29dc2e0 87510597f5
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 18:20:33 2018 +0300
IGNITE-10078 wip.
commit 1fc29dc2e0514aa20fd08dcf60536eded391faea
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 17:06:42 2018 +0300
IGNITE-10078 wip.
commit a84bd13a5aaeb203d84ef31f14fc967708c697f6
Merge: 26cbfe770b 4c48ae0ad2
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 17:04:25 2018 +0300
IGNITE-10078 wip.
commit 26cbfe770b37bf7d0f946868cc191b37baf94ef8
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 10 16:51:43 2018 +0300
IGNITE-10078 wip.
commit ada58ba6f146b6d556197fd658287173dc197629
Author: ascherbakoff <al...@gmail.com>
Date: Sun Dec 9 21:25:26 2018 +0300
IGNITE-10078 wip.
commit abb77c3d8167f6e3cbf5890cec8df9d930d802e1
Author: ascherbakoff <al...@gmail.com>
Date: Sun Dec 9 20:22:31 2018 +0300
IGNITE-10078 wip.
commit baab06840ac6ffeaf322be7ef0bedca6e6f90671
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Dec 7 20:05:45 2018 +0300
IGNITE-10078 wip.
commit 5ce148b0faa5415798a486b4d13fba8003d1623a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 6 18:47:35 2018 +0300
IGNITE-10078 wip.
commit aec4bf2336338bc995993045e3bca194606980ee
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 6 18:46:58 2018 +0300
IGNITE-10078 wip.
commit d3827452dcd0ca23b35b71f619e9beb854943618
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 6 16:45:12 2018 +0300
IGNITE-10078 wip.
commit ae248aae731c429d49b303a7c8dc3adf2cd53943
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 6 15:28:37 2018 +0300
IGNITE-10078 wip.
commit 272e08e602a0b04d966c1800efc142f44d615797
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 6 14:27:21 2018 +0300
IGNITE-10078 wip.
commit 3e747c8d12af9fed9851050c2dfa6d08439d68c0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Dec 6 13:44:13 2018 +0300
IGNITE-10078 wip.
commit a0e6a1d329911d40f68c1cd4b93f9d586878dab0
Author: ascherbakoff <al...@gmail.com>
Date: Wed Dec 5 23:25:06 2018 +0300
IGNITE-10078 wip.
commit 0a0328ee8c4faa2af1d0c31913b12a02d1d3fef5
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 5 20:10:54 2018 +0300
IGNITE-10078 wip.
commit 424855918b0f3c634b410fe4b90af0b3235a627a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 5 20:10:11 2018 +0300
IGNITE-10078 wip.
commit 39770b9ff5b273e80084306f5173f7496e9c29b0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Dec 5 19:09:15 2018 +0300
IGNITE-10078 wip.
commit 8cadd0666822420b1601f0acda3bf072296824a9
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 4 20:21:38 2018 +0300
IGNITE-10078 wip.
commit 5446ac7ef00b9521da9c586d72fd3a42d6a722b7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Dec 4 20:10:41 2018 +0300
IGNITE-10078 wip.
commit f45b1a5cefe332d538a1c505b9d1840d359da8a2
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 3 19:58:33 2018 +0300
IGNITE-10078 wip.
commit 191cf44411a230099bbe0ff97c589d3a617bae18
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 3 19:55:52 2018 +0300
IGNITE-10078 wip.
commit 68598271b519dc409680f86d5e303eaa24b91b3f
Merge: 0448651174 826cc5573f
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Dec 3 10:59:15 2018 +0300
Merge branch 'ignite-10078' of https://github.com/gridgain/apache-ignite into ignite-10078
commit 826cc5573f42daefa4914beba9d37df6d5e8faf8
Author: ascherbakoff <al...@gmail.com>
Date: Sun Dec 2 22:13:19 2018 +0300
IGNITE-10078 wip.
commit 603930c315e3b1999de0313e8ad2a4114bfa8cbe
Merge: 7a42fdbd82 2a0e354260
Author: ascherbakoff <al...@gmail.com>
Date: Fri Nov 30 12:05:35 2018 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit 0448651174c41d33a10df44add94c25cd62d6b2d
Merge: 7a42fdbd82 0e34042816
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Nov 28 16:52:10 2018 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit 7a42fdbd821c2e57c3edff16cca719e17505b68d
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Nov 22 13:21:36 2018 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionUpdateCounter.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
commit e9d52f94e9cb76f620bd9700aa1e7925f0a95842
Merge: a666157dc4 d81acdc29a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Nov 22 13:14:10 2018 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
# Conflicts:
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionUpdateCounter.java
# modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
commit a666157dc488c96fb5758daee24101c579e57b19
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Nov 21 19:10:03 2018 +0300
IGNITE-10078 wip.
commit 06aba801780def39cb00c23a3467a534f44b0b84
Author: ascherbakoff <al...@gmail.com>
Date: Wed Nov 21 01:31:03 2018 +0300
IGNITE-10078 wip.
commit 0b28c3d6da4285724116213d0b433160f62324cd
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Nov 19 17:05:32 2018 +0300
IGNITE-10078 wip.
commit 29eee5c2c2790ff474957760973dd617a0584a61
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Nov 19 17:02:32 2018 +0300
IGNITE-10078 wip.
commit aacff35491a07f93ea844e360f774a0708b155c8
Merge: 97aba597a9 355ce6fe88
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Nov 19 16:55:38 2018 +0300
IGNITE-10078 wip.
commit 97aba597a95ea4dc5e2684dfb7bc68c23e3a98d0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Nov 19 16:49:39 2018 +0300
IGNITE-10078 wip.
commit 30a8f30eef9d76d5aa72f2bc43882c7e47f5d221
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Nov 19 16:20:16 2018 +0300
IGNITE-10078 wip.
commit d176c8b30c303f19b5beeff640e7ed5b961f81c3
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Mon Nov 19 13:34:39 2018 +0300
IGNITE-10078 wip.
commit 22436a83dc0993461c08155b9e68b612dda3360a
Author: ascherbakoff <al...@gmail.com>
Date: Sun Nov 18 18:37:16 2018 +0300
IGNITE-10078 wip.
commit ef7a7857fc00bebe9fc9289ecdb687c43973e947
Author: ascherbakoff <al...@gmail.com>
Date: Sun Nov 18 17:43:15 2018 +0300
IGNITE-10078 wip.
commit e721aba13512433caf9b6420e00dfef76dce3c47
Author: ascherbakoff <al...@gmail.com>
Date: Sun Nov 18 17:31:20 2018 +0300
IGNITE-10078 wip.
commit 2efba0a10b64490ac435e0157a5eeca8c2712ecc
Author: ascherbakoff <al...@gmail.com>
Date: Sun Nov 18 17:12:14 2018 +0300
IGNITE-10078 wip.
commit e210e1cb26e774625252b1a2f11d295f80f48351
Author: ascherbakoff <al...@gmail.com>
Date: Sun Nov 18 17:06:45 2018 +0300
IGNITE-10078 wip.
commit 63d7ab771d664eafc1e5f18f7a2f51500113316b
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Fri Nov 16 18:51:59 2018 +0300
IGNITE-10078 wip.
commit 2317758268879c647d80367f44c121645713de78
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Nov 14 17:45:49 2018 +0300
IGNITE-10078 wip.
commit 64d7e2d2a235fb822b68ca21f02d85f43f88b019
Merge: defbcc167a 2439ade4d6
Author: ascherbakoff <al...@gmail.com>
Date: Mon Nov 12 22:42:43 2018 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit defbcc167aaf358991c2874b2b0d600b6248f921
Author: ascherbakoff <al...@gmail.com>
Date: Mon Nov 12 22:41:57 2018 +0300
IGNITE-10078 wip.
commit a43e5876cb81bb00fdbccc004d28e9c4ceda475a
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Tue Nov 6 10:31:38 2018 +0300
IGNITE-10078 wip.
commit beb579c83782ef4d2be91a29c43b99410e4f8d6d
Merge: 253df1059b 6f9c702cd7
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Thu Nov 1 12:51:24 2018 +0300
Merge branch 'master' of https://github.com/apache/ignite into ignite-10078
commit 253df1059b0ad0f003a711e63007df7b2e1b77c0
Author: Aleksei Scherbakov <al...@gmail.com>
Date: Wed Oct 31 18:38:07 2018 +0300
IGNITE-10078 wip.
Signed-off-by: Anton Vinogradov <av...@apache.org>
---
.../org/apache/ignite/IgniteSystemProperties.java | 7 +
.../src/main/java/org/apache/ignite/Ignition.java | 2 +-
.../apache/ignite/internal/pagemem/FullPageId.java | 6 +-
.../pagemem/wal/record/RollbackRecord.java | 115 +++
.../internal/pagemem/wal/record/WALRecord.java | 8 +-
.../delta/MetaPageUpdatePartitionDataRecord.java | 38 +-
.../delta/MetaPageUpdatePartitionDataRecordV2.java | 103 ++
.../wal/record/delta/PartitionMetaStateRecord.java | 8 +-
.../processors/affinity/AffinityAssignment.java | 2 +-
.../affinity/GridAffinityAssignmentCache.java | 14 +-
.../cache/CacheAffinitySharedManager.java | 336 ++----
.../processors/cache/CacheEntryInfoCollection.java | 19 +
.../processors/cache/CacheGroupContext.java | 13 +
.../processors/cache/GridCacheMapEntry.java | 51 +-
.../cache/GridCachePartitionExchangeManager.java | 10 +-
.../processors/cache/GridCacheProcessor.java | 32 +-
.../cache/GridCacheSharedTtlCleanupManager.java | 9 +-
.../internal/processors/cache/GridCacheUtils.java | 3 +-
.../cache/IgniteCacheOffheapManager.java | 61 +-
.../cache/IgniteCacheOffheapManagerImpl.java | 94 +-
.../cache/PartitionAtomicUpdateCounterImpl.java | 146 +++
....java => PartitionMvccTxUpdateCounterImpl.java} | 41 +-
.../PartitionTxUpdateCounterDebugWrapper.java | 201 ++++
.../cache/PartitionTxUpdateCounterImpl.java | 443 ++++++++
.../processors/cache/PartitionUpdateCounter.java | 223 ++--
.../internal/processors/cache/WalStateManager.java | 10 +-
.../GridDistributedTxRemoteAdapter.java | 3 +-
.../cache/distributed/dht/GridDhtCacheEntry.java | 28 +-
.../distributed/dht/GridDhtTxFinishFuture.java | 9 +-
.../distributed/dht/GridDhtTxFinishRequest.java | 7 -
.../distributed/dht/GridDhtTxPrepareFuture.java | 35 +-
.../dht/PartitionUpdateCountersMessage.java | 24 +
.../dht/preloader/GridDhtPartitionDemander.java | 6 +-
.../dht/preloader/GridDhtPartitionSupplier.java | 9 +
.../preloader/GridDhtPartitionsExchangeFuture.java | 75 +-
.../dht/preloader/GridDhtPreloader.java | 8 +-
.../dht/preloader/GridDhtPreloaderAssignments.java | 3 +-
.../preloader/IgniteDhtDemandedPartitionsMap.java | 3 +
.../dht/topology/GridClientPartitionTopology.java | 7 +-
.../dht/topology/GridDhtLocalPartition.java | 121 ++-
.../dht/topology/GridDhtPartitionTopology.java | 12 +-
.../dht/topology/GridDhtPartitionTopologyImpl.java | 120 ++-
.../dht/topology/PartitionsEvictManager.java | 52 +-
.../GridNearOptimisticTxPrepareFutureAdapter.java | 6 +-
.../cache/distributed/near/GridNearTxLocal.java | 15 +-
.../processors/cache/persistence/CacheDataRow.java | 8 +
.../cache/persistence/CacheDataRowAdapter.java | 2 +-
.../GridCacheDatabaseSharedManager.java | 78 +-
.../cache/persistence/GridCacheOffheapManager.java | 365 +++++--
.../processors/cache/persistence/RowStore.java | 6 +
.../processors/cache/persistence/Storable.java | 7 +
.../persistence/checkpoint/CheckpointEntry.java | 3 +-
.../persistence/freelist/AbstractFreeList.java | 37 +-
.../persistence/freelist/CacheFreeListImpl.java | 10 +-
.../cache/persistence/freelist/PagesList.java | 14 +-
.../SimpleDataRow.java} | 84 +-
.../cache/persistence/metastorage/MetaStorage.java | 133 +--
.../metastorage/MetastorageDataRow.java | 64 +-
.../metastorage/MetastorageRowStore.java | 32 +-
.../MetastoreDataPageIO.java} | 39 +-
.../PartitionMetaStorage.java} | 33 +-
.../partstorage/PartitionMetaStorageImpl.java | 140 +++
.../cache/persistence/tree/BPlusTree.java | 1 +
.../persistence/tree/io/AbstractDataPageIO.java | 1 -
.../cache/persistence/tree/io/PageIO.java | 7 +
.../persistence/tree/io/PagePartitionMetaIO.java | 34 +
.../persistence/tree/io/PagePartitionMetaIOV2.java | 51 +-
.../persistence/tree/io/SimpleDataPageIO.java | 35 +-
.../cache/persistence/tree/util/PageHandler.java | 3 +-
.../persistence/wal/FileWriteAheadLogManager.java | 19 +-
.../cache/persistence/wal/record/RecordTypes.java | 1 +
.../wal/serializer/RecordDataV1Serializer.java | 37 +-
.../wal/serializer/RecordDataV2Serializer.java | 22 +
.../CacheContinuousQueryEventBuffer.java | 14 +-
.../continuous/CacheContinuousQueryHandler.java | 6 +-
.../cache/transactions/IgniteTxEntry.java | 19 +
.../cache/transactions/IgniteTxHandler.java | 88 +-
.../cache/transactions/IgniteTxLocalAdapter.java | 34 +-
.../cache/transactions/IgniteTxManager.java | 4 +-
.../processors/cache/transactions/TxCounters.java | 25 +-
.../cluster/GridClusterStateProcessor.java | 3 +-
.../ignite/internal/stat/IoStatisticsHolder.java | 1 -
.../apache/ignite/internal/util/IgniteTree.java | 5 +-
.../apache/ignite/internal/util/IgniteUtils.java | 62 +-
modules/core/src/test/config/log4j-test.xml | 9 +
.../ignite/cache/ResetLostPartitionTest.java | 5 +-
.../internal/TestRecordingCommunicationSpi.java | 55 +-
.../processors/cache/IgniteCacheGroupsTest.java | 16 +-
.../CachePartitionLossDetectionOnNodeLeftTest.java | 115 +++
.../distributed/CacheRentingStateRepairTest.java | 25 +-
.../GridCacheRebalancingWithAsyncClearingTest.java | 12 +-
.../IgnitePdsCacheRebalancingAbstractTest.java | 4 +
.../IgnitePdsContinuousRestartTest.java | 10 +
...tePdsContinuousRestartTestWithExpiryPolicy.java | 1 +
.../persistence/IgnitePdsCorruptedStoreTest.java | 40 +-
.../IgnitePdsPartitionFilesDestroyTest.java | 8 +
.../db/IgnitePdsPartitionPreloadTest.java | 19 +-
.../persistence/db/wal/IgniteWalRebalanceTest.java | 2 +-
.../db/wal/IgniteWalSerializerVersionTest.java | 2 +-
.../db/wal/WalRecoveryTxLogicalRecordsTest.java | 162 ++-
.../transactions/PartitionUpdateCounterTest.java | 425 ++++++++
.../TxLocalDhtMixedCacheModesTest.java | 86 ++
.../TxPartitionCounterStateAbstractTest.java | 1081 ++++++++++++++++++++
...ounterStateConsistencyHistoryRebalanceTest.java | 193 ++++
.../TxPartitionCounterStateConsistencyTest.java | 490 +++++++++
...ateOnePrimaryOneBackupHistoryRebalanceTest.java | 41 +
...rtitionCounterStateOnePrimaryOneBackupTest.java | 491 +++++++++
...imaryTwoBackupsFailAllHistoryRebalanceTest.java | 100 ++
...ounterStateOnePrimaryTwoBackupsFailAllTest.java | 352 +++++++
...teOnePrimaryTwoBackupsHistoryRebalanceTest.java | 64 ++
...titionCounterStateOnePrimaryTwoBackupsTest.java | 913 +++++++++++++++++
.../TxPartitionCounterStatePutTest.java | 297 ++++++
...titionCounterStateTwoPrimaryTwoBackupsTest.java | 225 ++++
.../TxPartitionCounterStateWithFilterTest.java | 178 ++++
.../cache/transactions/TxRollbackAsyncTest.java | 2 +-
.../testframework/junits/GridAbstractTest.java | 3 +
.../ignite/testframework/junits/Repeat.java} | 51 +-
.../ignite/testframework/junits/RepeatRule.java | 67 ++
.../junits/common/GridCommonAbstractTest.java | 203 +++-
.../testsuites/IgniteCacheMvccTestSuite6.java | 4 +
.../testsuites/IgniteCacheMvccTestSuite9.java | 22 +
.../ignite/testsuites/IgniteCacheTestSuite6.java | 6 +
.../ignite/testsuites/IgniteCacheTestSuite9.java | 24 +
.../processors/cache/CacheIndexStreamerTest.java | 3 +
124 files changed, 8579 insertions(+), 1292 deletions(-)
diff --git a/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java b/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
index 800ca56..a8916b8 100644
--- a/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
+++ b/modules/core/src/main/java/org/apache/ignite/IgniteSystemProperties.java
@@ -1145,6 +1145,13 @@ public final class IgniteSystemProperties {
public static final String IGNITE_DIAGNOSTIC_WARN_LIMIT = "IGNITE_DIAGNOSTIC_WARN_LIMIT";
/**
+ * Flag to enable triggering failure handler for node if unrecoverable partition inconsistency is
+ * discovered during partition update counters exchange.
+ */
+ public static final String IGNITE_FAIL_NODE_ON_UNRECOVERABLE_PARTITION_INCONSISTENCY =
+ "IGNITE_FAIL_NODE_ON_UNRECOVERABLE_PARTITION_INCONSISTENCY";
+
+ /**
* Allow use composite _key, _val columns at the INSERT/UPDATE/MERGE statements.
*/
public static final String IGNITE_SQL_ALLOW_KEY_VAL_UPDATES = "IGNITE_SQL_ALLOW_KEY_VAL_UPDATES";
diff --git a/modules/core/src/main/java/org/apache/ignite/Ignition.java b/modules/core/src/main/java/org/apache/ignite/Ignition.java
index 53967d1..d5bd2a6 100644
--- a/modules/core/src/main/java/org/apache/ignite/Ignition.java
+++ b/modules/core/src/main/java/org/apache/ignite/Ignition.java
@@ -581,4 +581,4 @@ public class Ignition {
return TcpIgniteClient.start(cfg);
}
-}
\ No newline at end of file
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/FullPageId.java b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/FullPageId.java
index 9e24943..17c552d 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/FullPageId.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/FullPageId.java
@@ -40,10 +40,10 @@ import org.apache.ignite.internal.util.typedef.internal.U;
* <p>
* <h3>Page ID rotation</h3>
* There are scenarios when we reference one page (B) from within another page (A) by page ID. It is also
- * possible that this first page (B) is de-allocated and allocated again for a different purpose. In this
- * case we should have a mechanism to determine that page (B) cannot be used after reading it's ID in page (A).
+ * possible that this first page (B) is concurrently reused for a different purpose. In this
+ * case we should have a mechanism to determine that the reference from page (A) to page (B) is no longer valid.
* This is ensured by page ID rotation - together with page's (B) ID we should write some value that is incremented
- * each time a page is de-allocated (page ID rotation). This ID should be verified after page read and a page
+ * each time a page is reused (page ID rotation). This ID should be verified after page read and a page
* should be discarded if full ID is different.
* <p>
* Effective page ID is page ID with zeroed bits used for page ID rotation.
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RollbackRecord.java b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RollbackRecord.java
new file mode 100644
index 0000000..fd6a00e
--- /dev/null
+++ b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/RollbackRecord.java
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.pagemem.wal.record;
+
+import org.apache.ignite.internal.util.tostring.GridToStringInclude;
+import org.apache.ignite.internal.util.typedef.internal.S;
+
+/**
+ * Rollback record. Used to close gap in partition update sequence on tx rollback.
+ */
+public class RollbackRecord extends WALRecord {
+ /** Cache ID. */
+ @GridToStringInclude
+ protected int grpId;
+
+ /** Partition ID. */
+ @GridToStringInclude
+ protected int partId;
+
+ /** Rollback start. */
+ @GridToStringInclude
+ protected long start;
+
+ /** Rollback range. */
+ @GridToStringInclude
+ protected long range;
+
+ /**
+ * @param grpId Group id.
+ * @param partId Partition id.
+ * @param start Start.
+ * @param range Range.
+ */
+ public RollbackRecord(int grpId, int partId, long start, long range) {
+ this.grpId = grpId;
+ this.partId = partId;
+ this.start = start;
+ this.range = range;
+ }
+
+ /**
+ * @return Cache ID.
+ */
+ public int groupId() {
+ return grpId;
+ }
+
+ /**
+ * @return Partition ID.
+ */
+ public int partitionId() {
+ return partId;
+ }
+
+ /**
+ * @return Rollback start.
+ */
+ public long start() {
+ return start;
+ }
+
+ /**
+ * @return Rollback range.
+ */
+ public long range() {
+ return range;
+ }
+
+ /**
+ * Returns a number of overlapping update counters.
+ *
+ * @param from From counter (not inclusive).
+ * @param to To counter (inclusive).
+ */
+ public long overlap(long from, long to) {
+ long to0 = start + range;
+
+ // from lies within (start, to0]
+ if (start <= from && from < to0)
+ return Math.min(to0 - from, to - from);
+
+ // start lies within (from, to]
+ if (from <= start && start < to)
+ return Math.min(to - start, range);
+
+ return 0;
+ }
+
+
+
+ /** {@inheritDoc} */
+ @Override public RecordType type() {
+ return RecordType.ROLLBACK_TX_RECORD;
+ }
+
+ /** {@inheritDoc} */
+ @Override public String toString() {
+ return S.toString(RollbackRecord.class, this);
+ }
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java
index 5d72768..9cf2613 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/WALRecord.java
@@ -208,7 +208,13 @@ public abstract class WALRecord {
MVCC_DATA_RECORD (LOGICAL),
/** Mvcc Tx state change record. */
- MVCC_TX_RECORD (LOGICAL);
+ MVCC_TX_RECORD (LOGICAL),
+
+ /** Rollback tx record. */
+ ROLLBACK_TX_RECORD (LOGICAL),
+
+ /** */
+ PARTITION_META_PAGE_UPDATE_COUNTERS_V2 (PHYSICAL);
/**
* When you're adding a new record don't forget to choose record purpose explicitly
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/MetaPageUpdatePartitionDataRecord.java b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/MetaPageUpdatePartitionDataRecord.java
index 28294a9..3e2b67b 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/MetaPageUpdatePartitionDataRecord.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/MetaPageUpdatePartitionDataRecord.java
@@ -17,7 +17,11 @@
package org.apache.ignite.internal.pagemem.wal.record.delta;
+import java.io.DataInput;
+import java.io.IOException;
+import java.nio.ByteBuffer;
import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.internal.pagemem.PageIdUtils;
import org.apache.ignite.internal.pagemem.PageMemory;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PagePartitionMetaIO;
import org.apache.ignite.internal.util.typedef.internal.S;
@@ -57,8 +61,7 @@ public class MetaPageUpdatePartitionDataRecord extends PageDeltaRecord {
int partSize,
long cntrsPageId,
byte state,
- int allocatedIdxCandidate
- ) {
+ int allocatedIdxCandidate) {
super(grpId, pageId);
this.updateCntr = updateCntr;
@@ -70,6 +73,20 @@ public class MetaPageUpdatePartitionDataRecord extends PageDeltaRecord {
}
/**
+ * @param in Input.
+ */
+ public MetaPageUpdatePartitionDataRecord(DataInput in) throws IOException{
+ super(in.readInt(), in.readLong());
+
+ this.updateCntr = in.readLong();
+ this.globalRmvId = in.readLong();
+ this.partSize = in.readInt();
+ this.cntrsPageId = in.readLong();
+ this.state = in.readByte();
+ this.allocatedIdxCandidate = in.readInt();
+ }
+
+ /**
* @return Update counter.
*/
public long updateCounter() {
@@ -123,6 +140,21 @@ public class MetaPageUpdatePartitionDataRecord extends PageDeltaRecord {
return allocatedIdxCandidate;
}
+ /**
+ * @param buf Buffer.
+ */
+ public void toBytes(ByteBuffer buf) {
+ buf.putInt(groupId());
+ buf.putLong(pageId());
+
+ buf.putLong(updateCounter());
+ buf.putLong(globalRemoveId());
+ buf.putInt(partitionSize());
+ buf.putLong(countersPageId());
+ buf.put(state());
+ buf.putInt(allocatedIndexCandidate());
+ }
+
/** {@inheritDoc} */
@Override public RecordType type() {
return RecordType.PARTITION_META_PAGE_UPDATE_COUNTERS;
@@ -130,6 +162,6 @@ public class MetaPageUpdatePartitionDataRecord extends PageDeltaRecord {
/** {@inheritDoc} */
@Override public String toString() {
- return S.toString(MetaPageUpdatePartitionDataRecord.class, this, "super", super.toString());
+ return S.toString(MetaPageUpdatePartitionDataRecord.class, this, "partId", PageIdUtils.partId(pageId()), "super", super.toString());
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/MetaPageUpdatePartitionDataRecordV2.java b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/MetaPageUpdatePartitionDataRecordV2.java
new file mode 100644
index 0000000..ab3ccf8
--- /dev/null
+++ b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/MetaPageUpdatePartitionDataRecordV2.java
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.pagemem.wal.record.delta;
+
+import java.io.DataInput;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.internal.pagemem.PageIdUtils;
+import org.apache.ignite.internal.pagemem.PageMemory;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.PagePartitionMetaIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.PagePartitionMetaIOV2;
+import org.apache.ignite.internal.util.typedef.internal.S;
+
+/**
+ *
+ */
+public class MetaPageUpdatePartitionDataRecordV2 extends MetaPageUpdatePartitionDataRecord {
+ /** */
+ private long link;
+
+ /**
+ * @param grpId Group id.
+ * @param pageId Page id.
+ * @param updateCntr Update counter.
+ * @param globalRmvId Global remove id.
+ * @param partSize Partition size.
+ * @param cntrsPageId Cntrs page id.
+ * @param state State.
+ * @param allocatedIdxCandidate Allocated index candidate.
+ * @param link Link.
+ */
+ public MetaPageUpdatePartitionDataRecordV2(
+ int grpId,
+ long pageId,
+ long updateCntr,
+ long globalRmvId,
+ int partSize,
+ long cntrsPageId,
+ byte state,
+ int allocatedIdxCandidate,
+ long link) {
+ super(grpId, pageId, updateCntr, globalRmvId, partSize, cntrsPageId, state, allocatedIdxCandidate);
+ this.link = link;
+ }
+
+ /**
+ * @param in Input.
+ */
+ public MetaPageUpdatePartitionDataRecordV2(DataInput in) throws IOException {
+ super(in);
+
+ this.link = in.readLong();
+ }
+
+ /** {@inheritDoc} */
+ @Override public void applyDelta(PageMemory pageMem, long pageAddr) throws IgniteCheckedException {
+ super.applyDelta(pageMem, pageAddr);
+
+ PagePartitionMetaIOV2 io = (PagePartitionMetaIOV2)PagePartitionMetaIO.VERSIONS.forPage(pageAddr);
+
+ io.setGapsLink(pageAddr, link);
+ }
+
+ /**
+ *
+ */
+ public long link() {
+ return link;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void toBytes(ByteBuffer buf) {
+ super.toBytes(buf);
+
+ buf.putLong(link());
+ }
+
+ /** {@inheritDoc} */
+ @Override public RecordType type() {
+ return RecordType.PARTITION_META_PAGE_UPDATE_COUNTERS_V2;
+ }
+
+ /** {@inheritDoc} */
+ @Override public String toString() {
+ return S.toString(MetaPageUpdatePartitionDataRecordV2.class, this, "partId", PageIdUtils.partId(pageId()), "super", super.toString());
+ }
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/PartitionMetaStateRecord.java b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/PartitionMetaStateRecord.java
index e1e74c4..399ffc2 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/PartitionMetaStateRecord.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/pagemem/wal/record/delta/PartitionMetaStateRecord.java
@@ -35,12 +35,14 @@ public class PartitionMetaStateRecord extends WALRecord implements WalRecordCach
/** Partition id. */
private final int partId;
- /** Update counter. */
+ /** @deprecated Update counter. */
private final long updateCounter;
/**
* @param grpId Cache group ID.
- * @param state Page ID.
+ * @param partId Partition ID.
+ * @param state State.
+ * @param updateCounter Update counter.
*/
public PartitionMetaStateRecord(int grpId, int partId, GridDhtPartitionState state, long updateCounter) {
this.grpId = grpId;
@@ -74,7 +76,7 @@ public class PartitionMetaStateRecord extends WALRecord implements WalRecordCach
}
/**
- *
+ * @return Rollback counter.
*/
public long updateCounter() {
return updateCounter;
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/AffinityAssignment.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/AffinityAssignment.java
index b8b1089..502672c 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/AffinityAssignment.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/AffinityAssignment.java
@@ -74,7 +74,7 @@ public interface AffinityAssignment {
public Collection<UUID> getIds(int part);
/**
- * @return Nodes having parimary and backup assignments.
+ * @return Nodes having primary and backup assignments.
*/
public Set<ClusterNode> nodes();
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/GridAffinityAssignmentCache.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/GridAffinityAssignmentCache.java
index 7c2b1dc..53664b8 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/GridAffinityAssignmentCache.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/affinity/GridAffinityAssignmentCache.java
@@ -352,22 +352,22 @@ public class GridAffinityAssignmentCache {
if (hasBaseline && changedBaseline) {
recalculateBaselineAssignment(topVer, events, prevAssignment, sorted, blt);
- assignment = IdealAffinityAssignment.createWithPreservedPrimaries(
+ assignment = IdealAffinityAssignment.create(
topVer,
- baselineAssignmentWithoutOfflineNodes(topVer),
- baselineAssignment
+ sorted,
+ baselineAssignmentWithoutOfflineNodes(topVer)
);
}
else if (skipCalculation)
assignment = prevAssignment;
- else if (hasBaseline && !changedBaseline) {
+ else if (hasBaseline) {
if (baselineAssignment == null)
recalculateBaselineAssignment(topVer, events, prevAssignment, sorted, blt);
- assignment = IdealAffinityAssignment.createWithPreservedPrimaries(
+ assignment = IdealAffinityAssignment.create(
topVer,
- baselineAssignmentWithoutOfflineNodes(topVer),
- baselineAssignment
+ sorted,
+ baselineAssignmentWithoutOfflineNodes(topVer)
);
}
else {
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheAffinitySharedManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheAffinitySharedManager.java
index 9a80d8a..53471cc 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheAffinitySharedManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheAffinitySharedManager.java
@@ -18,7 +18,6 @@
package org.apache.ignite.internal.processors.cache;
import java.util.ArrayList;
-import java.util.BitSet;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
@@ -51,7 +50,6 @@ import org.apache.ignite.internal.managers.eventstorage.GridLocalEventListener;
import org.apache.ignite.internal.processors.affinity.AffinityAssignment;
import org.apache.ignite.internal.processors.affinity.AffinityTopologyVersion;
import org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache;
-import org.apache.ignite.internal.processors.affinity.IdealAffinityAssignment;
import org.apache.ignite.internal.processors.cache.distributed.dht.ClientCacheDhtTopologyFuture;
import org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAffinityAssignmentResponse;
import org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAssignmentFetchFuture;
@@ -269,7 +267,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
CacheAffinityChangeMessage msg = null;
synchronized (mux) {
- if (waitInfo == null || !waitInfo.topVer.equals(lastAffVer) )
+ if (waitInfo == null || !waitInfo.topVer.equals(lastAffVer))
return;
Map<Integer, UUID> partWait = waitInfo.waitGrps.get(checkGrpId);
@@ -324,7 +322,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
*/
public Set<Integer> waitGroups() {
synchronized (mux) {
- if (waitInfo == null || !waitInfo.topVer.equals(lastAffVer) )
+ if (waitInfo == null || !waitInfo.topVer.equals(lastAffVer))
return Collections.emptySet();
return new HashSet<>(waitInfo.waitGrps.keySet());
@@ -332,6 +330,21 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
}
/**
+ * @param grpId Group id.
+ * @param part Part.
+ * @param node Node.
+ * @param topVer Topology version.
+ */
+ public void addToWaitGroup(int grpId, int part, UUID node, AffinityTopologyVersion topVer) {
+ synchronized (mux) {
+ if (waitInfo == null)
+ waitInfo = new WaitRebalanceInfo(topVer);
+
+ waitInfo.add(grpId, part, node, null);
+ }
+ }
+
+ /**
* @param waitInfo Cache rebalance information.
* @return Message.
*/
@@ -495,6 +508,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
clientTop.fullUpdateCounters(),
Collections.<Integer>emptySet(),
null,
+ null,
null
);
}
@@ -560,7 +574,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
-1,
false);
- grp.topology().update(topVer, partMap, null, Collections.emptySet(), null, null);
+ grp.topology().update(topVer, partMap, null, Collections.emptySet(), null, null, null);
topFut.validate(grp, discoCache.allNodes());
}
@@ -791,8 +805,8 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
}
/**
- * Called during the rollback of the exchange partitions procedure
- * in order to stop the given cache even if it's not fully initialized (e.g. failed on cache init stage).
+ * Called during the rollback of the exchange partitions procedure in order to stop the given cache even if it's not
+ * fully initialized (e.g. failed on cache init stage).
*
* @param fut Exchange future.
* @param crd Coordinator flag.
@@ -803,7 +817,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
boolean crd,
final ExchangeActions exchActions
) {
- assert exchActions != null && !exchActions.empty() && exchActions.cacheStartRequests().isEmpty(): exchActions;
+ assert exchActions != null && !exchActions.empty() && exchActions.cacheStartRequests().isEmpty() : exchActions;
IgniteInternalFuture<?> res = cachesRegistry.update(exchActions);
@@ -1613,182 +1627,15 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
* @throws IgniteCheckedException If failed.
*/
public Map<Integer, CacheGroupAffinityMessage> onServerLeftWithExchangeMergeProtocol(
- final GridDhtPartitionsExchangeFuture fut) throws IgniteCheckedException
- {
- final ExchangeDiscoveryEvents evts = fut.context().events();
-
- assert fut.context().mergeExchanges();
- assert evts.hasServerLeft();
-
- if (evts.hasServerLeft() && evts.hasServerJoin())
- return onReassignmentEnforced(fut);
- else
- return onServerLeftWithExchangeMergeProtocolLightweight(fut);
- }
-
- /**
- * @param fut Current exchange future.
- * @return Computed difference with ideal affinity.
- * @throws IgniteCheckedException If failed.
- */
- private Map<Integer, CacheGroupAffinityMessage> onServerLeftWithExchangeMergeProtocolLightweight(
- final GridDhtPartitionsExchangeFuture fut) throws IgniteCheckedException
- {
+ final GridDhtPartitionsExchangeFuture fut) throws IgniteCheckedException {
final ExchangeDiscoveryEvents evts = fut.context().events();
- final AffinityTopologyVersion topVer = evts.topologyVersion();
assert fut.context().mergeExchanges();
assert evts.hasServerLeft();
- final WaitRebalanceInfo waitRebalanceInfo =
- new WaitRebalanceInfo(fut.context().events().lastServerEventVersion());
-
- final Map<Integer, Map<Integer, List<Long>>> diff = new ConcurrentHashMap<>();
-
- final Set<ClusterNode> aliveNodes = new HashSet<>(fut.context().events().discoveryCache().serverNodes());
-
- forAllRegisteredCacheGroups(new IgniteInClosureX<CacheGroupDescriptor>() {
- @Override public void applyx(CacheGroupDescriptor desc) throws IgniteCheckedException {
- AffinityTopologyVersion topVer = evts.topologyVersion();
-
- CacheGroupHolder grpHolder = getOrCreateGroupHolder(topVer, desc);
-
- IdealAffinityAssignment idealAssignment = grpHolder.affinity().calculate(topVer, evts, evts.discoveryCache());
-
- if (!grpHolder.rebalanceEnabled || fut.cacheGroupAddedOnExchange(desc.groupId(), desc.receivedFrom())) {
- grpHolder.affinity().initialize(topVer, idealAssignment.assignment());
-
- return;
- }
-
- AffinityTopologyVersion affTopVer = grpHolder.affinity().lastVersion();
-
- List<List<ClusterNode>> curAssignment = grpHolder.affinity().assignments(affTopVer);
-
- assert curAssignment != null;
-
- List<List<ClusterNode>> newAssignment = new ArrayList<>(idealAssignment.assignment());
-
- GridDhtPartitionTopology top = grpHolder.topology(fut.context().events().discoveryCache());
-
- BitSet processedPartitions = new BitSet(curAssignment.size());
-
- Map<Integer, List<Long>> cacheAffinityDiff = new HashMap<>();
-
- for (ClusterNode leftNode : evts.leftServerNodes()) {
- for (int p : idealAssignment.idealPrimaries(leftNode)) {
- List<ClusterNode> curOwners = curAssignment.get(p);
-
- if (curOwners.isEmpty())
- continue;
-
- List<ClusterNode> idealOwners = idealAssignment.assignment().get(p);
-
- List<ClusterNode> newOwners = null;
-
- if (idealOwners.isEmpty())
- newOwners = selectCurrentAliveOwners(aliveNodes, curOwners);
- else if (curOwners.get(0).equals(leftNode))
- newOwners = selectPrimaryTopologyOwnerFromIdealAssignment(
- grpHolder.aff,
- p,
- top,
- idealOwners,
- waitRebalanceInfo
- );
- else if (!curOwners.get(0).equals(idealOwners.get(0)))
- newOwners = latePrimaryAssignment(
- grpHolder.aff,
- p,
- curOwners.get(0),
- idealOwners,
- waitRebalanceInfo
- );
-
- if (newOwners != null) {
- newAssignment.set(p, newOwners);
-
- List<Long> clusterNodesAsOrder = newOwners.stream()
- .map(NODE_TO_ORDER::apply)
- .collect(Collectors.toList());
-
- cacheAffinityDiff.put(p, clusterNodesAsOrder);
-
- processedPartitions.set(p);
- }
- }
- }
-
- Set<Integer> partitionsWithChangedPrimary = grpHolder.affinity().partitionPrimariesDifferentToIdeal(affTopVer);
-
- // We need to re-check partitions for further correct late affinity assignment
- // where primary node is not as in ideal assignment.
- for (int p : partitionsWithChangedPrimary) {
- if (processedPartitions.get(p))
- continue;
-
- List<ClusterNode> curOwners = curAssignment.get(p);
-
- if (curOwners.isEmpty())
- continue;
-
- List<ClusterNode> idealOwners = idealAssignment.assignment().get(p);
-
- List<ClusterNode> newOwners = null;
-
- if (idealOwners.isEmpty())
- newOwners = selectCurrentAliveOwners(aliveNodes, curOwners);
- else if (!aliveNodes.contains(curOwners.get(0)))
- newOwners = selectPrimaryTopologyOwnerFromIdealAssignment(
- grpHolder.aff,
- p,
- top,
- idealOwners,
- waitRebalanceInfo
- );
- else if (!curOwners.get(0).equals(idealOwners.get(0)))
- // Current distribution was already not ideal. Preserve it for late affinity assignment.
- newOwners = latePrimaryAssignment(
- grpHolder.aff,
- p,
- curOwners.get(0),
- idealOwners,
- waitRebalanceInfo
- );
-
- if (newOwners != null) {
- newAssignment.set(p, newOwners);
-
- List<Long> clusterNodesAsOrder = newOwners.stream()
- .map(NODE_TO_ORDER::apply)
- .collect(Collectors.toList());
-
- cacheAffinityDiff.put(p, clusterNodesAsOrder);
- }
- }
-
- if (!cacheAffinityDiff.isEmpty())
- diff.put(grpHolder.groupId(), cacheAffinityDiff);
-
- grpHolder.affinity().initialize(topVer, newAssignment);
-
- fut.timeBag().finishLocalStage("Affinity initialization (on server left) " +
- "[grp=" + desc.cacheOrGroupName() + "]");
- }
- });
-
- synchronized (mux) {
- this.waitInfo = !waitRebalanceInfo.empty() ? waitRebalanceInfo : null;
-
- WaitRebalanceInfo info = this.waitInfo;
-
- if (log.isDebugEnabled()) {
- log.debug("Computed new affinity after node left [topVer=" + topVer +
- ", waitGrps=" + (info != null ? groupNames(info.waitGrps.keySet()) : null) + ']');
- }
- }
+ Map<Integer, CacheGroupAffinityMessage> result = onReassignmentEnforced(fut);
- return CacheGroupAffinityMessage.createAffinityDiffMessages(diff);
+ return result;
}
/**
@@ -1862,8 +1709,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
* @throws IgniteCheckedException If failed.
*/
public Map<Integer, CacheGroupAffinityMessage> onCustomEventWithEnforcedAffinityReassignment(
- final GridDhtPartitionsExchangeFuture fut) throws IgniteCheckedException
- {
+ final GridDhtPartitionsExchangeFuture fut) throws IgniteCheckedException {
assert DiscoveryCustomEvent.requiresCentralizedAffinityAssignment(fut.firstEvent());
Map<Integer, CacheGroupAffinityMessage> result = onReassignmentEnforced(fut);
@@ -1879,8 +1725,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
* @throws IgniteCheckedException If failed.
*/
private Map<Integer, CacheGroupAffinityMessage> onReassignmentEnforced(
- final GridDhtPartitionsExchangeFuture fut) throws IgniteCheckedException
- {
+ final GridDhtPartitionsExchangeFuture fut) throws IgniteCheckedException {
final ExchangeDiscoveryEvents evts = fut.context().events();
forAllRegisteredCacheGroups(new IgniteInClosureX<CacheGroupDescriptor>() {
@@ -1966,7 +1811,8 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
* @param crd Coordinator flag.
* @throws IgniteCheckedException If failed.
*/
- public void onBaselineTopologyChanged(final GridDhtPartitionsExchangeFuture fut, boolean crd) throws IgniteCheckedException {
+ public void onBaselineTopologyChanged(final GridDhtPartitionsExchangeFuture fut,
+ boolean crd) throws IgniteCheckedException {
assert !fut.firstEvent().eventNode().isClient();
WaitRebalanceInfo waitRebalanceInfo = initAffinityOnNodeJoin(fut, crd);
@@ -2094,8 +1940,8 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
* @param discoCache Discovery data cache.
* @param affCache Affinity.
* @param fetchFut Affinity fetch future.
- * @throws IgniteCheckedException If failed.
* @return Affinity assignment response.
+ * @throws IgniteCheckedException If failed.
*/
private GridDhtAffinityAssignmentResponse fetchAffinity(
AffinityTopologyVersion topVer,
@@ -2139,10 +1985,11 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
*
* @param fut Exchange future.
* @param crd Coordinator flag.
- * @throws IgniteCheckedException If failed.
* @return {@code True} if affinity should be assigned by coordinator.
+ * @throws IgniteCheckedException If failed.
*/
- public boolean onCentralizedAffinityChange(final GridDhtPartitionsExchangeFuture fut, boolean crd) throws IgniteCheckedException {
+ public boolean onCentralizedAffinityChange(final GridDhtPartitionsExchangeFuture fut,
+ boolean crd) throws IgniteCheckedException {
assert (fut.events().hasServerLeft() && !fut.firstEvent().eventNode().isClient()) ||
DiscoveryCustomEvent.requiresCentralizedAffinityAssignment(fut.firstEvent()) : fut.firstEvent();
@@ -2169,8 +2016,8 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
/**
* @param fut Exchange future.
* @param newAff {@code True} if there are no older nodes with affinity info available.
- * @throws IgniteCheckedException If failed.
* @return Future completed when caches initialization is done.
+ * @throws IgniteCheckedException If failed.
*/
public IgniteInternalFuture<?> initCoordinatorCaches(
final GridDhtPartitionsExchangeFuture fut,
@@ -2241,10 +2088,10 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
}
GridDhtAssignmentFetchFuture fetchFut = new GridDhtAssignmentFetchFuture(
- cctx,
- desc.groupId(),
- futureToFetchAffinity.topologyVersion(),
- futureToFetchAffinity.events().discoveryCache()
+ cctx,
+ desc.groupId(),
+ futureToFetchAffinity.topologyVersion(),
+ futureToFetchAffinity.events().discoveryCache()
);
fetchFut.init(false);
@@ -2255,13 +2102,13 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
fetchFut.listen(new IgniteInClosureX<IgniteInternalFuture<GridDhtAffinityAssignmentResponse>>() {
@Override public void applyx(IgniteInternalFuture<GridDhtAffinityAssignmentResponse> fetchFut)
- throws IgniteCheckedException {
+ throws IgniteCheckedException {
fetchAffinity(
- futureToFetchAffinity0.topologyVersion(),
- futureToFetchAffinity0.events(),
- futureToFetchAffinity0.events().discoveryCache(),
- aff,
- (GridDhtAssignmentFetchFuture)fetchFut
+ futureToFetchAffinity0.topologyVersion(),
+ futureToFetchAffinity0.events(),
+ futureToFetchAffinity0.events().discoveryCache(),
+ aff,
+ (GridDhtAssignmentFetchFuture)fetchFut
);
aff.calculate(topVer, fut.events(), fut.events().discoveryCache());
@@ -2375,30 +2222,30 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
if (cctx.localNode().isClient() && cctx.cache().cacheGroup(desc.groupId()) == null)
return;
- CacheGroupHolder cache = getOrCreateGroupHolder(evts.topologyVersion(), desc);
+ CacheGroupHolder grpHolder = getOrCreateGroupHolder(evts.topologyVersion(), desc);
- boolean latePrimary = cache.rebalanceEnabled;
+ boolean latePrimary = grpHolder.rebalanceEnabled;
boolean grpAdded = evts.nodeJoined(desc.receivedFrom());
initAffinityOnNodeJoin(evts,
grpAdded,
- cache.affinity(),
+ grpHolder.affinity(),
crd ? waitRebalanceInfo : null,
latePrimary);
if (crd && grpAdded) {
- AffinityAssignment aff = cache.aff.cachedAffinity(cache.aff.lastVersion());
+ AffinityAssignment aff = grpHolder.aff.cachedAffinity(grpHolder.aff.lastVersion());
assert evts.topologyVersion().equals(aff.topologyVersion()) : "Unexpected version [" +
- "grp=" + cache.aff.cacheOrGroupName() +
+ "grp=" + grpHolder.aff.cacheOrGroupName() +
", evts=" + evts.topologyVersion() +
- ", aff=" + cache.aff.lastVersion() + ']';
+ ", aff=" + grpHolder.aff.lastVersion() + ']';
Map<UUID, GridDhtPartitionMap> map = affinityFullMap(aff);
for (GridDhtPartitionMap map0 : map.values())
- cache.topology(fut.context().events().discoveryCache()).update(fut.exchangeId(), map0, true);
+ grpHolder.topology(fut.context().events().discoveryCache()).update(fut.exchangeId(), map0, true);
}
cctx.exchange().exchangerUpdateHeartbeat();
@@ -2466,77 +2313,40 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
assert affTopVer.topologyVersion() > 0 : "Affinity is not initialized [grp=" + aff.cacheOrGroupName() +
", topVer=" + affTopVer + ", node=" + cctx.localNodeId() + ']';
- assert aff.idealAssignmentRaw() != null : "Previous assignment is not available.";
+ List<List<ClusterNode>> curAff = aff.assignments(affTopVer);
+
+ assert aff.idealAssignment() != null : "Previous assignment is not available.";
- IdealAffinityAssignment idealAssignment = aff.calculate(evts.topologyVersion(), evts, evts.discoveryCache());
- List<List<ClusterNode>> curAssignment = aff.assignments(affTopVer);
+ List<List<ClusterNode>> idealAssignment = aff.calculate(evts.topologyVersion(), evts, evts.discoveryCache()).assignment();
List<List<ClusterNode>> newAssignment = null;
if (latePrimary) {
- BitSet processedPartitions = new BitSet(curAssignment.size());
+ for (int p = 0; p < idealAssignment.size(); p++) {
+ List<ClusterNode> newNodes = idealAssignment.get(p);
+ List<ClusterNode> curNodes = curAff.get(p);
- // Late affinity assignment to changed primaries.
- for (ClusterNode joinedNode : evts.joinedServerNodes()) {
- Set<Integer> primaries = idealAssignment.idealPrimaries(joinedNode);
-
- for (int p : primaries) {
- List<ClusterNode> curNodes = curAssignment.get(p);
-
- if (curNodes.isEmpty())
- continue;
-
- ClusterNode curPrimary = curNodes.get(0);
+ ClusterNode curPrimary = !curNodes.isEmpty() ? curNodes.get(0) : null;
+ ClusterNode newPrimary = !newNodes.isEmpty() ? newNodes.get(0) : null;
+ if (curPrimary != null && newPrimary != null && !curPrimary.equals(newPrimary)) {
assert cctx.discovery().node(evts.topologyVersion(), curPrimary.id()) != null : curPrimary;
- List<ClusterNode> idealNodes = idealAssignment.assignment().get(p);
-
- List<ClusterNode> newNodes = latePrimaryAssignment(aff,
+ List<ClusterNode> nodes0 = latePrimaryAssignment(aff,
p,
curPrimary,
- idealNodes,
- rebalanceInfo);
-
- if (newAssignment == null)
- newAssignment = new ArrayList<>(idealAssignment.assignment());
-
- newAssignment.set(p, newNodes);
-
- processedPartitions.set(p);
- }
- }
-
- Set<Integer> partitionsWithChangedPrimary = aff.partitionPrimariesDifferentToIdeal(affTopVer);
-
- for (int p : partitionsWithChangedPrimary) {
- // Already processed above.
- if (processedPartitions.get(p))
- continue;
-
- List<ClusterNode> curNodes = curAssignment.get(p);
-
- if (curNodes.isEmpty())
- continue;
-
- List<ClusterNode> idealOwners = idealAssignment.assignment().get(p);
-
- if (!curNodes.get(0).equals(idealOwners.get(0))) {
- List<ClusterNode> newNodes = latePrimaryAssignment(aff,
- p,
- curNodes.get(0),
- idealOwners,
+ newNodes,
rebalanceInfo);
if (newAssignment == null)
- newAssignment = new ArrayList<>(idealAssignment.assignment());
+ newAssignment = new ArrayList<>(idealAssignment);
- newAssignment.set(p, newNodes);
+ newAssignment.set(p, nodes0);
}
}
}
if (newAssignment == null)
- newAssignment = idealAssignment.assignment();
+ newAssignment = idealAssignment;
aff.initialize(evts.topologyVersion(), newAssignment);
}
@@ -2608,14 +2418,13 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
}
/**
- * Initializes current affinity assignment based on partitions availability.
- * Nodes that have most recent data will be considered affinity nodes.
+ * Initializes current affinity assignment based on partitions availability. Nodes that have most recent data will
+ * be considered affinity nodes.
*
* @param topVer Topology version.
* @param fut Exchange future.
* @param c Closure converting affinity diff.
* @param initAff {@code True} if need initialize affinity.
- *
* @return Affinity assignment for each of registered cache group.
*/
private <T> Map<Integer, Map<Integer, List<T>>> initAffinityBasedOnPartitionsAvailability(
@@ -2989,7 +2798,7 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
GridCacheSharedContext cctx,
CacheGroupDescriptor grpDesc,
AffinityTopologyVersion topVer
- ) throws IgniteCheckedException {
+ ) throws IgniteCheckedException {
return create(cctx, grpDesc, topVer, null);
}
@@ -3128,14 +2937,13 @@ public class CacheAffinitySharedManager<K, V> extends GridCacheSharedManagerAdap
* @param waitNode Node rebalancing data.
* @param assignment New assignment.
*/
- void add(Integer grpId, Integer part, UUID waitNode, List<ClusterNode> assignment) {
- assert !F.isEmpty(assignment) : assignment;
-
+ void add(Integer grpId, Integer part, UUID waitNode, @Nullable List<ClusterNode> assignment) {
deploymentIds.putIfAbsent(grpId, cachesRegistry.group(grpId).deploymentId());
waitGrps.computeIfAbsent(grpId, k -> new HashMap<>()).put(part, waitNode);
- assignments.computeIfAbsent(grpId, k -> new HashMap<>()).put(part, assignment);
+ if (assignment != null)
+ assignments.computeIfAbsent(grpId, k -> new HashMap<>()).put(part, assignment);
}
/** {@inheritDoc} */
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheEntryInfoCollection.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheEntryInfoCollection.java
index 968afd5..b6da77f 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheEntryInfoCollection.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheEntryInfoCollection.java
@@ -128,4 +128,23 @@ public class CacheEntryInfoCollection implements Message {
@Override public byte fieldsCount() {
return 1;
}
+
+ /** {@inheritDoc} */
+ @Override public String toString() {
+ StringBuilder b = new StringBuilder();
+ b.append("[");
+
+ for (int i = 0; i < infos().size(); i++) {
+ GridCacheEntryInfo info = infos().get(i);
+
+ Object k = info.key().value(null, false);
+
+ b.append("[key=").append(k == null ? "null" : k).append(", ver=").
+ append(info.version()).append(", val=").append(info.value()).append(']');
+ }
+
+ b.append(']');
+
+ return b.toString();
+ }
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java
index 29efa76..68dffbf 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/CacheGroupContext.java
@@ -65,6 +65,7 @@ import org.apache.ignite.lang.IgnitePredicate;
import org.apache.ignite.mxbean.CacheGroupMetricsMXBean;
import org.jetbrains.annotations.Nullable;
+import static org.apache.ignite.cache.CacheAtomicityMode.ATOMIC;
import static org.apache.ignite.cache.CacheAtomicityMode.TRANSACTIONAL_SNAPSHOT;
import static org.apache.ignite.cache.CacheMode.LOCAL;
import static org.apache.ignite.cache.CacheMode.REPLICATED;
@@ -182,6 +183,8 @@ public class CacheGroupContext {
/** Statistics holder to track IO operations for data pages. */
private final IoStatisticsHolder statHolderData;
+ /** */
+ private volatile boolean hasAtomicCaches;
/**
* @param ctx Context.
@@ -357,6 +360,9 @@ public class CacheGroupContext {
if (!drEnabled && cctx.isDrEnabled())
drEnabled = true;
+
+ if (!hasAtomicCaches)
+ hasAtomicCaches = cctx.config().getAtomicityMode() == ATOMIC;
}
/**
@@ -1278,4 +1284,11 @@ public class CacheGroupContext {
public IoStatisticsHolder statisticsHolderData() {
return statHolderData;
}
+
+ /**
+ * @return {@code True} if group has atomic caches.
+ */
+ public boolean hasAtomicCaches() {
+ return hasAtomicCaches;
+ }
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMapEntry.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMapEntry.java
index 5ca9c41..54dc706 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMapEntry.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMapEntry.java
@@ -1542,10 +1542,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
if (cctx.deferredDelete() && deletedUnlocked() && !isInternal() && !detached())
deletedUnlocked(false);
- updateCntr0 = nextPartitionCounter(topVer, tx == null || tx.local(), updateCntr);
-
- if (updateCntr != null && updateCntr != 0)
- updateCntr0 = updateCntr;
+ updateCntr0 = nextPartitionCounter(tx, updateCntr);
if (tx != null && cctx.group().persistenceEnabled() && cctx.group().walEnabled())
logPtr = logTxUpdate(tx, val, expireTime, updateCntr0);
@@ -1765,10 +1762,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
}
}
- updateCntr0 = nextPartitionCounter(topVer, tx == null || tx.local(), updateCntr);
-
- if (updateCntr != null && updateCntr != 0)
- updateCntr0 = updateCntr;
+ updateCntr0 = nextPartitionCounter(tx, updateCntr);
if (tx != null && cctx.group().persistenceEnabled() && cctx.group().walEnabled())
logPtr = logTxUpdate(tx, null, 0, updateCntr0);
@@ -2189,7 +2183,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
cctx.cache().metrics0().onInvokeRemove(old != null);
if (lsnrCol != null) {
- long updateCntr = nextPartitionCounter(AffinityTopologyVersion.NONE, true, null);
+ long updateCntr = nextPartitionCounter(AffinityTopologyVersion.NONE, true, false, null);
cctx.continuousQueries().onEntryUpdated(
lsnrCol,
@@ -2368,12 +2362,9 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
else
evtVal = (CacheObject)writeObj;
- long updateCntr0 = nextPartitionCounter(topVer, primary, updateCntr);
+ assert !primary && updateCntr != null;
- if (updateCntr != null)
- updateCntr0 = updateCntr;
-
- onUpdateFinished(updateCntr0);
+ onUpdateFinished(updateCntr);
cctx.continuousQueries().onEntryUpdated(
key,
@@ -2383,7 +2374,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
partition(),
primary,
false,
- updateCntr0,
+ updateCntr,
null,
topVer);
}
@@ -3447,7 +3438,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
long updateCntr = 0;
if (!preload)
- updateCntr = nextPartitionCounter(topVer, true, null);
+ updateCntr = nextPartitionCounter(topVer, true, true, null);
if (walEnabled) {
if (cctx.mvccEnabled()) {
@@ -3536,10 +3527,20 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
/**
* @param topVer Topology version for current operation.
* @param primary Primary node update flag.
+ * @param initial {@code True} if initial value.
* @param primaryCntr Counter assigned on primary node.
* @return Update counter.
*/
- protected long nextPartitionCounter(AffinityTopologyVersion topVer, boolean primary, @Nullable Long primaryCntr) {
+ protected long nextPartitionCounter(AffinityTopologyVersion topVer, boolean primary, boolean initial,
+ @Nullable Long primaryCntr) {
+ return 0;
+ }
+
+ /**
+ * @param tx Tx.
+ * @param updateCntr Update counter.
+ */
+ protected long nextPartitionCounter(IgniteInternalTx tx, @Nullable Long updateCntr) {
return 0;
}
@@ -5574,8 +5575,8 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
cctx.cacheId(),
entry.key(),
val,
- res.resultType() == ResultType.PREV_NULL ? CREATE :
- (res.resultType() == ResultType.REMOVED_NOT_NULL) ? DELETE : UPDATE,
+ res.resultType() == ResultType.PREV_NULL ? CREATE :
+ (res.resultType() == ResultType.REMOVED_NOT_NULL) ? DELETE : UPDATE,
tx.nearXidVersion(),
newVer,
expireTime,
@@ -5751,7 +5752,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
oldRow);
treeOp = oldRow != null && oldRow.link() == newRow.link() ?
- IgniteTree.OperationType.NOOP : IgniteTree.OperationType.PUT;
+ IgniteTree.OperationType.IN_PLACE : IgniteTree.OperationType.PUT;
}
else
treeOp = oldRow != null ? IgniteTree.OperationType.REMOVE : IgniteTree.OperationType.NOOP;
@@ -6376,10 +6377,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
", locNodeId=" + cctx.localNodeId() + ']';
}
- long updateCntr0 = entry.nextPartitionCounter(topVer, primary, updateCntr);
-
- if (updateCntr != null)
- updateCntr0 = updateCntr;
+ long updateCntr0 = entry.nextPartitionCounter(topVer, primary, false, updateCntr);
entry.logUpdate(op, updated, newVer, newExpireTime, updateCntr0);
@@ -6466,10 +6464,7 @@ public abstract class GridCacheMapEntry extends GridMetadataAwareAdapter impleme
// Must persist inside synchronization in non-tx mode.
cctx.store().remove(null, entry.key);
- long updateCntr0 = entry.nextPartitionCounter(topVer, primary, updateCntr);
-
- if (updateCntr != null)
- updateCntr0 = updateCntr;
+ long updateCntr0 = entry.nextPartitionCounter(topVer, primary, false, updateCntr);
entry.logUpdate(op, null, newVer, 0, updateCntr0);
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
index 50a282d..79cecfb 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCachePartitionExchangeManager.java
@@ -1773,7 +1773,9 @@ public class GridCachePartitionExchangeManager<K, V> extends GridCacheSharedMana
null,
msg.partsToReload(cctx.localNodeId(), grpId),
partsSizes.getOrDefault(grpId, Collections.emptyMap()),
- msg.topologyVersion());
+ msg.topologyVersion(),
+ null
+ );
}
}
@@ -1815,8 +1817,8 @@ public class GridCachePartitionExchangeManager<K, V> extends GridCacheSharedMana
try {
if (msg.exchangeId() == null) {
- if (log.isTraceEnabled())
- log.trace("Received local partition update [nodeId=" + node.id() + ", parts=" +
+ if (log.isDebugEnabled())
+ log.debug("Received local partition update [nodeId=" + node.id() + ", parts=" +
msg + ']');
boolean updated = false;
@@ -1865,7 +1867,7 @@ public class GridCachePartitionExchangeManager<K, V> extends GridCacheSharedMana
U.warn(log, "Client node tries to connect but its exchange " +
"info is cleaned up from exchange history. " +
"Consider increasing 'IGNITE_EXCHANGE_HISTORY_SIZE' property " +
- "or start clients in smaller batches. " +
+ "or start clients in smaller batches. " +
"Current settings and versions: " +
"[IGNITE_EXCHANGE_HISTORY_SIZE=" + EXCHANGE_HISTORY_SIZE + ", " +
"initVer=" + initVer + ", " +
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
index 48797ee..ec4b8cf 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheProcessor.java
@@ -119,7 +119,6 @@ import org.apache.ignite.internal.processors.cache.persistence.metastorage.MetaS
import org.apache.ignite.internal.processors.cache.persistence.metastorage.MetastorageLifecycleListener;
import org.apache.ignite.internal.processors.cache.persistence.metastorage.ReadOnlyMetastorage;
import org.apache.ignite.internal.processors.cache.persistence.partstate.GroupPartitionId;
-import org.apache.ignite.internal.processors.cache.persistence.partstate.PartitionRecoverState;
import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteCacheSnapshotManager;
import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotDiscoveryMessage;
import org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseList;
@@ -236,6 +235,10 @@ public class GridCacheProcessor extends GridProcessorAdapter {
private static final String INVALID_REGION_CONFIGURATION_MESSAGE = "Failed to join node " +
"(Incompatible data region configuration [region=%s, locNodeId=%s, isPersistenceEnabled=%s, rmtNodeId=%s, isPersistenceEnabled=%s])";
+ /** Template of message of failed node join because encryption settings are different for the same cache. */
+ private static final String ENCRYPT_MISMATCH_MESSAGE = "Failed to join node to the cluster " +
+ "(encryption settings are different for cache '%s' : local=%s, remote=%s.)";
+
/** */
private final boolean startClientCaches =
IgniteSystemProperties.getBoolean(IgniteSystemProperties.IGNITE_START_CACHES_ON_JOIN, false);
@@ -1069,7 +1072,7 @@ public class GridCacheProcessor extends GridProcessorAdapter {
}
/**
- * @param node Remote node to check.
+ * @param rmtNode Remote node to check.
* @return Data storage configuration
*/
private DataStorageConfiguration extractDataStorage(ClusterNode rmtNode) {
@@ -3489,12 +3492,12 @@ public class GridCacheProcessor extends GridProcessorAdapter {
}
}
- DynamicCacheDescriptor localDesc = cacheDescriptor(cacheInfo.cacheData().config().getName());
+ DynamicCacheDescriptor locDesc = cacheDescriptor(cacheInfo.cacheData().config().getName());
- if (localDesc == null)
+ if (locDesc == null)
continue;
- QuerySchemaPatch schemaPatch = localDesc.makeSchemaPatch(cacheInfo.cacheData().queryEntities());
+ QuerySchemaPatch schemaPatch = locDesc.makeSchemaPatch(cacheInfo.cacheData().queryEntities());
if (schemaPatch.hasConflicts() || (isGridActive && !schemaPatch.isEmpty())) {
if (errorMsg.length() > 0)
@@ -3502,9 +3505,22 @@ public class GridCacheProcessor extends GridProcessorAdapter {
if (schemaPatch.hasConflicts())
errorMsg.append(String.format(MERGE_OF_CONFIG_CONFLICTS_MESSAGE,
- localDesc.cacheName(), schemaPatch.getConflictsMessage()));
+ locDesc.cacheName(), schemaPatch.getConflictsMessage()));
else
- errorMsg.append(String.format(MERGE_OF_CONFIG_REQUIRED_MESSAGE, localDesc.cacheName()));
+ errorMsg.append(String.format(MERGE_OF_CONFIG_REQUIRED_MESSAGE, locDesc.cacheName()));
+ }
+
+ // This check must be done on join, otherwise group encryption key will be
+ // written to metastore regardless of validation check and could trigger WAL write failures.
+ boolean locEnc = locDesc.cacheConfiguration().isEncryptionEnabled();
+ boolean rmtEnc = cacheInfo.cacheData().config().isEncryptionEnabled();
+
+ if (locEnc != rmtEnc) {
+ if (errorMsg.length() > 0)
+ errorMsg.append("\n");
+
+ // Message will be printed on remote node, so need to swap local and remote.
+ errorMsg.append(String.format(ENCRYPT_MISMATCH_MESSAGE, locDesc.cacheName(), rmtEnc, locEnc));
}
}
@@ -6016,7 +6032,7 @@ public class GridCacheProcessor extends GridProcessorAdapter {
*/
private void restorePartitionStates(
Collection<CacheGroupContext> forGroups,
- Map<GroupPartitionId, PartitionRecoverState> partitionStates
+ Map<GroupPartitionId, Integer> partitionStates
) throws IgniteCheckedException {
long startRestorePart = U.currentTimeMillis();
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheSharedTtlCleanupManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheSharedTtlCleanupManager.java
index 2254c5e..3623fd9 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheSharedTtlCleanupManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheSharedTtlCleanupManager.java
@@ -23,6 +23,7 @@ import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.failure.FailureContext;
import org.apache.ignite.internal.IgniteInterruptedCheckedException;
import org.apache.ignite.internal.util.typedef.X;
+import org.apache.ignite.internal.NodeStoppingException;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.internal.util.worker.GridWorker;
import org.apache.ignite.thread.IgniteThread;
@@ -164,7 +165,13 @@ public class GridCacheSharedTtlCleanupManager extends GridCacheSharedManagerAdap
}
}
catch (Throwable t) {
- if (!(X.hasCause(t, IgniteInterruptedCheckedException.class, InterruptedException.class)))
+ if (X.hasCause(t, NodeStoppingException.class)) {
+ isCancelled = true; // Treat node stopping as valid worker cancellation.
+
+ return;
+ }
+
+ if (!(t instanceof IgniteInterruptedCheckedException || t instanceof InterruptedException))
err = t;
throw t;
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheUtils.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheUtils.java
index 48c86b9..cc5dd42 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheUtils.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheUtils.java
@@ -849,6 +849,7 @@ public class GridCacheUtils {
return tx.getClass().getSimpleName() + "[xid=" + tx.xid() +
", xidVersion=" + tx.xidVersion() +
+ ", nearXidVersion=" + tx.nearXidVersion() +
", concurrency=" + tx.concurrency() +
", isolation=" + tx.isolation() +
", state=" + tx.state() +
@@ -857,7 +858,7 @@ public class GridCacheUtils {
", nodeId=" + tx.nodeId() +
", timeout=" + tx.timeout() +
", duration=" + (U.currentTimeMillis() - tx.startTime()) +
- (tx instanceof GridNearTxLocal ? ", label=" + ((GridNearTxLocal)tx).label() : "") +
+ (tx instanceof GridNearTxLocal ? ", label=" + tx.label() : "") +
']';
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManager.java
index b7e8ec7..74a0b30 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManager.java
@@ -31,8 +31,9 @@ import org.apache.ignite.internal.processors.cache.persistence.CacheDataRow;
import org.apache.ignite.internal.processors.cache.persistence.CacheSearchRow;
import org.apache.ignite.internal.processors.cache.persistence.RootPage;
import org.apache.ignite.internal.processors.cache.persistence.RowStore;
+import org.apache.ignite.internal.processors.cache.persistence.freelist.SimpleDataRow;
import org.apache.ignite.internal.processors.cache.persistence.partstate.GroupPartitionId;
-import org.apache.ignite.internal.processors.cache.persistence.partstate.PartitionRecoverState;
+import org.apache.ignite.internal.processors.cache.persistence.partstorage.PartitionMetaStorage;
import org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseList;
import org.apache.ignite.internal.processors.cache.tree.PendingEntriesTree;
import org.apache.ignite.internal.processors.cache.tree.mvcc.data.MvccUpdateResult;
@@ -90,7 +91,7 @@ public interface IgniteCacheOffheapManager {
* @return Number of processed partitions.
* @throws IgniteCheckedException If failed.
*/
- long restorePartitionStates(Map<GroupPartitionId, PartitionRecoverState> partitionRecoveryStates) throws IgniteCheckedException;
+ long restorePartitionStates(Map<GroupPartitionId, Integer> partitionRecoveryStates) throws IgniteCheckedException;
/**
* Partition counter update callback. May be overridden by plugin-provided subclasses.
@@ -104,9 +105,10 @@ public interface IgniteCacheOffheapManager {
* Initial counter will be updated on state restore only
*
* @param part Partition
- * @param cntr New initial counter
+ * @param start Start.
+ * @param delta Delta.
*/
- public void onPartitionInitialCounterUpdated(int part, long cntr);
+ public void onPartitionInitialCounterUpdated(int part, long start, long delta);
/**
* Partition counter provider. May be overridden by plugin-provided subclasses.
@@ -584,6 +586,13 @@ public interface IgniteCacheOffheapManager {
*/
interface CacheDataStore {
/**
+ * Initialize data store if it exists.
+ *
+ * @return {@code True} if initialized.
+ */
+ boolean init();
+
+ /**
* @return Partition ID.
*/
int partId();
@@ -594,13 +603,6 @@ public interface IgniteCacheOffheapManager {
String name();
/**
- * @param size Size to init.
- * @param updCntr Update counter to init.
- * @param cacheSizes Cache sizes if store belongs to group containing multiple caches.
- */
- void init(long size, long updCntr, @Nullable Map<Integer, Long> cacheSizes);
-
- /**
* @param cacheId Cache ID.
* @return Size.
*/
@@ -630,22 +632,36 @@ public interface IgniteCacheOffheapManager {
void updateSize(int cacheId, long delta);
/**
- * @return Update counter.
+ * @return Update counter (LWM).
*/
long updateCounter();
/**
+ * @return Reserved counter (HWM).
+ */
+ long reservedCounter();
+
+ /**
+ * @return Update counter or {@code null} if store is not yet created.
+ */
+ @Nullable PartitionUpdateCounter partUpdateCounter();
+
+ /**
+ * @param delta Delta.
+ */
+ long reserve(long delta);
+
+ /**
* @param val Update counter.
*/
void updateCounter(long val);
/**
* Updates counters from start value by delta value.
- *
* @param start Start.
- * @param delta Delta
+ * @param delta Delta.
*/
- void updateCounter(long start, long delta);
+ boolean updateCounter(long start, long delta);
/**
* @return Next update counter.
@@ -1017,9 +1033,10 @@ public interface IgniteCacheOffheapManager {
public RowStore rowStore();
/**
- * @param cntr Counter.
+ * @param start Counter.
+ * @param delta Delta.
*/
- public void updateInitialCounter(long cntr);
+ public void updateInitialCounter(long start, long delta);
/**
* Inject rows cache cleaner.
@@ -1047,5 +1064,15 @@ public interface IgniteCacheOffheapManager {
* @throws IgniteCheckedException If failed.
*/
public void preload() throws IgniteCheckedException;
+
+ /**
+ * Reset counters for partition.
+ */
+ void resetUpdateCounter();
+
+ /**
+ * Partition storage.
+ */
+ public PartitionMetaStorage<SimpleDataRow> partStorage();
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
index a0ebcdc..3706754 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/IgniteCacheOffheapManagerImpl.java
@@ -36,6 +36,9 @@ import javax.cache.processor.EntryProcessor;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.IgniteException;
import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.IgniteSystemProperties;
+import org.apache.ignite.failure.FailureContext;
+import org.apache.ignite.failure.FailureType;
import org.apache.ignite.internal.NodeStoppingException;
import org.apache.ignite.internal.pagemem.FullPageId;
import org.apache.ignite.internal.pagemem.wal.record.delta.DataPageMvccMarkUpdatedRecord;
@@ -59,8 +62,9 @@ import org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapt
import org.apache.ignite.internal.processors.cache.persistence.CacheSearchRow;
import org.apache.ignite.internal.processors.cache.persistence.RootPage;
import org.apache.ignite.internal.processors.cache.persistence.RowStore;
+import org.apache.ignite.internal.processors.cache.persistence.freelist.SimpleDataRow;
import org.apache.ignite.internal.processors.cache.persistence.partstate.GroupPartitionId;
-import org.apache.ignite.internal.processors.cache.persistence.partstate.PartitionRecoverState;
+import org.apache.ignite.internal.processors.cache.persistence.partstorage.PartitionMetaStorage;
import org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.BPlusIO;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.DataPageIO;
@@ -139,6 +143,10 @@ import static org.apache.ignite.internal.util.IgniteTree.OperationType.PUT;
*/
public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager {
/** */
+ private final boolean failNodeOnPartitionInconsistency = Boolean.getBoolean(
+ IgniteSystemProperties.IGNITE_FAIL_NODE_ON_UNRECOVERABLE_PARTITION_INCONSISTENCY);
+
+ /** */
protected GridCacheSharedContext ctx;
/** */
@@ -251,7 +259,7 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
}
/** {@inheritDoc} */
- @Override public long restorePartitionStates(Map<GroupPartitionId, PartitionRecoverState> partitionRecoveryStates) throws IgniteCheckedException {
+ @Override public long restorePartitionStates(Map<GroupPartitionId, Integer> partitionRecoveryStates) throws IgniteCheckedException {
return 0; // No-op.
}
@@ -692,7 +700,7 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
}
/** {@inheritDoc} */
- @Override public void onPartitionInitialCounterUpdated(int part, long cntr) {
+ @Override public void onPartitionInitialCounterUpdated(int part, long start, long delta) {
// No-op.
}
@@ -1398,7 +1406,7 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
private final CacheDataTree dataTree;
/** Update counter. */
- protected final PartitionUpdateCounter pCntr = new PartitionUpdateCounter(log);
+ protected final PartitionUpdateCounter pCntr;
/** Partition size. */
private final AtomicLong storageSize = new AtomicLong();
@@ -1431,6 +1439,14 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
this.name = name;
this.rowStore = rowStore;
this.dataTree = dataTree;
+ if (grp.mvccEnabled())
+ pCntr = new PartitionMvccTxUpdateCounterImpl();
+ else if (grp.hasAtomicCaches())
+ pCntr = new PartitionAtomicUpdateCounterImpl();
+ else {
+ pCntr = ctx.logger(PartitionTxUpdateCounterDebugWrapper.class).isDebugEnabled() ?
+ new PartitionTxUpdateCounterDebugWrapper(grp, partId) : new PartitionTxUpdateCounterImpl();
+ }
}
/**
@@ -1448,6 +1464,11 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
}
/** {@inheritDoc} */
+ @Override public boolean init() {
+ return false;
+ }
+
+ /** {@inheritDoc} */
@Override public int partId() {
return partId;
}
@@ -1527,14 +1548,17 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
return pCntr.initial();
}
- /** {@inheritDoc} */
- @Override public void updateInitialCounter(long cntr) {
- pCntr.updateInitial(cntr);
+ /** {@inheritDoc}
+ * @param start Start.
+ * @param delta Delta.
+ */
+ @Override public void updateInitialCounter(long start, long delta) {
+ pCntr.updateInitial(start, delta);
}
/** {@inheritDoc} */
@Override public long getAndIncrementUpdateCounter(long delta) {
- return pCntr.getAndAdd(delta);
+ return pCntr.reserve(delta);
}
/** {@inheritDoc} */
@@ -1543,13 +1567,39 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
}
/** {@inheritDoc} */
+ @Override public long reservedCounter() {
+ return pCntr.reserved();
+ }
+
+ /** {@inheritDoc} */
+ @Override public PartitionUpdateCounter partUpdateCounter() {
+ return pCntr;
+ }
+
+ /** {@inheritDoc} */
+ @Override public long reserve(long delta) {
+ return pCntr.reserve(delta);
+ }
+
+ /** {@inheritDoc} */
@Override public void updateCounter(long val) {
- pCntr.update(val);
+ try {
+ pCntr.update(val);
+ }
+ catch (IgniteCheckedException e) {
+ U.error(log, "Failed to update partition counter. " +
+ "Most probably a node with most actual data is out of topology or data streamer is used " +
+ "in preload mode (allowOverride=false) concurrently with cache transactions [grpName=" +
+ grp.name() + ", partId=" + partId + ']', e);
+
+ if (failNodeOnPartitionInconsistency)
+ ctx.kernalContext().failure().process(new FailureContext(FailureType.CRITICAL_ERROR, e));
+ }
}
/** {@inheritDoc} */
- @Override public void updateCounter(long start, long delta) {
- pCntr.update(start, delta);
+ @Override public boolean updateCounter(long start, long delta) {
+ return pCntr.update(start, delta);
}
/** {@inheritDoc} */
@@ -1638,6 +1688,7 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
}
case NOOP:
+ case IN_PLACE:
break;
default:
@@ -2891,9 +2942,14 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
rowStore().setRowCacheCleaner(rowCacheCleaner);
}
- /** {@inheritDoc} */
- @Override public void init(long size, long updCntr, @Nullable Map<Integer, Long> cacheSizes) {
- pCntr.init(updCntr);
+ /**
+ * @param size Size to init.
+ * @param updCntr Update counter.
+ * @param cacheSizes Cache sizes if store belongs to group containing multiple caches.
+ * @param cntrUpdData Counter updates.
+ */
+ public void restoreState(long size, long updCntr, @Nullable Map<Integer, Long> cacheSizes, byte[] cntrUpdData) {
+ pCntr.init(updCntr, cntrUpdData);
storageSize.set(size);
@@ -2913,6 +2969,16 @@ public class IgniteCacheOffheapManagerImpl implements IgniteCacheOffheapManager
// No-op.
}
+ /** {@inheritDoc} */
+ @Override public void resetUpdateCounter() {
+ pCntr.reset();
+ }
+
+ /** {@inheritDoc} */
+ @Override public PartitionMetaStorage<SimpleDataRow> partStorage() {
+ return null;
+ }
+
/**
* @param cctx Cache context.
* @param key Key.
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionAtomicUpdateCounterImpl.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionAtomicUpdateCounterImpl.java
new file mode 100644
index 0000000..32a004a
--- /dev/null
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionAtomicUpdateCounterImpl.java
@@ -0,0 +1,146 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache;
+
+import java.util.Iterator;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.internal.util.GridEmptyIterator;
+import org.apache.ignite.internal.util.GridLongList;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Partition update counter for non-tx scenarios without support for tracking missed updates.
+ * TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11797
+ */
+public class PartitionAtomicUpdateCounterImpl implements PartitionUpdateCounter {
+ /** Counter of applied updates in partition. */
+ private final AtomicLong cntr = new AtomicLong();
+
+ /**
+ * Initial counter is set to update with max sequence number after WAL recovery.
+ */
+ private long initCntr;
+
+ /** {@inheritDoc} */
+ @Override public void init(long initUpdCntr, @Nullable byte[] cntrUpdData) {
+ cntr.set(initUpdCntr);
+
+ initCntr = initUpdCntr;
+ }
+
+ /** {@inheritDoc} */
+ @Override public long initial() {
+ return initCntr;
+ }
+
+ /** {@inheritDoc} */
+ @Override public long get() {
+ return cntr.get();
+ }
+
+ /** {@inheritDoc} */
+ @Override public long next() {
+ return cntr.incrementAndGet();
+ }
+
+ /** {@inheritDoc} */
+ @SuppressWarnings("StatementWithEmptyBody")
+ @Override public void update(long val) {
+ long cur;
+
+ while(val > (cur = cntr.get()) && !cntr.compareAndSet(cur, val));
+ }
+
+ /** {@inheritDoc} */
+ @Override public boolean update(long start, long delta) {
+ update(start + delta);
+
+ return false; // Prevents RollbackRecord in mixed tx-atomic mode.
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized void updateInitial(long start, long delta) {
+ update(start + delta);
+
+ initCntr = get();
+ }
+
+ /** {@inheritDoc} */
+ @Override public GridLongList finalizeUpdateCounters() {
+ return new GridLongList(0);
+ }
+
+ /** {@inheritDoc} */
+ @Override public long reserve(long delta) {
+ return next(delta);
+ }
+
+ /** {@inheritDoc} */
+ @Override public long next(long delta) {
+ return cntr.getAndAdd(delta);
+ }
+
+ /** {@inheritDoc} */
+ @Override public boolean sequential() {
+ return true;
+ }
+
+ /** {@inheritDoc} */
+ @Override public @Nullable byte[] getBytes() {
+ return null;
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized void reset() {
+ initCntr = 0;
+
+ cntr.set(0);
+ }
+
+ /** {@inheritDoc} */
+ @Override public boolean equals(Object o) {
+ if (this == o)
+ return true;
+ if (o == null || getClass() != o.getClass())
+ return false;
+
+ PartitionAtomicUpdateCounterImpl cntr = (PartitionAtomicUpdateCounterImpl)o;
+
+ return this.cntr.get() == cntr.cntr.get();
+ }
+
+ /** {@inheritDoc} */
+ @Override public long reserved() {
+ return get();
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized boolean empty() {
+ return get() == 0;
+ }
+
+ /** {@inheritDoc} */
+ @Override public Iterator<long[]> iterator() {
+ return new GridEmptyIterator<>();
+ }
+
+ /** {@inheritDoc} */
+ @Override public String toString() {
+ return "Counter [init=" + initCntr + ", val=" + get() + ']';
+ }
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionMvccTxUpdateCounterImpl.java
similarity index 50%
copy from modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
copy to modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionMvccTxUpdateCounterImpl.java
index 133f0a1..2f5d77a 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionMvccTxUpdateCounterImpl.java
@@ -15,38 +15,19 @@
* limitations under the License.
*/
-package org.apache.ignite.internal.processors.cache.persistence;
-
-import org.apache.ignite.IgniteCheckedException;
+package org.apache.ignite.internal.processors.cache;
/**
- * Simple interface for data, store in some RowStore.
+ * Update counter implementation for MVCC mode.
*/
-public interface Storable {
- /**
- * @param link Link for this row.
- */
- public void link(long link);
-
- /**
- * @return Link for this row.
- */
- public long link();
-
- /**
- * @return Partition.
- */
- public int partition();
-
- /**
- * @return Row size in page.
- * @throws IgniteCheckedException If failed.
- */
- public int size() throws IgniteCheckedException;
+public class PartitionMvccTxUpdateCounterImpl extends PartitionTxUpdateCounterImpl {
+ /** {@inheritDoc} */
+ @Override public long reserve(long delta) {
+ return next(delta);
+ }
- /**
- * @return Row header size in page. Header is indivisible part of row
- * which is entirely available on the very first page followed by the row link.
- */
- public int headerSize();
+ /** {@inheritDoc} */
+ @Override public long reserved() {
+ return get();
+ }
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionTxUpdateCounterDebugWrapper.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionTxUpdateCounterDebugWrapper.java
new file mode 100644
index 0000000..a63479b
--- /dev/null
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionTxUpdateCounterDebugWrapper.java
@@ -0,0 +1,201 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache;
+
+import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.IgniteLogger;
+import org.apache.ignite.internal.util.GridLongList;
+import org.apache.ignite.internal.util.typedef.internal.SB;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Update counter implementation useful for debugging.
+ */
+public class PartitionTxUpdateCounterDebugWrapper extends PartitionTxUpdateCounterImpl {
+ /** */
+ private IgniteLogger log;
+
+ /** */
+ private int partId;
+
+ /** */
+ private CacheGroupContext grp;
+
+ /**
+ * @param grp Group.
+ * @param partId Part id.
+ */
+ public PartitionTxUpdateCounterDebugWrapper(CacheGroupContext grp, int partId) {
+ this.log = grp.shared().logger(getClass());
+ this.partId = partId;
+ this.grp = grp;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void init(long initUpdCntr, @Nullable byte[] cntrUpdData) {
+ super.init(initUpdCntr, cntrUpdData);
+
+ log.debug("[op=init" +
+ ", grpId=" + grp.groupId() +
+ ", grpName=" + grp.cacheOrGroupName() +
+ ", caches=" + grp.caches() +
+ ", atomicity=" + grp.config().getAtomicityMode() +
+ ", syncMode=" + grp.config().getWriteSynchronizationMode() +
+ ", mode=" + grp.config().getCacheMode() +
+ ", partId=" + partId +
+ ", gapsLen=" + (cntrUpdData != null ? cntrUpdData.length : 0) +
+ ", cur=" + toString() +
+ ']');
+ }
+
+ /** {@inheritDoc} */
+ @Override public void updateInitial(long start, long delta) {
+ SB sb = new SB();
+
+ sb.a("[op=updateInitial" +
+ ", grpId=" + grp.groupId() +
+ ", partId=" + partId +
+ ", range=(" + start + "," + delta + ")" +
+ ", before=" + toString());
+
+ try {
+ super.updateInitial(start, delta);
+ }
+ finally {
+ log.debug(sb.a(", after=" + toString() +
+ ']').toString());
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public long next() {
+ SB sb = new SB();
+
+ sb.a("[op=next" +
+ ", grpId=" + grp.groupId() +
+ ", partId=" + partId +
+ ", before=" + toString());
+
+ try {
+ return super.next();
+ }
+ finally {
+ log.debug(sb.a(", after=" + toString() +
+ ']').toString());
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public long next(long delta) {
+ SB sb = new SB();
+
+ sb.a("[op=next" +
+ ", grpId=" + grp.groupId() +
+ ", partId=" + partId +
+ ", delta=" + delta +
+ ", before=" + toString());
+
+ try {
+ return super.next();
+ }
+ finally {
+ log.debug(sb.a(", after=" + toString() +
+ ']').toString());
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized void update(long val) throws IgniteCheckedException {
+ SB sb = new SB();
+
+ sb.a("[op=set" +
+ ", grpId=" + grp.groupId() +
+ ", partId=" + partId +
+ ", val=" + val +
+ ", before=" + toString());
+
+ try {
+ super.update(val);
+ }
+ finally {
+ log.debug(sb.a(", after=" + toString() +
+ ']').toString());
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized GridLongList finalizeUpdateCounters() {
+ SB sb = new SB();
+
+ sb.a("[op=finalizeUpdateCounters" +
+ ", grpId=" + grp.groupId() +
+ ", partId=" + partId +
+ ", before=" + toString() +
+ ']');
+
+ try {
+ return super.finalizeUpdateCounters();
+ }
+ finally {
+ log.debug(sb.a(", after=" + toString() +
+ ']').toString());
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized long reserve(long delta) {
+ SB sb = new SB();
+
+ sb.a("[op=reserve" +
+ ", grpId=" + grp.groupId() +
+ ", partId=" + partId +
+ ", delta=" + delta +
+ ", before=" + toString());
+
+ try {
+ return super.reserve(delta);
+ }
+ finally {
+ log.debug(sb.a(", after=" + toString() +
+ ']').toString());
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized boolean update(long start, long delta) {
+ SB sb = new SB();
+
+ sb.a("[op=update" +
+ ", grpId=" + grp.groupId() +
+ ", partId=" + partId +
+ ", delta=(" + start + "," + delta + ")" +
+ ", before=" + toString());
+
+ boolean updated = false;
+
+ try {
+ updated = super.update(start, delta);
+ }
+ finally {
+ log.debug(sb.a(", after=" + toString() +
+ ']').toString());
+ }
+
+ return updated;
+ }
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionTxUpdateCounterImpl.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionTxUpdateCounterImpl.java
new file mode 100644
index 0000000..7b6e911
--- /dev/null
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionTxUpdateCounterImpl.java
@@ -0,0 +1,443 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.util.Iterator;
+import java.util.NavigableSet;
+import java.util.TreeSet;
+import java.util.concurrent.atomic.AtomicLong;
+import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.IgniteException;
+import org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl;
+import org.apache.ignite.internal.util.GridLongList;
+import org.apache.ignite.internal.util.typedef.F;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Update counter implementation used for transactional cache groups.
+ */
+public class PartitionTxUpdateCounterImpl implements PartitionUpdateCounter {
+ /**
+ * Max allowed missed updates. Overflow will trigger critical failure handler to prevent OOM.
+ */
+ public static final int MAX_MISSED_UPDATES = 10_000;
+
+ /** Counter updates serialization version. */
+ private static final byte VERSION = 1;
+
+ /** Queue of finished out of order counter updates. */
+ private TreeSet<Item> queue = new TreeSet<>();
+
+ /** LWM. */
+ private final AtomicLong cntr = new AtomicLong();
+
+ /** HWM. */
+ private final AtomicLong reserveCntr = new AtomicLong();
+
+ /**
+ * Initial counter points to last sequential update after WAL recovery.
+ * @deprecated TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11794
+ */
+ @Deprecated private long initCntr;
+
+ /** {@inheritDoc} */
+ @Override public void init(long initUpdCntr, @Nullable byte[] cntrUpdData) {
+ cntr.set(initUpdCntr);
+
+ initCntr = initUpdCntr;
+
+ queue = fromBytes(cntrUpdData);
+ }
+
+ /** {@inheritDoc} */
+ @Override public long initial() {
+ return initCntr;
+ }
+
+ /** {@inheritDoc} */
+ @Override public long get() {
+ return cntr.get();
+ }
+
+ /** */
+ protected synchronized long highestAppliedCounter() {
+ return queue.isEmpty() ? cntr.get() : queue.last().absolute();
+ }
+
+ /**
+ * @return Next update counter. For tx mode called by {@link DataStreamerImpl} IsolatedUpdater.
+ */
+ @Override public long next() {
+ long next = cntr.incrementAndGet();
+
+ reserveCntr.set(next);
+
+ return next;
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized void update(long val) throws IgniteCheckedException {
+ // Absolute counter should be not less than last applied update.
+ // Otherwise supplier doesn't contain some updates and rebalancing couldn't restore consistency.
+ // Best behavior is to stop node by failure handler in such a case.
+ if (!gaps().isEmpty() && val < highestAppliedCounter())
+ throw new IgniteCheckedException("Failed to update the counter [newVal=" + val + ", curState=" + this + ']');
+
+ long cur = cntr.get();
+
+ // Reserved update counter is updated only on exchange or in non-tx mode.
+ reserveCntr.set(Math.max(cur, val));
+
+ if (val <= cur)
+ return;
+
+ cntr.set(val);
+
+ if (!queue.isEmpty())
+ queue.clear();
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized boolean update(long start, long delta) {
+ long cur = cntr.get(), next;
+
+ if (cur > start)
+ return false;
+
+ if (cur < start) {
+ // Try merge with adjacent gaps in sequence.
+ Item tmp = new Item(start, delta);
+ Item ref = tmp;
+
+ NavigableSet<Item> set = queue.headSet(tmp, false);
+
+ // Merge with previous, possibly modifying previous.
+ if (!set.isEmpty()) {
+ Item last = set.last();
+
+ if (last.start + last.delta == start) {
+ tmp = last;
+
+ last.delta += delta;
+ }
+ else if (last.within(start) && last.within(start + delta - 1))
+ return false;
+ }
+
+ // Merge with next, possibly modifying previous and removing next.
+ if (!(set = queue.tailSet(tmp, false)).isEmpty()) {
+ Item first = set.first();
+
+ if (tmp.start + tmp.delta == first.start) {
+ if (ref != tmp) {
+ tmp.delta += first.delta;
+
+ set.pollFirst(); // Merge and remove obsolete head.
+ }
+ else {
+ tmp = first;
+
+ first.start = start;
+ first.delta += delta;
+ }
+ }
+ }
+
+ if (tmp != ref)
+ return true;
+
+ return offer(new Item(start, delta)); // backup node with gaps
+ }
+
+ while (true) {
+ boolean res = cntr.compareAndSet(cur, next = start + delta);
+
+ assert res;
+
+ Item peek = peek();
+
+ if (peek == null || peek.start != next)
+ return true;
+
+ Item item = poll();
+
+ assert peek == item;
+
+ start = item.start;
+ delta = item.delta;
+ cur = next;
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public void updateInitial(long start, long delta) {
+ update(start, delta);
+
+ initCntr = get();
+ }
+
+ /** */
+ private Item poll() {
+ return queue.pollFirst();
+ }
+
+ /** */
+ private Item peek() {
+ return queue.isEmpty() ? null : queue.first();
+ }
+
+ /**
+ * @param item Item.
+ */
+ private boolean offer(Item item) {
+ if (queue.size() == MAX_MISSED_UPDATES) // Should trigger failure handler.
+ throw new IgniteException("Too many gaps [cntr=" + this + ']');
+
+ return queue.add(item);
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized GridLongList finalizeUpdateCounters() {
+ Item item = poll();
+
+ GridLongList gaps = null;
+
+ while (item != null) {
+ if (gaps == null)
+ gaps = new GridLongList((queue.size() + 1) * 2);
+
+ long start = cntr.get() + 1;
+ long end = item.start;
+
+ gaps.add(start);
+ gaps.add(end);
+
+ // Close pending ranges.
+ cntr.set(item.start + item.delta);
+
+ item = poll();
+ }
+
+ return gaps;
+ }
+
+ /** {@inheritDoc} */
+ @Override public long reserve(long delta) {
+ long cntr = get();
+
+ long reserved = reserveCntr.getAndAdd(delta);
+
+ assert reserved >= cntr : "LWM after HWM: lwm=" + cntr + ", hwm=" + reserved;
+
+ return reserved;
+ }
+
+ /** {@inheritDoc} */
+ @Override public long next(long delta) {
+ return cntr.getAndAdd(delta);
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized boolean sequential() {
+ return gaps().isEmpty();
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized @Nullable byte[] getBytes() {
+ if (queue.isEmpty())
+ return null;
+
+ try {
+ ByteArrayOutputStream bos = new ByteArrayOutputStream();
+
+ DataOutputStream dos = new DataOutputStream(bos);
+
+ dos.writeByte(VERSION);
+
+ int size = queue.size();
+
+ dos.writeInt(size);
+
+ for (Item item : queue) {
+ dos.writeLong(item.start);
+ dos.writeLong(item.delta);
+ }
+
+ bos.close();
+
+ return bos.toByteArray();
+ }
+ catch (IOException e) {
+ throw new IgniteException(e);
+ }
+ }
+
+ /**
+ * @param raw Raw bytes.
+ */
+ private @Nullable TreeSet<Item> fromBytes(@Nullable byte[] raw) {
+ if (raw == null)
+ return new TreeSet<>();
+
+ TreeSet<Item> ret = new TreeSet<>();
+
+ try {
+ ByteArrayInputStream bis = new ByteArrayInputStream(raw);
+
+ DataInputStream dis = new DataInputStream(bis);
+
+ dis.readByte(); // Version.
+
+ int cnt = dis.readInt(); // Holes count.
+
+ while(cnt-- > 0)
+ ret.add(new Item(dis.readLong(), dis.readLong()));
+
+ return ret;
+ }
+ catch (IOException e) {
+ throw new IgniteException(e);
+ }
+ }
+
+ /** */
+ private TreeSet<Item> gaps() {
+ return queue;
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized void reset() {
+ cntr.set(0);
+
+ reserveCntr.set(0);
+
+ queue = new TreeSet<>();
+ }
+
+ /**
+ * Update counter task. Update from start value by delta value.
+ */
+ private static class Item implements Comparable<Item> {
+ /** */
+ private long start;
+
+ /** */
+ private long delta;
+
+ /**
+ * @param start Start value.
+ * @param delta Delta value.
+ */
+ private Item(long start, long delta) {
+ this.start = start;
+ this.delta = delta;
+ }
+
+ /** {@inheritDoc} */
+ @Override public int compareTo(@NotNull Item o) {
+ return Long.compare(this.start, o.start);
+ }
+
+ /** {@inheritDoc} */
+ @Override public String toString() {
+ return "Item [" +
+ "start=" + start +
+ ", delta=" + delta +
+ ']';
+ }
+
+ /** */
+ public long start() {
+ return start;
+ }
+
+ /** */
+ public long delta() {
+ return delta;
+ }
+
+ /** */
+ public long absolute() {
+ return start + delta;
+ }
+
+ /** */
+ public boolean within(long cntr) {
+ return cntr - start < delta;
+ }
+
+ /** {@inheritDoc} */
+ @Override public boolean equals(Object o) {
+ if (this == o)
+ return true;
+ if (o == null || getClass() != o.getClass())
+ return false;
+
+ Item item = (Item)o;
+
+ if (start != item.start)
+ return false;
+
+ return (delta != item.delta);
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public boolean equals(Object o) {
+ if (this == o)
+ return true;
+ if (o == null || getClass() != o.getClass())
+ return false;
+
+ PartitionTxUpdateCounterImpl cntr = (PartitionTxUpdateCounterImpl)o;
+
+ if (!queue.equals(cntr.queue))
+ return false;
+
+ return this.cntr.get() == cntr.cntr.get();
+ }
+
+ /** {@inheritDoc} */
+ @Override public long reserved() {
+ return reserveCntr.get();
+ }
+
+ /** {@inheritDoc} */
+ @Override public synchronized boolean empty() {
+ return get() == 0 && sequential();
+ }
+
+ /** {@inheritDoc} */
+ @Override public Iterator<long[]> iterator() {
+ return F.iterator(queue.iterator(), item -> {
+ return new long[] {item.start, item.delta};
+ }, true);
+ }
+
+ /** {@inheritDoc} */
+ @Override public String toString() {
+ return "Counter [lwm=" + get() + ", holes=" + queue +
+ ", maxApplied=" + highestAppliedCounter() + ", hwm=" + reserveCntr.get() + ']';
+ }
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionUpdateCounter.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionUpdateCounter.java
old mode 100644
new mode 100755
index 39d8d5f..64aff27
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionUpdateCounter.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/PartitionUpdateCounter.java
@@ -17,219 +17,116 @@
package org.apache.ignite.internal.processors.cache;
-import java.util.TreeSet;
-import java.util.concurrent.atomic.AtomicLong;
-import org.apache.ignite.IgniteLogger;
+import java.util.Iterator;
+import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.internal.util.GridLongList;
-import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.Nullable;
/**
- * Partition update counter with MVCC delta updates capabilities.
+ * Partition update counter maintains three entities for tracking partition update state.
+ <ol>
+ * <li><b>Low water mark (LWM)</b> or update counter - lowest applied sequential update number.</li>
+ * <li><b>High water mark (HWM)</b> or reservation counter - highest seen but unapplied yet update number.</li>
+ * <li>Out-of-order applied updates in range between LWM and HWM.</li>
+ * </ol>
*/
-public class PartitionUpdateCounter {
- /** */
- private IgniteLogger log;
-
- /** Queue of counter update tasks*/
- private final TreeSet<Item> queue = new TreeSet<>();
-
- /** Counter. */
- private final AtomicLong cntr = new AtomicLong();
-
- /** Initial counter. */
- private long initCntr;
-
+public interface PartitionUpdateCounter extends Iterable<long[]> {
/**
- * @param log Logger.
+ * Restores update counter state.
+ *
+ * @param initUpdCntr LWM.
+ * @param cntrUpdData Counter updates raw data.
*/
- PartitionUpdateCounter(IgniteLogger log) {
- this.log = log;
- }
+ public void init(long initUpdCntr, @Nullable byte[] cntrUpdData);
/**
- * Sets init counter.
- *
- * @param updateCntr Init counter valus.
+ * @deprecated TODO LWM should be used as initial counter https://ggsystems.atlassian.net/browse/GG-17396
*/
- public void init(long updateCntr) {
- initCntr = updateCntr;
+ public long initial();
- cntr.set(updateCntr);
- }
+ /**
+ * Get LWM.
+ */
+ public long get();
/**
- * @return Initial counter value.
+ * Increment LWM by 1.
+ *
+ * @return New LWM.
*/
- public long initial() {
- return initCntr;
- }
+ public long next();
/**
- * @return Current update counter value.
+ * Increment LWM by delta.
+ *
+ * @param delta Delta.
*/
- public long get() {
- return cntr.get();
- }
+ public long next(long delta);
/**
- * Adds delta to current counter value.
+ * Increment HWM by delta.
*
* @param delta Delta.
- * @return Value before add.
+ * @return New HWM.
*/
- public long getAndAdd(long delta) {
- return cntr.getAndAdd(delta);
- }
+ public long reserve(long delta);
/**
- * @return Next update counter.
+ * Returns HWM.
*/
- public long next() {
- return cntr.incrementAndGet();
- }
+ public long reserved();
/**
- * Sets value to update counter,
+ * Sets update counter to absolute value. All missed updates will be discarded.
*
- * @param val Values.
+ * @param val Absolute value.
+ * @throws IgniteCheckedException if counter cannot be set to passed value due to incompatibility with current state.
*/
- public void update(long val) {
- while (true) {
- long val0 = cntr.get();
-
- if (val0 >= val)
- break;
-
- if (cntr.compareAndSet(val0, val))
- break;
- }
- }
+ public void update(long val) throws IgniteCheckedException;
/**
- * Updates counter by delta from start position.
+ * Applies counter update out of range. Update ranges must not intersect.
*
- * @param start Start.
+ * @param start Start (<= lwm).
* @param delta Delta.
+ * @return {@code True} if update was actually applied.
*/
- public synchronized void update(long start, long delta) {
- long cur = cntr.get(), next;
-
- if (cur > start) {
- log.warning("Stale update counter task [cur=" + cur + ", start=" + start + ", delta=" + delta + ']');
-
- return;
- }
-
- if (cur < start) {
- // backup node with gaps
- offer(new Item(start, delta));
-
- return;
- }
-
- while (true) {
- boolean res = cntr.compareAndSet(cur, next = start + delta);
-
- assert res;
-
- Item peek = peek();
-
- if (peek == null || peek.start != next)
- return;
-
- Item item = poll();
-
- assert peek == item;
-
- start = item.start;
- delta = item.delta;
- cur = next;
- }
- }
+ public boolean update(long start, long delta);
/**
- * @param cntr Sets initial counter.
+ * Reset counter internal state to zero.
*/
- public void updateInitial(long cntr) {
- if (get() < cntr)
- update(cntr);
-
- initCntr = cntr;
- }
+ public void reset();
/**
- * @return Retrieves the minimum update counter task from queue.
+ * @param start Counter.
+ * @param delta Delta.
+ * @deprecated TODO https://ggsystems.atlassian.net/browse/GG-17396
*/
- private Item poll() {
- return queue.pollFirst();
- }
+ public void updateInitial(long start, long delta);
/**
- * @return Checks the minimum update counter task from queue.
+ * Flushes pending update counters closing all possible gaps.
+ *
+ * @return Even-length array of pairs [start, end] for each gap.
*/
- private Item peek() {
- return queue.isEmpty() ? null : queue.first();
+ public GridLongList finalizeUpdateCounters();
- }
+ /** */
+ public @Nullable byte[] getBytes();
/**
- * @param item Adds update task to priority queue.
+ * @return {@code True} if counter has no missed updates.
*/
- private void offer(Item item) {
- queue.add(item);
- }
+ public boolean sequential();
/**
- * Flushes pending update counters closing all possible gaps.
- *
- * @return Even-length array of pairs [start, end] for each gap.
+ * @return {@code True} if counter has not seen any update.
*/
- public synchronized GridLongList finalizeUpdateCounters() {
- Item item = poll();
-
- GridLongList gaps = null;
-
- while (item != null) {
- if (gaps == null)
- gaps = new GridLongList((queue.size() + 1) * 2);
-
- long start = cntr.get() + 1;
- long end = item.start;
-
- gaps.add(start);
- gaps.add(end);
-
- // Close pending ranges.
- update(item.start + item.delta);
-
- item = poll();
- }
-
- return gaps;
- }
+ public boolean empty();
/**
- * Update counter task. Update from start value by delta value.
+ * @return Iterator for pairs [start, delta] for each out-of-order update in the update counter sequence.
*/
- private static class Item implements Comparable<Item> {
- /** */
- private final long start;
-
- /** */
- private final long delta;
-
- /**
- * @param start Start value.
- * @param delta Delta value.
- */
- private Item(long start, long delta) {
- this.start = start;
- this.delta = delta;
- }
-
- /** {@inheritDoc} */
- @Override public int compareTo(@NotNull Item o) {
- return Long.compare(this.start, o.start);
- }
- }
+ @Override public Iterator<long[]> iterator();
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/WalStateManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/WalStateManager.java
index 9d962db..335caf3 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/WalStateManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/WalStateManager.java
@@ -417,7 +417,7 @@ public class WalStateManager extends GridCacheSharedManagerAdapter {
if (hasNonEmptyOwning)
break;
- if (locPart.updateCounter() > 0) {
+ if (!locPart.isEmpty()) {
hasNonEmptyOwning = true;
break;
@@ -435,9 +435,8 @@ public class WalStateManager extends GridCacheSharedManagerAdapter {
", grpId=" + grp.groupId() + ", hasOwning=" + hasOwning + ", hasMoving=" + hasMoving +
", WALState=" + grp.walEnabled() + ", parts=" + parts);
- if (hasOwning && !grp.localWalEnabled()) {
+ if (hasOwning && !grp.localWalEnabled())
grpsToEnableWal.add(grp.groupId());
- }
else if (hasMoving && !hasOwning && grp.localWalEnabled()) {
grpsToDisableWal.add(grp.groupId());
@@ -498,10 +497,13 @@ public class WalStateManager extends GridCacheSharedManagerAdapter {
tmpDisabledWal = null;
}
+ // Pending updates in groups with disabled WAL are not protected from crash.
+ // Need to trigger checkpoint for attempt to persist them.
CheckpointFuture cpFut = triggerCheckpoint("wal-local-state-changed-rebalance-finished-" + topVer);
assert cpFut != null;
+ // It's safe to switch partitions to owning state only if checkpoint was successfully finished.
cpFut.finishFuture().listen(new IgniteInClosureX<IgniteInternalFuture>() {
@Override public void applyx(IgniteInternalFuture future) {
for (Integer grpId0 : session0.disabledGrps) {
@@ -511,12 +513,12 @@ public class WalStateManager extends GridCacheSharedManagerAdapter {
grp.topology().ownMoving(topVer);
else if (log.isDebugEnabled())
log.debug("Cache group was destroyed before checkpoint finished, [grpId=" + grpId0 + ']');
-
}
if (log.isDebugEnabled())
log.debug("Refresh partitions due to rebalance finished");
+ // Trigger exchange for switching to ideal assignment when all nodes are ready.
cctx.exchange().refreshPartitions();
}
});
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/GridDistributedTxRemoteAdapter.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/GridDistributedTxRemoteAdapter.java
index 3e68306..22e3dd5 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/GridDistributedTxRemoteAdapter.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/GridDistributedTxRemoteAdapter.java
@@ -915,12 +915,11 @@ public abstract class GridDistributedTxRemoteAdapter extends IgniteTxAdapter
TxCounters counters = txCounters(false);
if (counters != null)
- cctx.tm().txHandler().applyPartitionsUpdatesCounters(counters.updateCounters());
+ cctx.tm().txHandler().applyPartitionsUpdatesCounters(counters.updateCounters(), true, false);
state(ROLLED_BACK);
cctx.mvccCaching().onTxFinished(this, false);
-
}
}
catch (IgniteCheckedException | RuntimeException | Error e) {
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtCacheEntry.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtCacheEntry.java
index a1eb01c..584ede2 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtCacheEntry.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtCacheEntry.java
@@ -47,6 +47,7 @@ import org.apache.ignite.internal.util.tostring.GridToStringInclude;
import org.apache.ignite.internal.util.typedef.C1;
import org.apache.ignite.internal.util.typedef.CI1;
import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.internal.CU;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.lang.IgniteBiTuple;
import org.apache.ignite.lang.IgniteClosure;
@@ -98,8 +99,33 @@ public class GridDhtCacheEntry extends GridDistributedCacheEntry {
/** {@inheritDoc} */
@Override protected long nextPartitionCounter(AffinityTopologyVersion topVer,
boolean primary,
+ boolean init,
@Nullable Long primaryCntr) {
- return locPart.nextUpdateCounter(cctx.cacheId(), topVer, primary, primaryCntr);
+ try {
+ return locPart.nextUpdateCounter(cctx.cacheId(), topVer, primary, init, primaryCntr);
+ }
+ catch (Throwable t) {
+ log.error("Failed to update counter for atomic cache [" +
+ ", initial=" + init +
+ ", primaryCntr=" + primaryCntr +
+ ", part=" + locPart + ']', t);
+
+ throw t;
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override protected long nextPartitionCounter(IgniteInternalTx tx, @Nullable Long primaryCntr) {
+ try {
+ return locPart.nextUpdateCounter(cctx.cacheId(), tx, primaryCntr);
+ }
+ catch (Throwable t) {
+ log.error("Failed to update counter for tx cache [" +
+ ", primaryCntr=" + primaryCntr +
+ ", part=" + locPart + ", tx=" + CU.txString(tx) + ']', t);
+
+ throw t;
+ }
}
/** {@inheritDoc} */
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishFuture.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishFuture.java
index b00ad56..dc29ebe 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishFuture.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishFuture.java
@@ -17,7 +17,6 @@
package org.apache.ignite.internal.processors.cache.distributed.dht;
-import java.util.ArrayList;
import java.util.Collection;
import java.util.Map;
import java.util.UUID;
@@ -38,7 +37,6 @@ import org.apache.ignite.internal.processors.cache.distributed.GridDistributedTx
import org.apache.ignite.internal.processors.cache.mvcc.MvccFuture;
import org.apache.ignite.internal.processors.cache.mvcc.MvccSnapshot;
import org.apache.ignite.internal.processors.cache.transactions.IgniteInternalTx;
-import org.apache.ignite.internal.processors.cache.transactions.IgniteTxEntry;
import org.apache.ignite.internal.processors.cache.version.GridCacheVersion;
import org.apache.ignite.internal.util.future.GridFutureAdapter;
import org.apache.ignite.internal.util.tostring.GridToStringExclude;
@@ -437,11 +435,6 @@ public final class GridDhtTxFinishFuture<K, V> extends GridCacheCompoundIdentity
add(fut); // Append new future.
- Collection<Long> updCntrs = new ArrayList<>(dhtMapping.entries().size());
-
- for (IgniteTxEntry e : dhtMapping.entries())
- updCntrs.add(e.updateCounter());
-
GridDhtTxFinishRequest req = new GridDhtTxFinishRequest(
tx.nearNodeId(),
futId,
@@ -465,7 +458,7 @@ public final class GridDhtTxFinishFuture<K, V> extends GridCacheCompoundIdentity
tx.subjectId(),
tx.taskNameHash(),
tx.activeCachesDeploymentEnabled(),
- updCntrs,
+ null,
false,
false,
mvccSnapshot,
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishRequest.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishRequest.java
index 04f411d..9498e32 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishRequest.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxFinishRequest.java
@@ -257,13 +257,6 @@ public class GridDhtTxFinishRequest extends GridDistributedTxFinishRequest {
waitRemoteTxs,
mvccSnapshot,
updCntrs);
-
- if (updateIdxs != null && !updateIdxs.isEmpty()) {
- partUpdateCnt = new GridLongList(updateIdxs.size());
-
- for (Long idx : updateIdxs)
- partUpdateCnt.add(idx);
- }
}
/**
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java
index fb485a5..83edb29 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java
@@ -71,6 +71,7 @@ import org.apache.ignite.internal.processors.cache.mvcc.txlog.TxState;
import org.apache.ignite.internal.processors.cache.transactions.IgniteInternalTx;
import org.apache.ignite.internal.processors.cache.transactions.IgniteTxEntry;
import org.apache.ignite.internal.processors.cache.transactions.IgniteTxKey;
+import org.apache.ignite.internal.processors.cache.transactions.TxCounters;
import org.apache.ignite.internal.processors.cache.version.GridCacheVersion;
import org.apache.ignite.internal.processors.dr.GridDrType;
import org.apache.ignite.internal.processors.timeout.GridTimeoutObjectAdapter;
@@ -1258,6 +1259,8 @@ public final class GridDhtTxPrepareFuture extends GridCacheCompoundFuture<Ignite
*
*/
private void prepare0() {
+ boolean error = false;
+
try {
if (tx.serializable() && tx.optimistic()) {
IgniteCheckedException err0;
@@ -1297,10 +1300,24 @@ public final class GridDhtTxPrepareFuture extends GridCacheCompoundFuture<Ignite
// We are holding transaction-level locks for entries here, so we can get next write version.
tx.writeVersion(cctx.versions().next(tx.topologyVersion()));
+ TxCounters counters = tx.txCounters(true);
+
// Assign keys to primary nodes.
if (!F.isEmpty(req.writes())) {
- for (IgniteTxEntry write : req.writes())
- map(tx.entry(write.txKey()));
+ for (IgniteTxEntry write : req.writes()) {
+ IgniteTxEntry entry = tx.entry(write.txKey());
+
+ assert entry != null && entry.cached() != null : entry;
+
+ // Counter shouldn't be reserved for mvcc, local cache entries, NOOP operations and NOOP transforms.
+ if (!entry.cached().isLocal() && entry.op() != NOOP &&
+ !(entry.op() == TRANSFORM &&
+ (entry.entryProcessorCalculatedValue() == null || // Possible for txs over cachestore
+ entry.entryProcessorCalculatedValue().get1() == NOOP)))
+ counters.incrementUpdateCounter(entry.cacheId(), entry.cached().partition());
+
+ map(entry);
+ }
}
if (!F.isEmpty(req.reads())) {
@@ -1312,6 +1329,12 @@ public final class GridDhtTxPrepareFuture extends GridCacheCompoundFuture<Ignite
return;
if (last) {
+ if (!tx.txState().mvccEnabled()) {
+ /** For MVCC counters are assigned on enlisting. */
+ /** See usage of {@link TxCounters#incrementUpdateCounter(int, int)} ) */
+ tx.calculatePartitionUpdateCounters();
+ }
+
recheckOnePhaseCommit();
if (tx.onePhaseCommit())
@@ -1320,8 +1343,14 @@ public final class GridDhtTxPrepareFuture extends GridCacheCompoundFuture<Ignite
sendPrepareRequests();
}
}
+ catch (Throwable t) {
+ error = true;
+
+ throw t;
+ }
finally {
- markInitialized();
+ if (!error) // Prevent marking future as initialized on error.
+ markInitialized();
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/PartitionUpdateCountersMessage.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/PartitionUpdateCountersMessage.java
index f30d51d..ba7b590 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/PartitionUpdateCountersMessage.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/PartitionUpdateCountersMessage.java
@@ -19,9 +19,11 @@ package org.apache.ignite.internal.processors.cache.distributed.dht;
import java.nio.ByteBuffer;
import java.util.Arrays;
+import java.util.Map;
import org.apache.ignite.internal.GridDirectTransient;
import org.apache.ignite.internal.IgniteCodeGeneratingFail;
import org.apache.ignite.internal.util.GridUnsafe;
+import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.plugin.extensions.communication.Message;
import org.apache.ignite.plugin.extensions.communication.MessageReader;
import org.apache.ignite.plugin.extensions.communication.MessageWriter;
@@ -47,6 +49,10 @@ public class PartitionUpdateCountersMessage implements Message {
@GridDirectTransient
private int size;
+ /** Used for assigning counters to cache entries during tx finish. */
+ @GridDirectTransient
+ private Map<Integer, Long> counters;
+
/** */
public PartitionUpdateCountersMessage() {
// No-op.
@@ -145,6 +151,24 @@ public class PartitionUpdateCountersMessage implements Message {
}
/**
+ * Calculate next counter for partition.
+ *
+ * @param partId Partition id.
+ *
+ * @return Next counter for partition.
+ */
+ public Long nextCounter(int partId) {
+ if (counters == null) {
+ counters = U.newHashMap(size);
+
+ for (int i = 0; i < size; i++)
+ counters.put(partition(i), initialCounter(i));
+ }
+
+ return counters.computeIfPresent(partId, (key, cntr) -> cntr + 1);
+ }
+
+ /**
* Clears message.
*/
public void clear() {
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java
index 5c617a8..ac28c0c 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java
@@ -1025,8 +1025,10 @@ public class GridDhtPartitionDemander {
try {
cached = cctx.cache().entryEx(entry.key(), topVer);
- if (log.isTraceEnabled())
- log.trace("Rebalancing key [key=" + entry.key() + ", part=" + p + ", node=" + from.id() + ']');
+ if (log.isTraceEnabled()) {
+ log.trace("Rebalancing key [key=" + entry.key() + ", part=" + p + ", fromNode=" +
+ from.id() + ", grpId=" + grp.groupId() + ']');
+ }
if (preloadPred == null || preloadPred.apply(entry)) {
if (cached.initialValue(
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionSupplier.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionSupplier.java
index a9a2027..c03cc30 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionSupplier.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionSupplier.java
@@ -445,6 +445,15 @@ class GridDhtPartitionSupplier {
log.info("Finished supplying rebalancing [" + supplyRoutineInfo(topicId, nodeId, demandMsg) + "]");
}
catch (Throwable t) {
+ if (iter != null && !iter.isClosed()) {
+ try {
+ iter.close();
+ }
+ catch (IgniteCheckedException e) {
+ t.addSuppressed(e);
+ }
+ }
+
if (grp.shared().kernalContext().isStopping())
return;
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
index b975295..561653a 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionsExchangeFuture.java
@@ -295,7 +295,7 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
@GridToStringExclude
private volatile IgniteDhtPartitionHistorySuppliersMap partHistSuppliers = new IgniteDhtPartitionHistorySuppliersMap();
- /** */
+ /** Reserved max available history for calculation of history supplier on coordinator. */
private volatile Map<Integer, Map<Integer, Long>> partHistReserved;
/** */
@@ -352,6 +352,9 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
/** Discovery lag / Clocks discrepancy, calculated on coordinator when all single messages are received. */
private T2<Long, UUID> discoveryLag;
+ /** TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11799 */
+ private Map<Integer, Set<Integer>> clearingPartitions;
+
/**
* @param cctx Cache context.
* @param busyLock Busy lock.
@@ -1050,6 +1053,7 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
clientTop.fullUpdateCounters(),
Collections.emptySet(),
null,
+ null,
null);
}
finally {
@@ -1433,6 +1437,8 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
cctx.exchange().exchangerBlockingSectionEnd();
}
+ clearingPartitions = new HashMap();
+
timeBag.finishGlobalStage("WAL history reservation");
// Skipping wait on local join is available when all cluster nodes have the same protocol.
@@ -1671,7 +1677,7 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
cctx.exchange().exchangerBlockingSectionBegin();
try {
- // This avoids unnessesary waiting for rollback.
+ // This avoids unnecessary waiting for rollback.
partReleaseFut.get(curTimeout > 0 && !txRolledBack ?
Math.min(curTimeout, waitTimeout) : waitTimeout, TimeUnit.MILLISECONDS);
@@ -3116,10 +3122,10 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
continue;
if (localReserved != null) {
- Long localCntr = localReserved.get(p);
+ Long localHistCntr = localReserved.get(p);
- if (localCntr != null && localCntr <= minCntr && maxCntrObj.nodes.contains(cctx.localNodeId())) {
- partHistSuppliers.put(cctx.localNodeId(), top.groupId(), p, localCntr);
+ if (localHistCntr != null && localHistCntr <= minCntr && maxCntrObj.nodes.contains(cctx.localNodeId())) {
+ partHistSuppliers.put(cctx.localNodeId(), top.groupId(), p, localHistCntr);
haveHistory.add(p);
@@ -3150,7 +3156,7 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
top.globalPartSizes(partSizes);
- Map<UUID, Set<Integer>> partitionsToRebalance = top.resetOwners(ownersByUpdCounters, haveHistory);
+ Map<UUID, Set<Integer>> partitionsToRebalance = top.resetOwners(ownersByUpdCounters, haveHistory, this);
for (Map.Entry<UUID, Set<Integer>> e : partitionsToRebalance.entrySet()) {
UUID nodeId = e.getKey();
@@ -3457,13 +3463,8 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
&& ((SnapshotDiscoveryMessage)discoveryCustomMessage).needAssignPartitions())
assignPartitionsStates();
}
- else {
- if (exchCtx.events().hasServerJoin())
- assignPartitionsStates();
-
- if (exchCtx.events().hasServerLeft())
- detectLostPartitions(resTopVer);
- }
+ else if (exchCtx.events().hasServerJoin())
+ assignPartitionsStates();
// Recalculate new affinity based on partitions availability.
if (!exchCtx.mergeExchanges() && forceAffReassignment) {
@@ -3488,6 +3489,12 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
GridDhtPartitionsFullMessage msg = createPartitionsMessage(true,
minVer.compareToIgnoreTimestamp(PARTIAL_COUNTERS_MAP_SINCE) >= 0);
+ // Lost partition detection should be done after full message is prepared otherwise in case of IGNORE policy
+ // lost partitions will be moved to OWNING state and after what send to other nodes resulting in
+ // wrong lost state calculation (another possibility to consider - calculate lost state only on coordinator).
+ if (firstDiscoEvt.type() != EVT_DISCOVERY_CUSTOM_EVT && exchCtx.events().hasServerLeft())
+ detectLostPartitions(resTopVer);
+
if (exchCtx.mergeExchanges()) {
assert !centralizedAff;
@@ -3776,7 +3783,7 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
int parallelismLvl = U.availableThreadCount(cctx.kernalContext(), GridIoPolicy.SYSTEM_POOL, 2);
try {
- U.<CacheGroupContext, Void>doInParallel(
+ U.<CacheGroupContext, Void>doInParallelUninterruptibly(
parallelismLvl,
cctx.kernalContext().getSystemExecutorService(),
nonLocalCacheGroups(),
@@ -4219,7 +4226,8 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
cntrMap,
msg.partsToReload(cctx.localNodeId(), grpId),
partsSizes.getOrDefault(grpId, Collections.emptyMap()),
- null);
+ null,
+ this);
}
else {
GridDhtPartitionTopology top = cctx.exchange().clientTopology(grpId, events().discoveryCache());
@@ -4232,7 +4240,8 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
cntrMap,
Collections.emptySet(),
null,
- null);
+ null,
+ this);
}
return null;
@@ -5024,6 +5033,40 @@ public class GridDhtPartitionsExchangeFuture extends GridDhtTopologyFutureAdapte
}
/**
+ * If partition is clearing or already cleared we need full rebalance even if supplier is exists.
+ * (it still could be used by other demanders)
+ *
+ * @param grp Group.
+ * @param part Partition.
+ */
+ public boolean isClearingPartition(CacheGroupContext grp, int part) {
+ if (!grp.persistenceEnabled())
+ return false;
+
+ synchronized (mux) {
+ if (clearingPartitions == null)
+ return false;
+
+ Set<Integer> parts = clearingPartitions.get(grp.groupId());
+
+ return parts != null && parts.contains(part);
+ }
+ }
+
+ /**
+ * @param grp Group.
+ * @param part Partition.
+ */
+ public void addClearingPartition(CacheGroupContext grp, int part) {
+ if (!grp.persistenceEnabled())
+ return;
+
+ synchronized (mux) {
+ clearingPartitions.computeIfAbsent(grp.groupId(), k -> new HashSet()).add(part);
+ }
+ }
+
+ /**
*
*/
private static class FinishState {
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloader.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloader.java
index 8d7ce59..a40d08d 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloader.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloader.java
@@ -254,6 +254,10 @@ public class GridDhtPreloader extends GridCachePreloaderAdapter {
if (part.state() == RENTING) {
if (part.reserve()) {
part.moving();
+
+ if (exchFut != null)
+ exchFut.addClearingPartition(grp, part.id());
+
part.clearAsync();
part.release();
@@ -278,7 +282,8 @@ public class GridDhtPreloader extends GridCachePreloaderAdapter {
histSupplier = ctx.discovery().node(nodeId);
}
- if (histSupplier != null) {
+ // Clearing partition should always be fully reloaded.
+ if (histSupplier != null && !exchFut.isClearingPartition(grp, p)) {
assert grp.persistenceEnabled();
assert remoteOwners(p, topVer).contains(histSupplier) : remoteOwners(p, topVer);
@@ -292,6 +297,7 @@ public class GridDhtPreloader extends GridCachePreloaderAdapter {
);
}
+ // TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11790
msg.partitions().addHistorical(p, part.initialUpdateCounter(), countersMap.updateCounter(p), partitions);
}
else {
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloaderAssignments.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloaderAssignments.java
index 6f98889..8783c73 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloaderAssignments.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPreloaderAssignments.java
@@ -81,7 +81,6 @@ public class GridDhtPreloaderAssignments extends ConcurrentHashMap<ClusterNode,
/** {@inheritDoc} */
@Override public String toString() {
- return S.toString(GridDhtPreloaderAssignments.class, this, "exchId", exchangeId,
- "super", super.toString());
+ return S.toString(GridDhtPreloaderAssignments.class, this, "super", super.toString());
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/IgniteDhtDemandedPartitionsMap.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/IgniteDhtDemandedPartitionsMap.java
index 4db8795..ea14597 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/IgniteDhtDemandedPartitionsMap.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/IgniteDhtDemandedPartitionsMap.java
@@ -22,6 +22,7 @@ import java.io.Serializable;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
+import org.apache.ignite.internal.util.tostring.GridToStringInclude;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.jetbrains.annotations.Nullable;
@@ -36,6 +37,7 @@ public class IgniteDhtDemandedPartitionsMap implements Serializable {
private CachePartitionPartialCountersMap historical;
/** Set of partitions that will be preloaded from all it's current data. */
+ @GridToStringInclude
private Set<Integer> full;
public IgniteDhtDemandedPartitionsMap(
@@ -46,6 +48,7 @@ public class IgniteDhtDemandedPartitionsMap implements Serializable {
}
public IgniteDhtDemandedPartitionsMap() {
+ // No-op.
}
/**
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridClientPartitionTopology.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridClientPartitionTopology.java
index 1bae941..c094b6f 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridClientPartitionTopology.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridClientPartitionTopology.java
@@ -729,7 +729,8 @@ public class GridClientPartitionTopology implements GridDhtPartitionTopology {
@Nullable CachePartitionFullCountersMap cntrMap,
Set<Integer> partsToReload,
@Nullable Map<Integer, Long> partSizes,
- @Nullable AffinityTopologyVersion msgTopVer) {
+ @Nullable AffinityTopologyVersion msgTopVer,
+ @Nullable GridDhtPartitionsExchangeFuture exchFut) {
if (log.isDebugEnabled())
log.debug("Updating full partition map [exchVer=" + exchangeVer + ", parts=" + fullMapString() + ']');
@@ -1162,7 +1163,9 @@ public class GridClientPartitionTopology implements GridDhtPartitionTopology {
}
/** {@inheritDoc} */
- @Override public Map<UUID, Set<Integer>> resetOwners(Map<Integer, Set<UUID>> ownersByUpdCounters, Set<Integer> haveHistory) {
+ @Override public Map<UUID, Set<Integer>> resetOwners(Map<Integer, Set<UUID>> ownersByUpdCounters,
+ Set<Integer> haveHistory,
+ GridDhtPartitionsExchangeFuture exchFut) {
Map<UUID, Set<Integer>> result = new HashMap<>();
lock.writeLock().lock();
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtLocalPartition.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtLocalPartition.java
index c1f742e..a131a21 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtLocalPartition.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtLocalPartition.java
@@ -53,12 +53,15 @@ import org.apache.ignite.internal.processors.cache.distributed.dht.GridReservabl
import org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader;
import org.apache.ignite.internal.processors.cache.extras.GridCacheObsoleteEntryExtras;
import org.apache.ignite.internal.processors.cache.persistence.CacheDataRow;
+import org.apache.ignite.internal.processors.cache.transactions.IgniteInternalTx;
+import org.apache.ignite.internal.processors.cache.transactions.TxCounters;
import org.apache.ignite.internal.processors.cache.version.GridCacheVersion;
import org.apache.ignite.internal.processors.query.GridQueryRowCacheCleaner;
import org.apache.ignite.internal.util.GridLongList;
import org.apache.ignite.internal.util.future.GridFutureAdapter;
import org.apache.ignite.internal.util.lang.GridIterator;
import org.apache.ignite.internal.util.tostring.GridToStringExclude;
+import org.apache.ignite.internal.util.typedef.internal.LT;
import org.apache.ignite.internal.util.typedef.internal.S;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.lang.IgniteInClosure;
@@ -218,7 +221,7 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
// Log partition creation for further crash recovery purposes.
if (grp.walEnabled() && !recovery)
- ctx.wal().log(new PartitionMetaStateRecord(grp.groupId(), id, state(), updateCounter()));
+ ctx.wal().log(new PartitionMetaStateRecord(grp.groupId(), id, state(), 0));
// Inject row cache cleaner on store creation
// Used in case the cache with enabled SqlOnheapCache is single cache at the cache group
@@ -396,9 +399,21 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
}
/**
- *
+ * TODO FIXME Get rid of deferred delete queue https://issues.apache.org/jira/browse/IGNITE-11704
*/
public void cleanupRemoveQueue() {
+ if (state() == MOVING) {
+ if (rmvQueue.sizex() >= rmvQueueMaxSize) {
+ LT.warn(log, "Deletion queue cleanup for moving partition was delayed until rebalance is finished. " +
+ "[grpId=" + this.grp.groupId() +
+ ", partId=" + id() +
+ ", grpParts=" + this.grp.affinity().partitions() +
+ ", maxRmvQueueSize=" + rmvQueueMaxSize + ']');
+ }
+
+ return;
+ }
+
while (rmvQueue.sizex() >= rmvQueueMaxSize) {
RemovedEntryHolder item = rmvQueue.pollFirst();
@@ -535,7 +550,7 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
this.state.compareAndSet(state0, setPartState(state0, toState));
try {
- ctx.wal().log(new PartitionMetaStateRecord(grp.groupId(), id, toState, updateCounter()));
+ ctx.wal().log(new PartitionMetaStateRecord(grp.groupId(), id, toState, 0));
}
catch (IgniteCheckedException e) {
U.error(log, "Error while writing to log", e);
@@ -560,7 +575,7 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
if (update) {
try {
- ctx.wal().log(new PartitionMetaStateRecord(grp.groupId(), id, toState, updateCounter()));
+ ctx.wal().log(new PartitionMetaStateRecord(grp.groupId(), id, toState, 0));
}
catch (IgniteCheckedException e) {
U.error(log, "Failed to log partition state change to WAL.", e);
@@ -717,7 +732,7 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
}
}
- ctx.evict().evictPartitionAsync(grp,this);
+ ctx.evict().evictPartitionAsync(grp, this);
}
/**
@@ -970,40 +985,82 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
}
/**
+ * Returns new update counter for primary node or passed counter for backup node.
+ * <p>
+ * Used for non-tx cases.
+ * <p>
+ * Counter generation/update logic is delegated to counter implementation.
+ *
* @param cacheId ID of cache initiated counter update.
* @param topVer Topology version for current operation.
+ * @param init {@code True} if initial update.
* @return Next update index.
*/
- public long nextUpdateCounter(int cacheId, AffinityTopologyVersion topVer, boolean primary, @Nullable Long primaryCntr) {
- long nextCntr = store.nextUpdateCounter();
+ public long nextUpdateCounter(int cacheId, AffinityTopologyVersion topVer, boolean primary, boolean init,
+ @Nullable Long primaryCntr) {
+ long nextCntr;
+
+ if (primaryCntr == null) // Primary node.
+ nextCntr = store.nextUpdateCounter();
+ else {
+ assert !init : "Initial update must generate a counter for partition " + this;
+
+ // Backup.
+ assert primaryCntr != 0;
+
+ store.updateCounter(nextCntr = primaryCntr);
+ }
if (grp.sharedGroup())
- grp.onPartitionCounterUpdate(cacheId, id, primaryCntr != null ? primaryCntr : nextCntr, topVer, primary);
+ grp.onPartitionCounterUpdate(cacheId, id, nextCntr, topVer, primary);
- // This is first update in partition, we should log partition state information for further crash recovery.
- if (nextCntr == 1) {
- if (grp.persistenceEnabled() && grp.walEnabled())
- try {
- ctx.wal().log(new PartitionMetaStateRecord(grp.groupId(), id, state(), 0));
- }
- catch (IgniteCheckedException e) {
- U.error(log, "Failed to log partition state snapshot to WAL.", e);
+ return nextCntr;
+ }
- ctx.kernalContext().failure().process(new FailureContext(FailureType.CRITICAL_ERROR, e));
- }
+ /**
+ * Used for transactions.
+ *
+ * @param cacheId Cache id.
+ * @param tx Tx.
+ * @param primaryCntr Primary counter.
+ */
+ public long nextUpdateCounter(int cacheId, IgniteInternalTx tx, @Nullable Long primaryCntr) {
+ Long nextCntr;
+
+ if (primaryCntr != null)
+ nextCntr = primaryCntr;
+ else {
+ TxCounters txCounters = tx.txCounters(false);
+
+ assert txCounters != null : "Must have counters for tx [nearXidVer=" + tx.nearXidVersion() + ']';
+
+ // Null must never be returned on primary node.
+ nextCntr = txCounters.generateNextCounter(cacheId, id());
+
+ assert nextCntr != null : this;
}
+ if (grp.sharedGroup())
+ grp.onPartitionCounterUpdate(cacheId, id, nextCntr, tx.topologyVersion(), tx.local());
+
return nextCntr;
}
/**
- * @return Current update counter.
+ * @return Current update counter (LWM).
*/
public long updateCounter() {
return store.updateCounter();
}
/**
+ * @return Current reserved counter (HWM).
+ */
+ public long reservedCounter() {
+ return store.reservedCounter();
+ }
+
+ /**
* @param val Update counter value.
*/
public void updateCounter(long val) {
@@ -1018,14 +1075,7 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
}
/**
- * @param val Initial update counter value.
- */
- public void initialUpdateCounter(long val) {
- store.updateInitialCounter(val);
- }
-
- /**
- * Updates MVCC cache update counter on primary node.
+ * Increments cache update counter on primary node.
*
* @param delta Value to be added to update counter.
* @return Update counter value before update.
@@ -1040,8 +1090,15 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
* @param start Start position
* @param delta Delta.
*/
- public void updateCounter(long start, long delta) {
- store.updateCounter(start, delta);
+ public boolean updateCounter(long start, long delta) {
+ return store.updateCounter(start, delta);
+ }
+
+ /**
+ * Reset partition counters.
+ */
+ public void resetCounters() {
+ store.resetUpdateCounter();
}
/**
@@ -1086,7 +1143,8 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
try {
CacheDataRow row = it0.next();
- // Do not clear fresh rows in case of single partition clearing.
+ // Do not clear fresh rows in case of partition reloading.
+ // This is required because updates are possible to moving partition which is currently cleared.
if (row.version().compareTo(clearVer) >= 0 && (state() == MOVING && clear))
continue;
@@ -1256,7 +1314,8 @@ public class GridDhtLocalPartition extends GridCacheConcurrentMapImpl implements
"state", state(),
"reservations", reservations(),
"empty", isEmpty(),
- "createTime", U.format(createTime));
+ "createTime", U.format(createTime),
+ "cntr", dataStore().partUpdateCounter());
}
/** {@inheritDoc} */
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopology.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopology.java
index 171ac33..f2b0dd0 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopology.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopology.java
@@ -292,6 +292,7 @@ public interface GridDhtPartitionTopology {
* @param partsToReload Set of partitions that need to be reloaded.
* @param msgTopVer Topology version from incoming message. This value is not null only for case message is not
* related to exchange. Value should be not less than previous 'Topology version from exchange'.
+ * @param exchFut Future which is not null for initial partition update on exchange.
* @return {@code True} if local state was changed.
*/
public boolean update(
@@ -300,7 +301,9 @@ public interface GridDhtPartitionTopology {
@Nullable CachePartitionFullCountersMap cntrMap,
Set<Integer> partsToReload,
@Nullable Map<Integer, Long> partSizes,
- @Nullable AffinityTopologyVersion msgTopVer);
+ @Nullable AffinityTopologyVersion msgTopVer,
+ @Nullable GridDhtPartitionsExchangeFuture exchFut
+ );
/**
* @param exchId Exchange ID.
@@ -359,7 +362,7 @@ public interface GridDhtPartitionTopology {
public CachePartitionFullCountersMap fullUpdateCounters();
/**
- * @param skipZeros {@code True} for adding zero counter to map.
+ * @param skipZeros {@code True} to exclude zero counters from map.
* @return Partition update counters.
*/
public CachePartitionPartialCountersMap localUpdateCounters(boolean skipZeros);
@@ -420,13 +423,16 @@ public interface GridDhtPartitionTopology {
/**
* Calculates nodes and partitions which have non-actual state and must be rebalanced.
* State of all current owners that aren't contained in the given {@code ownersByUpdCounters} will be reset to MOVING.
+ * Called on coordinator during assignment of partition states.
*
* @param ownersByUpdCounters Map (partition, set of node IDs that have most actual state about partition
* (update counter is maximal) and should hold OWNING state for such partition).
* @param haveHistory Set of partitions which have WAL history to rebalance.
+ * @param exchFut Exchange future for operation.
* @return Map (nodeId, set of partitions that should be rebalanced <b>fully</b> by this node).
*/
- public Map<UUID, Set<Integer>> resetOwners(Map<Integer, Set<UUID>> ownersByUpdCounters, Set<Integer> haveHistory);
+ public Map<UUID, Set<Integer>> resetOwners(Map<Integer, Set<UUID>> ownersByUpdCounters, Set<Integer> haveHistory,
+ GridDhtPartitionsExchangeFuture exchFut);
/**
* Callback on exchange done.
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java
index a17321f..b28c9ee 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/GridDhtPartitionTopologyImpl.java
@@ -42,6 +42,7 @@ import org.apache.ignite.events.EventType;
import org.apache.ignite.internal.IgniteInternalFuture;
import org.apache.ignite.internal.IgniteInterruptedCheckedException;
import org.apache.ignite.internal.managers.discovery.DiscoCache;
+import org.apache.ignite.internal.pagemem.wal.record.RollbackRecord;
import org.apache.ignite.internal.processors.affinity.AffinityAssignment;
import org.apache.ignite.internal.processors.affinity.AffinityTopologyVersion;
import org.apache.ignite.internal.processors.cache.CacheGroupContext;
@@ -428,7 +429,7 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
}
}
else
- createPartitions(affVer, affAssignment, updateSeq);
+ createPartitions(affVer, affAssignment, updateSeq, exchFut);
}
else {
// If preloader is disabled, then we simply clear out
@@ -478,8 +479,10 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
* @param affVer Affinity version.
* @param aff Affinity assignments.
* @param updateSeq Update sequence.
+ * @param exchFut
*/
- private void createPartitions(AffinityTopologyVersion affVer, List<List<ClusterNode>> aff, long updateSeq) {
+ private void createPartitions(AffinityTopologyVersion affVer, List<List<ClusterNode>> aff, long updateSeq,
+ GridDhtPartitionsExchangeFuture exchFut) {
if (!grp.affinityNode())
return;
@@ -493,8 +496,13 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
if (localNode(p, aff)) {
// This will make sure that all non-existing partitions
// will be created in MOVING state.
+ boolean existing = locParts.get(p) != null;
+
GridDhtLocalPartition locPart = getOrCreatePartition(p);
+ if (existing && locPart.state() == MOVING && !locPart.isEmpty())
+ exchFut.addClearingPartition(grp, p);
+
updateSeq = updateLocal(p, locPart.state(), updateSeq, affVer);
}
}
@@ -758,7 +766,7 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
if (partitionLocalNode(p, topVer)) {
// Prepare partition to rebalance if it's not happened on full map update phase.
if (locPart == null || locPart.state() == RENTING || locPart.state() == EVICTED)
- locPart = rebalancePartition(p, false);
+ locPart = rebalancePartition(p, true, exchFut);
GridDhtPartitionState state = locPart.state();
@@ -772,6 +780,9 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
log.debug("Will not own partition (there are owners to rebalance from) " +
"[grp=" + grp.cacheOrGroupName() + ", p=" + p + ", owners = " + owners + ']');
}
+
+ if (exchFut.isClearingPartition(grp, p))
+ locPart.clearAsync();
}
else
updateSeq = updateLocal(p, locPart.state(), updateSeq, topVer);
@@ -1359,8 +1370,9 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
*/
private boolean shouldOverridePartitionMap(GridDhtPartitionMap currentMap, GridDhtPartitionMap newMap) {
return newMap != null &&
- (newMap.topologyVersion().compareTo(currentMap.topologyVersion()) > 0 ||
- newMap.topologyVersion().compareTo(currentMap.topologyVersion()) == 0 && newMap.updateSequence() > currentMap.updateSequence());
+ (newMap.topologyVersion().compareTo(currentMap.topologyVersion()) > 0 ||
+ newMap.topologyVersion().compareTo(currentMap.topologyVersion()) == 0 &&
+ newMap.updateSequence() > currentMap.updateSequence());
}
/** {@inheritDoc} */
@@ -1370,7 +1382,8 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
@Nullable CachePartitionFullCountersMap incomeCntrMap,
Set<Integer> partsToReload,
@Nullable Map<Integer, Long> partSizes,
- @Nullable AffinityTopologyVersion msgTopVer) {
+ @Nullable AffinityTopologyVersion msgTopVer,
+ @Nullable GridDhtPartitionsExchangeFuture exchFut) {
if (log.isDebugEnabled()) {
log.debug("Updating full partition map " +
"[grp=" + grp.cacheOrGroupName() + ", exchVer=" + exchangeVer + ", fullMap=" + fullMapString() + ']');
@@ -1408,17 +1421,21 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
long updCntr = incomeCntrMap.updateCounter(part.id());
long curCntr = part.updateCounter();
- if (updCntr != 0 && updCntr > curCntr) {
+ // Avoid zero counter update to empty partition to prevent lazy init.
+ if (updCntr != 0 || curCntr != 0) {
part.updateCounter(updCntr);
- if (log.isDebugEnabled())
- log.debug("Partition update counter has updated [grp=" + grp.cacheOrGroupName() + ", p=" + part.id()
- + ", state=" + part.state() + ", prevCntr=" + curCntr + ", nextCntr=" + updCntr + "]");
+ if (updCntr > curCntr) {
+ if (log.isDebugEnabled())
+ log.debug("Partition update counter has updated [grp=" + grp.cacheOrGroupName() + ", p=" + part.id()
+ + ", state=" + part.state() + ", prevCntr=" + curCntr + ", nextCntr=" + updCntr + "]");
+ }
}
}
}
}
+ // TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11800
if (exchangeVer != null) {
// Ignore if exchange already finished or new exchange started.
if (readyTopVer.compareTo(exchangeVer) > 0 || lastTopChangeVer.compareTo(exchangeVer) > 0) {
@@ -1557,6 +1574,9 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
nodeMap != null &&
grp.persistenceEnabled() &&
readyTopVer.initialized()) {
+
+ assert exchFut != null;
+
for (Map.Entry<Integer, GridDhtPartitionState> e : nodeMap.entrySet()) {
int p = e.getKey();
GridDhtPartitionState state = e.getValue();
@@ -1575,9 +1595,7 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
}
}
else if (state == MOVING) {
- boolean haveHistory = !partsToReload.contains(p);
-
- rebalancePartition(p, haveHistory);
+ rebalancePartition(p, partsToReload.contains(p), exchFut);
changed = true;
}
@@ -1698,12 +1716,21 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
continue;
long updCntr = cntrMap.updateCounter(part.id());
+ long locUpdCntr = part.updateCounter();
- if (updCntr > part.updateCounter())
+ if (updCntr != 0 || locUpdCntr != 0) { // Avoid creation of empty partition.
part.updateCounter(updCntr);
- else if (part.updateCounter() > 0) {
+
+ if (updCntr > locUpdCntr) {
+ if (log.isDebugEnabled())
+ log.debug("Partition update counter has updated [grp=" + grp.cacheOrGroupName() + ", p=" + part.id()
+ + ", state=" + part.state() + ", prevCntr=" + locUpdCntr + ", nextCntr=" + updCntr + "]");
+ }
+ }
+
+ if (locUpdCntr > updCntr) {
cntrMap.initialUpdateCounter(part.id(), part.initialUpdateCounter());
- cntrMap.updateCounter(part.id(), part.updateCounter());
+ cntrMap.updateCounter(part.id(), locUpdCntr);
}
}
finally {
@@ -2105,7 +2132,7 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
if (recentlyLost != null) {
U.warn(log, "Detected lost partitions [grp=" + grp.cacheOrGroupName()
+ ", parts=" + S.compact(recentlyLost)
- + ", plc=" + plc + "]");
+ + ", plc=" + plc + ", topVer=" + resTopVer + "]");
}
if (lostParts != null && plc != PartitionLossPolicy.IGNORE)
@@ -2147,11 +2174,8 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
if (marked) {
updateLocal(locPart.id(), locPart.state(), updSeq, resTopVer);
- long updateCntr = locPart.updateCounter();
-
- //Set update counters to 0, for full rebalance.
- locPart.updateCounter(updateCntr, -updateCntr);
- locPart.initialUpdateCounter(0);
+ // Reset counters to zero for triggering full rebalance.
+ locPart.resetCounters();
}
}
}
@@ -2188,7 +2212,9 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
}
/** {@inheritDoc} */
- @Override public Map<UUID, Set<Integer>> resetOwners(Map<Integer, Set<UUID>> ownersByUpdCounters, Set<Integer> haveHistory) {
+ @Override public Map<UUID, Set<Integer>> resetOwners(Map<Integer, Set<UUID>> ownersByUpdCounters,
+ Set<Integer> haveHistory,
+ GridDhtPartitionsExchangeFuture exchFut) {
Map<UUID, Set<Integer>> result = new HashMap<>();
ctx.database().checkpointReadLock();
@@ -2208,10 +2234,9 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
continue;
if (!newOwners.contains(ctx.localNodeId())) {
- rebalancePartition(part, haveHistory.contains(part));
+ rebalancePartition(part, !haveHistory.contains(part), exchFut);
- result.computeIfAbsent(ctx.localNodeId(), n -> new HashSet<>());
- result.get(ctx.localNodeId()).add(part);
+ result.computeIfAbsent(ctx.localNodeId(), n -> new HashSet<>()).add(part);
}
}
@@ -2237,8 +2262,7 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
if (partMap.nodeId().equals(ctx.localNodeId()))
updateSeq.setIfGreater(partMap.updateSequence());
- result.computeIfAbsent(remoteNodeId, n -> new HashSet<>());
- result.get(remoteNodeId).add(part);
+ result.computeIfAbsent(remoteNodeId, n -> new HashSet<>()).add(part);
}
}
}
@@ -2247,6 +2271,10 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
UUID nodeId = entry.getKey();
Set<Integer> rebalancedParts = entry.getValue();
+ // Add to wait groups to ensure late assignment switch after all partitions are rebalanced.
+ for (Integer part : rebalancedParts)
+ ctx.cache().context().affinity().addToWaitGroup(groupId(), part, nodeId, topologyVersionFuture().initialVersion());
+
if (!rebalancedParts.isEmpty()) {
Set<Integer> historical = rebalancedParts.stream()
.filter(haveHistory::contains)
@@ -2281,10 +2309,11 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
* Prevents ongoing renting if required.
*
* @param p Partition id.
- * @param haveHistory If {@code true} there is WAL history to rebalance partition,
- * in other case partition will be cleared for full rebalance.
+ * @param clear If {@code true} partition have to be cleared before rebalance.
+ * Required in case of full state transfer to handle removals on supplier.
+ * @param exchFut Future related to partition state change.
*/
- private GridDhtLocalPartition rebalancePartition(int p, boolean haveHistory) {
+ private GridDhtLocalPartition rebalancePartition(int p, boolean clear, GridDhtPartitionsExchangeFuture exchFut) {
GridDhtLocalPartition part = getOrCreatePartition(p);
// Prevent renting.
@@ -2303,8 +2332,11 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
if (part.state() != MOVING)
part.moving();
- if (!haveHistory)
+ if (clear) {
+ exchFut.addClearingPartition(grp, part.id());
+
part.clearAsync();
+ }
assert part.state() == MOVING : part;
@@ -2656,9 +2688,6 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
* Pre-processes partition update counters before exchange.
*/
@Override public void finalizeUpdateCounters() {
- if (!grp.mvccEnabled())
- return;
-
// It is need to acquire checkpoint lock before topology lock acquiring.
ctx.database().checkpointReadLock();
@@ -2680,6 +2709,25 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
GridLongList gaps = part.finalizeUpdateCounters();
if (gaps != null) {
+ for (int j = 0; j < gaps.size() / 2; j++) {
+ long gapStart = gaps.get(j * 2);
+ long gapStop = gaps.get(j * 2 + 1);
+
+ if (part.group().persistenceEnabled() &&
+ part.group().walEnabled() &&
+ !part.group().mvccEnabled()) {
+ RollbackRecord rec = new RollbackRecord(part.group().groupId(), part.id(),
+ gapStart - 1, gapStop - gapStart + 1);
+
+ try {
+ ctx.wal().log(rec);
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
+ }
+ }
+
for (GridCacheContext ctx0 : grp.caches())
ctx0.continuousQueries().closeBackupUpdateCountersGaps(ctx0, part.id(), topVer, gaps);
}
@@ -2964,7 +3012,7 @@ public class GridDhtPartitionTopologyImpl implements GridDhtPartitionTopology {
* Checks consistency after all operations.
*/
private void consistencyCheck() {
- // no-op
+ // No-op.
}
/**
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/PartitionsEvictManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/PartitionsEvictManager.java
index 404e194..c458d3e 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/PartitionsEvictManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/topology/PartitionsEvictManager.java
@@ -70,7 +70,7 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
private volatile boolean stop;
/** Check stop eviction context. */
- private final EvictionContext sharedEvictionContext = () -> stop;
+ private final EvictionContext sharedEvictionCtx = () -> stop;
/** Number of maximum concurrent operations. */
private volatile int threads;
@@ -94,13 +94,13 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
*
* @param grp Group context.
*/
- public void onCacheGroupStopped(CacheGroupContext grp){
- GroupEvictionContext groupEvictionContext = evictionGroupsMap.remove(grp.groupId());
+ public void onCacheGroupStopped(CacheGroupContext grp){
+ GroupEvictionContext grpEvictionCtx = evictionGroupsMap.remove(grp.groupId());
- if (groupEvictionContext != null){
- groupEvictionContext.stop();
+ if (grpEvictionCtx != null){
+ grpEvictionCtx.stop();
- groupEvictionContext.awaitFinishAll();
+ grpEvictionCtx.awaitFinishAll();
}
}
@@ -111,23 +111,23 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
* @param part Partition to evict.
*/
public void evictPartitionAsync(CacheGroupContext grp, GridDhtLocalPartition part) {
- GroupEvictionContext groupEvictionContext = evictionGroupsMap.computeIfAbsent(
+ GroupEvictionContext grpEvictionCtx = evictionGroupsMap.computeIfAbsent(
grp.groupId(), (k) -> new GroupEvictionContext(grp));
// Check node stop.
- if (groupEvictionContext.shouldStop())
+ if (grpEvictionCtx.shouldStop())
return;
int bucket;
synchronized (mux) {
- if (!groupEvictionContext.partIds.add(part.id()))
+ if (!grpEvictionCtx.partIds.add(part.id()))
return;
- bucket = evictionQueue.offer(new PartitionEvictionTask(part, groupEvictionContext));
+ bucket = evictionQueue.offer(new PartitionEvictionTask(part, grpEvictionCtx));
}
- groupEvictionContext.totalTasks.incrementAndGet();
+ grpEvictionCtx.totalTasks.incrementAndGet();
if (log.isDebugEnabled())
log.debug("Partition has been scheduled for eviction [grp=" + grp.cacheOrGroupName()
@@ -143,7 +143,7 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
*/
private void scheduleNextPartitionEviction(int bucket) {
// Check node stop.
- if (sharedEvictionContext.shouldStop())
+ if (sharedEvictionCtx.shouldStop())
return;
synchronized (mux) {
@@ -178,17 +178,17 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
// Print current eviction progress.
showProgress();
- GroupEvictionContext groupEvictionContext = evictionTask.groupEvictionCtx;
+ GroupEvictionContext grpEvictionCtx = evictionTask.grpEvictionCtx;
// Check that group or node stopping.
- if (groupEvictionContext.shouldStop())
+ if (grpEvictionCtx.shouldStop())
continue;
// Get permit for this task.
permits--;
// Register task future, may need if group or node will be stopped.
- groupEvictionContext.taskScheduled(evictionTask);
+ grpEvictionCtx.taskScheduled(evictionTask);
evictionTask.finishFut.listen(f -> {
synchronized (mux) {
@@ -294,7 +294,7 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
/** {@inheritDoc} */
@Override public boolean shouldStop() {
- return stop || sharedEvictionContext.shouldStop();
+ return stop || sharedEvictionCtx.shouldStop();
}
/**
@@ -384,35 +384,35 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
private final long size;
/** Eviction context. */
- private final GroupEvictionContext groupEvictionCtx;
+ private final GroupEvictionContext grpEvictionCtx;
/** */
private final GridFutureAdapter<?> finishFut = new GridFutureAdapter<>();
/**
* @param part Partition.
- * @param groupEvictionCtx Eviction context.
+ * @param grpEvictionCtx Eviction context.
*/
private PartitionEvictionTask(
GridDhtLocalPartition part,
- GroupEvictionContext groupEvictionCtx
+ GroupEvictionContext grpEvictionCtx
) {
this.part = part;
- this.groupEvictionCtx = groupEvictionCtx;
+ this.grpEvictionCtx = grpEvictionCtx;
size = part.fullSize();
}
/** {@inheritDoc} */
@Override public void run() {
- if (groupEvictionCtx.shouldStop()) {
+ if (grpEvictionCtx.shouldStop()) {
finishFut.onDone();
return;
}
try {
- boolean success = part.tryClear(groupEvictionCtx);
+ boolean success = part.tryClear(grpEvictionCtx);
if (success) {
if (part.state() == GridDhtPartitionState.EVICTED && part.markForDestroy())
@@ -425,7 +425,7 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
// Re-offer partition if clear was unsuccessful due to partition reservation.
if (!success)
- evictPartitionAsync(groupEvictionCtx.grp, part);
+ evictPartitionAsync(grpEvictionCtx.grp, part);
}
catch (Throwable ex) {
finishFut.onDone(ex);
@@ -435,9 +435,8 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
false,
true);
}
- else{
+ else
LT.error(log, ex, "Partition eviction failed, this can cause grid hang.");
- }
}
}
}
@@ -523,9 +522,8 @@ public class PartitionsEvictManager extends GridCacheSharedManagerAdapter {
int size(){
int size = 0;
- for (Queue<PartitionEvictionTask> queue : buckets) {
+ for (Queue<PartitionEvictionTask> queue : buckets)
size += queue.size();
- }
return size;
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearOptimisticTxPrepareFutureAdapter.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearOptimisticTxPrepareFutureAdapter.java
index 913e9bc..13b03fe 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearOptimisticTxPrepareFutureAdapter.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearOptimisticTxPrepareFutureAdapter.java
@@ -147,7 +147,11 @@ public abstract class GridNearOptimisticTxPrepareFutureAdapter extends GridNearT
}
if (topFut.isDone()) {
- topVer = topFut.topologyVersion();
+ if ((topVer = topFut.topologyVersion()) == null && topFut.error() != null) {
+ onDone(topFut.error()); // Prevent stack overflow if topFut has error.
+
+ return;
+ }
if (remap)
tx.onRemap(topVer);
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearTxLocal.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearTxLocal.java
index b3b43b9..797221e 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearTxLocal.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/near/GridNearTxLocal.java
@@ -3831,13 +3831,24 @@ public class GridNearTxLocal extends GridDhtTxLocalAdapter implements GridTimeou
return chainFinishFuture(finishFut, true, true, false);
if (!fastFinish) {
- final IgniteInternalFuture<?> prepareFut = prepareNearTxLocal();
+ IgniteInternalFuture<?> prepareFut;
+ try {
+ prepareFut = prepareNearTxLocal();
+ }
+ catch (Throwable t) {
+ prepareFut = prepFut;
+
+ // Properly finish prepFut in case of unchecked error.
+ assert prepareFut != null; // Prep future must be set.
+
+ ((GridNearTxPrepareFutureAdapter)prepFut).onDone(t);
+ }
prepareFut.listen(new CI1<IgniteInternalFuture<?>>() {
@Override public void apply(IgniteInternalFuture<?> f) {
try {
// Make sure that here are no exceptions.
- prepareFut.get();
+ f.get();
fut.finish(true, true, false);
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRow.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRow.java
index 92f06a3..746b94a 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRow.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRow.java
@@ -20,6 +20,9 @@ package org.apache.ignite.internal.processors.cache.persistence;
import org.apache.ignite.internal.processors.cache.CacheObject;
import org.apache.ignite.internal.processors.cache.KeyCacheObject;
import org.apache.ignite.internal.processors.cache.mvcc.MvccUpdateVersionAware;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.DataPageIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
import org.apache.ignite.internal.processors.cache.version.GridCacheVersion;
/**
@@ -55,4 +58,9 @@ public interface CacheDataRow extends MvccUpdateVersionAware, CacheSearchRow, St
* @param key Key.
*/
public void key(KeyCacheObject key);
+
+ /** {@inheritDoc} */
+ @Override public default IOVersions<? extends AbstractDataPageIO> ioVersions() {
+ return DataPageIO.VERSIONS;
+ }
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRowAdapter.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRowAdapter.java
index 3e3dc5c..b551f47 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRowAdapter.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/CacheDataRowAdapter.java
@@ -233,7 +233,7 @@ public class CacheDataRowAdapter implements CacheDataRow {
IoStatisticsHolder statHolder,
boolean readCacheId,
RowData rowData,
- IncompleteObject<?> incomplete,
+ @Nullable IncompleteObject<?> incomplete,
boolean skipVer
) throws IgniteCheckedException {
assert link != 0 : "link";
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
index 498ccf1..3c40cdf 100755
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheDatabaseSharedManager.java
@@ -111,6 +111,7 @@ import org.apache.ignite.internal.pagemem.wal.record.MetastoreDataRecord;
import org.apache.ignite.internal.pagemem.wal.record.MvccDataEntry;
import org.apache.ignite.internal.pagemem.wal.record.MvccTxRecord;
import org.apache.ignite.internal.pagemem.wal.record.PageSnapshot;
+import org.apache.ignite.internal.pagemem.wal.record.RollbackRecord;
import org.apache.ignite.internal.pagemem.wal.record.WALRecord;
import org.apache.ignite.internal.pagemem.wal.record.WalRecordCacheGroupAware;
import org.apache.ignite.internal.pagemem.wal.record.delta.PageDeltaRecord;
@@ -142,7 +143,6 @@ import org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemor
import org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl;
import org.apache.ignite.internal.processors.cache.persistence.partstate.GroupPartitionId;
import org.apache.ignite.internal.processors.cache.persistence.partstate.PartitionAllocationMap;
-import org.apache.ignite.internal.processors.cache.persistence.partstate.PartitionRecoverState;
import org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteCacheSnapshotManager;
import org.apache.ignite.internal.processors.cache.persistence.snapshot.SnapshotOperation;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO;
@@ -2730,9 +2730,12 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
Semaphore semaphore = new Semaphore(semaphorePertmits(exec));
+ Map<GroupPartitionId, Integer> partitionRecoveryStates = new HashMap<>();
+
WALIterator it = cctx.wal().replay(status.startPtr, recordTypePredicate);
- RestoreLogicalState restoreLogicalState = new RestoreLogicalState(status, it, lastArchivedSegment, cacheGroupsPredicate);
+ RestoreLogicalState restoreLogicalState =
+ new RestoreLogicalState(status, it, lastArchivedSegment, cacheGroupsPredicate, partitionRecoveryStates);
try {
while (it.hasNextX()) {
@@ -2742,6 +2745,36 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
break;
switch (rec.type()) {
+ case CHECKPOINT_RECORD: // Calculate initial partition states
+ CheckpointRecord cpRec = (CheckpointRecord)rec;
+
+ for (Map.Entry<Integer, CacheState> entry : cpRec.cacheGroupStates().entrySet()) {
+ CacheState cacheState = entry.getValue();
+
+ for (int i = 0; i < cacheState.size(); i++) {
+ int partId = cacheState.partitionByIndex(i);
+ byte state = cacheState.stateByIndex(i);
+
+ partitionRecoveryStates.put(new GroupPartitionId(entry.getKey(), partId), (int)state);
+ }
+ }
+
+ break;
+
+ case ROLLBACK_TX_RECORD:
+ RollbackRecord rbRec = (RollbackRecord)rec;
+
+ CacheGroupContext ctx = cctx.cache().cacheGroup(rbRec.groupId());
+
+ if (ctx != null && !ctx.isLocal()) {
+ ctx.topology().forceCreatePartition(rbRec.partitionId());
+
+ ctx.offheap().onPartitionInitialCounterUpdated(rbRec.partitionId(), rbRec.start(),
+ rbRec.range());
+ }
+
+ break;
+
case MVCC_DATA_RECORD:
case DATA_RECORD:
case ENCRYPTED_DATA_RECORD:
@@ -2773,7 +2806,7 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
}
applied.incrementAndGet();
- }, cacheId, dataEntry.partitionId(), exec, semaphore);
+ }, cacheDesc.groupId(), dataEntry.partitionId(), exec, semaphore);
}
break;
@@ -2794,11 +2827,7 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
metaStateRecord.groupId(), metaStateRecord.partitionId()
);
- PartitionRecoverState state = new PartitionRecoverState(
- (int)metaStateRecord.state(), metaStateRecord.updateCounter()
- );
-
- restoreLogicalState.partitionRecoveryStates.put(groupPartitionId, state);
+ restoreLogicalState.partitionRecoveryStates.put(groupPartitionId, (int)metaStateRecord.state());
break;
@@ -2925,7 +2954,7 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
}
if (dataEntry.partitionCounter() != 0)
- cacheCtx.offheap().onPartitionInitialCounterUpdated(partId, dataEntry.partitionCounter());
+ cacheCtx.offheap().onPartitionInitialCounterUpdated(partId, dataEntry.partitionCounter() - 1, 1);
break;
@@ -2944,7 +2973,7 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
cacheCtx.offheap().remove(cacheCtx, dataEntry.key(), partId, locPart);
if (dataEntry.partitionCounter() != 0)
- cacheCtx.offheap().onPartitionInitialCounterUpdated(partId, dataEntry.partitionCounter());
+ cacheCtx.offheap().onPartitionInitialCounterUpdated(partId, dataEntry.partitionCounter() - 1, 1);
break;
@@ -5347,13 +5376,13 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
assert pageStore != null : "Persistent cache should have initialize page store manager.";
for (int p = 0; p < grp.affinity().partitions(); p++) {
- if (grp.topology().localPartition(p) != null) {
- GridDhtLocalPartition part = grp.topology().localPartition(p);
+ GridDhtLocalPartition part = grp.topology().localPartition(p);
+ if (part != null) {
log.info("Partition [grp=" + grp.cacheOrGroupName()
+ ", id=" + p
+ ", state=" + part.state()
- + ", counter=" + part.updateCounter()
+ + ", counter=" + part.dataStore().partUpdateCounter()
+ ", size=" + part.fullSize() + "]");
continue;
@@ -5379,17 +5408,17 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
try {
PagePartitionMetaIO io = PagePartitionMetaIO.VERSIONS.forPage(pageAddr);
- GridDhtPartitionState partitionState = fromOrdinal(io.getPartitionState(pageAddr));
+ GridDhtPartitionState partState = fromOrdinal(io.getPartitionState(pageAddr));
- String state = partitionState != null ? partitionState.toString() : "N/A";
+ String state = partState != null ? partState.toString() : "N/A";
- long updateCounter = io.getUpdateCounter(pageAddr);
+ long updateCntr = io.getUpdateCounter(pageAddr);
long size = io.getSize(pageAddr);
log.info("Partition [grp=" + grp.cacheOrGroupName()
+ ", id=" + p
+ ", state=" + state
- + ", counter=" + updateCounter
+ + ", counter=" + updateCntr
+ ", size=" + size + "]");
}
finally {
@@ -5453,11 +5482,12 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
}
/**
- * @return WAL records predicate that passes only logical and mixed WAL records.
+ * @return WAL records predicate that passes only logical and mixed WAL records +
+ * CP record (used for restoring initial partition states).
*/
private IgniteBiPredicate<WALRecord.RecordType, WALPointer> logicalRecords() {
return (type, ptr) -> type.purpose() == WALRecord.RecordPurpose.LOGICAL
- || type.purpose() == WALRecord.RecordPurpose.MIXED;
+ || type.purpose() == WALRecord.RecordPurpose.MIXED || type == CHECKPOINT_RECORD;
}
/**
@@ -5670,19 +5700,23 @@ public class GridCacheDatabaseSharedManager extends IgniteCacheDatabaseSharedMan
*/
public class RestoreLogicalState extends RestoreStateContext {
/** States of partitions recovered during applying logical updates. */
- private final Map<GroupPartitionId, PartitionRecoverState> partitionRecoveryStates = new HashMap<>();
+ private final Map<GroupPartitionId, Integer> partitionRecoveryStates;
/**
* @param lastArchivedSegment Last archived segment index.
+ * @param partitionRecoveryStates Initial partition recovery states.
*/
- public RestoreLogicalState(CheckpointStatus status, WALIterator iterator, long lastArchivedSegment, IgnitePredicate<Integer> cacheGroupsPredicate) {
+ public RestoreLogicalState(CheckpointStatus status, WALIterator iterator, long lastArchivedSegment,
+ IgnitePredicate<Integer> cacheGroupsPredicate, Map<GroupPartitionId, Integer> partitionRecoveryStates) {
super(status, iterator, lastArchivedSegment, cacheGroupsPredicate);
+
+ this.partitionRecoveryStates = partitionRecoveryStates;
}
/**
* @return Map of restored partition states for cache groups.
*/
- public Map<GroupPartitionId, PartitionRecoverState> partitionRecoveryStates() {
+ public Map<GroupPartitionId, Integer> partitionRecoveryStates() {
return Collections.unmodifiableMap(partitionRecoveryStates);
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java
index 84a23d3..942a713 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/GridCacheOffheapManager.java
@@ -17,6 +17,7 @@
package org.apache.ignite.internal.processors.cache.persistence;
+import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
@@ -31,6 +32,7 @@ import java.util.concurrent.atomic.AtomicBoolean;
import javax.cache.processor.EntryProcessor;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.IgniteException;
+import org.apache.ignite.IgniteLogger;
import org.apache.ignite.failure.FailureContext;
import org.apache.ignite.failure.FailureType;
import org.apache.ignite.internal.pagemem.FullPageId;
@@ -45,9 +47,10 @@ import org.apache.ignite.internal.pagemem.wal.WALPointer;
import org.apache.ignite.internal.pagemem.wal.record.DataEntry;
import org.apache.ignite.internal.pagemem.wal.record.DataRecord;
import org.apache.ignite.internal.pagemem.wal.record.PageSnapshot;
+import org.apache.ignite.internal.pagemem.wal.record.RollbackRecord;
import org.apache.ignite.internal.pagemem.wal.record.WALRecord;
import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageInitRecord;
-import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageUpdatePartitionDataRecord;
+import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageUpdatePartitionDataRecordV2;
import org.apache.ignite.internal.pagemem.wal.record.delta.PartitionDestroyRecord;
import org.apache.ignite.internal.processors.affinity.AffinityTopologyVersion;
import org.apache.ignite.internal.processors.cache.CacheEntryPredicate;
@@ -59,6 +62,7 @@ import org.apache.ignite.internal.processors.cache.GridCacheMvccEntryInfo;
import org.apache.ignite.internal.processors.cache.GridCacheTtlManager;
import org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl;
import org.apache.ignite.internal.processors.cache.KeyCacheObject;
+import org.apache.ignite.internal.processors.cache.PartitionUpdateCounter;
import org.apache.ignite.internal.processors.cache.distributed.dht.preloader.CachePartitionPartialCountersMap;
import org.apache.ignite.internal.processors.cache.distributed.dht.preloader.IgniteHistoricalIterator;
import org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition;
@@ -66,12 +70,15 @@ import org.apache.ignite.internal.processors.cache.distributed.dht.topology.Grid
import org.apache.ignite.internal.processors.cache.mvcc.MvccSnapshot;
import org.apache.ignite.internal.processors.cache.mvcc.MvccVersion;
import org.apache.ignite.internal.processors.cache.persistence.freelist.CacheFreeListImpl;
+import org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList;
+import org.apache.ignite.internal.processors.cache.persistence.freelist.SimpleDataRow;
import org.apache.ignite.internal.processors.cache.persistence.migration.UpgradePendingTreeToPerPartitionTask;
import org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryEx;
import org.apache.ignite.internal.processors.cache.persistence.partstate.GroupPartitionId;
import org.apache.ignite.internal.processors.cache.persistence.partstate.PagesAllocationRange;
import org.apache.ignite.internal.processors.cache.persistence.partstate.PartitionAllocationMap;
-import org.apache.ignite.internal.processors.cache.persistence.partstate.PartitionRecoverState;
+import org.apache.ignite.internal.processors.cache.persistence.partstorage.PartitionMetaStorage;
+import org.apache.ignite.internal.processors.cache.persistence.partstorage.PartitionMetaStorageImpl;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PageMetaIO;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PagePartitionCountersIO;
@@ -267,14 +274,18 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
if (rowStore0 != null) {
((CacheFreeListImpl)rowStore0.freeList()).saveMetadata();
+ PartitionMetaStorage<SimpleDataRow> partStore = store.partStorage();
+
long updCntr = store.updateCounter();
long size = store.fullSize();
long rmvId = globalRemoveId().get();
+ byte[] updCntrsBytes = store.partUpdateCounter().getBytes();
+
PageMemoryEx pageMem = (PageMemoryEx)grp.dataRegion().pageMemory();
IgniteWriteAheadLogManager wal = this.ctx.wal();
- if (size > 0 || updCntr > 0) {
+ if (size > 0 || updCntr > 0 || !store.partUpdateCounter().sequential()) {
GridDhtPartitionState state = null;
// localPartition will not acquire writeLock here because create=false.
@@ -313,7 +324,46 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
boolean changed = false;
try {
- PagePartitionMetaIO io = PageIO.getPageIO(partMetaPageAddr);
+ PagePartitionMetaIOV2 io = PageIO.getPageIO(partMetaPageAddr);
+
+ long link = io.getGapsLink(partMetaPageAddr);
+
+ if (updCntrsBytes == null && link != 0) {
+ partStore.removeDataRowByLink(link, grp.statisticsHolderData());
+
+ io.setGapsLink(partMetaPageAddr, (link = 0));
+
+ changed = true;
+ }
+ else if (updCntrsBytes != null && link == 0) {
+ SimpleDataRow row = new SimpleDataRow(store.partId(), updCntrsBytes);
+
+ partStore.insertDataRow(row, grp.statisticsHolderData());
+
+ io.setGapsLink(partMetaPageAddr, (link = row.link()));
+
+ changed = true;
+ }
+ else if (updCntrsBytes != null && link != 0) {
+ byte[] prev = partStore.readRow(link);
+
+ assert prev != null : "Read null gaps using link=" + link;
+
+ if (!Arrays.equals(prev, updCntrsBytes)) {
+ partStore.removeDataRowByLink(link, grp.statisticsHolderData());
+
+ SimpleDataRow row = new SimpleDataRow(store.partId(), updCntrsBytes);
+
+ partStore.insertDataRow(row, grp.statisticsHolderData());
+
+ io.setGapsLink(partMetaPageAddr, (link = row.link()));
+
+ changed = true;
+ }
+ }
+
+ if (changed)
+ partStore.saveMetadata();
changed |= io.setUpdateCounter(partMetaPageAddr, updCntr);
changed |= io.setGlobalRemoveId(partMetaPageAddr, rmvId);
@@ -386,7 +436,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
pageCnt = io.getCandidatePageCount(partMetaPageAddr);
if (changed && PageHandler.isWalDeltaRecordNeeded(pageMem, grpId, partMetaId, partMetaPage, wal, null))
- wal.log(new MetaPageUpdatePartitionDataRecord(
+ wal.log(new MetaPageUpdatePartitionDataRecordV2(
grpId,
partMetaId,
updCntr,
@@ -394,7 +444,8 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
(int)size, // TODO: Partition size may be long
cntrsPageId,
state == null ? -1 : (byte)state.ordinal(),
- pageCnt
+ pageCnt,
+ link
));
}
finally {
@@ -413,7 +464,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
/** {@inheritDoc} */
- @Override public long restorePartitionStates(Map<GroupPartitionId, PartitionRecoverState> partitionRecoveryStates) throws IgniteCheckedException {
+ @Override public long restorePartitionStates(Map<GroupPartitionId, Integer> partitionRecoveryStates) throws IgniteCheckedException {
if (grp.isLocal() || !grp.affinityNode() || !grp.dataRegion().config().isPersistenceEnabled())
return 0;
@@ -425,7 +476,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
PageMemoryEx pageMem = (PageMemoryEx)grp.dataRegion().pageMemory();
for (int p = 0; p < grp.affinity().partitions(); p++) {
- PartitionRecoverState recoverState = partitionRecoveryStates.get(new GroupPartitionId(grp.groupId(), p));
+ Integer recoverState = partitionRecoveryStates.get(new GroupPartitionId(grp.groupId(), p));
if (ctx.pageStore().exists(grp.groupId(), p)) {
ctx.pageStore().ensure(grp.groupId(), p);
@@ -446,7 +497,8 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
GridDhtLocalPartition part = grp.topology().forceCreatePartition(p);
- onPartitionInitialCounterUpdated(p, 0);
+ // Triggers initialization of existing(having datafile) partition before acquiring cp read lock.
+ part.dataStore().init();
ctx.database().checkpointReadLock();
@@ -463,17 +515,9 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
PagePartitionMetaIO io = PagePartitionMetaIO.VERSIONS.forPage(pageAddr);
if (recoverState != null) {
- io.setPartitionState(pageAddr, (byte) recoverState.stateId());
+ io.setPartitionState(pageAddr, (byte) recoverState.intValue());
- changed = updateState(part, recoverState.stateId());
-
- if (recoverState.stateId() == GridDhtPartitionState.OWNING.ordinal()
- || (recoverState.stateId() == GridDhtPartitionState.MOVING.ordinal()
- && part.initialUpdateCounter() < recoverState.updateCounter())) {
- part.initialUpdateCounter(recoverState.updateCounter());
-
- changed = true;
- }
+ changed = updateState(part, recoverState);
if (log.isDebugEnabled())
log.debug("Restored partition state (from WAL) " +
@@ -503,12 +547,10 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
ctx.database().checkpointReadUnlock();
}
}
- else if (recoverState != null) {
+ else if (recoverState != null && recoverState >= 0) { // Pre-create partition if having valid state.
GridDhtLocalPartition part = grp.topology().forceCreatePartition(p);
- onPartitionInitialCounterUpdated(p, recoverState.updateCounter());
-
- updateState(part, recoverState.stateId());
+ updateState(part, recoverState);
processed++;
@@ -816,15 +858,12 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
/** {@inheritDoc} */
- @Override public void onPartitionInitialCounterUpdated(int part, long cntr) {
+ @Override public void onPartitionInitialCounterUpdated(int part, long start, long delta) {
CacheDataStore store = partDataStores.get(part);
assert store != null;
- long oldCnt = store.initialUpdateCounter();
-
- if (oldCnt < cntr)
- store.updateInitialCounter(cntr);
+ store.updateInitialCounter(start, delta);
}
/** {@inheritDoc} */
@@ -911,6 +950,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
return new Metas(
new RootPage(new FullPageId(metastoreRoot, grpId), allocated),
new RootPage(new FullPageId(reuseListRoot, grpId), allocated),
+ null,
null);
}
finally {
@@ -951,7 +991,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
WALIterator it = grp.shared().wal().replay(minPtr);
- WALHistoricalIterator iterator = new WALHistoricalIterator(grp, partCntrs, it);
+ WALHistoricalIterator iterator = new WALHistoricalIterator(log, grp, partCntrs, it);
// Add historical partitions which are unabled to reserve to missing set.
missing.addAll(iterator.missingParts);
@@ -1024,7 +1064,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
for (CacheDataStore store : partDataStores.values()) {
assert store instanceof GridCacheDataStore;
- CacheFreeListImpl freeList = ((GridCacheDataStore)store).freeList;
+ AbstractFreeList freeList = ((GridCacheDataStore)store).freeList;
if (freeList == null)
continue;
@@ -1046,7 +1086,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
for (CacheDataStore store : partDataStores.values()) {
assert store instanceof GridCacheDataStore;
- CacheFreeListImpl freeList = ((GridCacheDataStore)store).freeList;
+ AbstractFreeList freeList = ((GridCacheDataStore)store).freeList;
if (freeList == null)
continue;
@@ -1094,6 +1134,9 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
/** */
private static final long serialVersionUID = 0L;
+ /** Logger. */
+ private IgniteLogger log;
+
/** Cache context. */
private final CacheGroupContext grp;
@@ -1118,24 +1161,34 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
/** */
private DataEntry next;
- /** Flag indicates that partition belongs to current {@link #next} is finished and no longer needs to rebalance. */
- private boolean reachedPartitionEnd;
+ /**
+ * Rebalanced counters in the range from initialUpdateCntr to updateCntr.
+ * Invariant: initUpdCntr[idx] + rebalancedCntrs[idx] = updateCntr[idx]
+ */
+ private long[] rebalancedCntrs;
- /** Flag indicates that update counters for requested partitions have been reached and done.
- * It means that no further iteration is needed. */
- private boolean doneAllPartitions;
+ /** A partition what will be finished on next iteration. */
+ private int donePart = -1;
/**
+ * @param log Logger.
* @param grp Cache context.
* @param walIt WAL iterator.
*/
- private WALHistoricalIterator(CacheGroupContext grp, CachePartitionPartialCountersMap partMap, WALIterator walIt) {
+ private WALHistoricalIterator(IgniteLogger log, CacheGroupContext grp, CachePartitionPartialCountersMap partMap,
+ WALIterator walIt) {
+ this.log = log;
this.grp = grp;
this.partMap = partMap;
this.walIt = walIt;
cacheIds = grp.cacheIds();
+ rebalancedCntrs = new long[partMap.size()];
+
+ for (int i = 0; i < rebalancedCntrs.length; i++)
+ rebalancedCntrs[i] = partMap.initialUpdateCounterAt(i);
+
reservePartitions();
advance();
@@ -1194,13 +1247,19 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
CacheDataRow val = new DataEntryRow(next);
- if (reachedPartitionEnd) {
- doneParts.add(next.partitionId());
+ if (donePart != -1) {
+ int pIdx = partMap.partitionIndex(donePart);
+
+ if (log.isDebugEnabled()) {
+ log.debug("Partition done [grpId=" + grp.groupId() +
+ ", partId=" + donePart +
+ ", from=" + partMap.initialUpdateCounterAt(pIdx) +
+ ", to=" + partMap.updateCounterAt(pIdx) + ']');
+ }
- reachedPartitionEnd = false;
+ doneParts.add(donePart);
- if (doneParts.size() == partMap.size())
- doneAllPartitions = true;
+ donePart = -1;
}
advance();
@@ -1259,10 +1318,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
private void advance() {
next = null;
- if (doneAllPartitions)
- return;
-
- while (true) {
+ outer: while (doneParts.size() != partMap.size()) {
if (entryIt != null) {
while (entryIt.hasNext()) {
DataEntry entry = entryIt.next();
@@ -1277,8 +1333,10 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
long to = partMap.updateCounterAt(idx);
if (entry.partitionCounter() > from && entry.partitionCounter() <= to) {
- if (entry.partitionCounter() == to)
- reachedPartitionEnd = true;
+ // Partition will be marked as done for current entry on next iteration.
+ if (++rebalancedCntrs[idx] == to ||
+ entry.partitionCounter() == to && grp.hasAtomicCaches())
+ donePart = entry.partitionId();
next = entry;
@@ -1290,6 +1348,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
entryIt = null;
+ // Search for next DataEntry while applying rollback counters.
while (walIt.hasNext()) {
IgniteBiTuple<WALPointer, WALRecord> rec = walIt.next();
@@ -1297,14 +1356,38 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
DataRecord data = (DataRecord)rec.get2();
entryIt = data.writeEntries().iterator();
+
// Move on to the next valid data entry.
+ continue outer;
+ }
+ else if (rec.get2() instanceof RollbackRecord) {
+ RollbackRecord rbRec = (RollbackRecord)rec.get2();
- break;
+ if (grp.groupId() == rbRec.groupId()) {
+ int idx = partMap.partitionIndex(rbRec.partitionId());
+
+ if (idx < 0 || missingParts.contains(idx))
+ continue;
+
+ long from = partMap.initialUpdateCounterAt(idx);
+ long to = partMap.updateCounterAt(idx);
+
+ rebalancedCntrs[idx] += rbRec.overlap(from, to);
+
+ if (rebalancedCntrs[idx] == partMap.updateCounterAt(idx)) {
+ if (log.isDebugEnabled()) {
+ log.debug("Partition done [partId=" + donePart +
+ " from=" + from + " to=" + to + ']');
+ }
+
+ doneParts.add(rbRec.partitionId()); // Add to done set immediately.
+ }
+ }
}
}
- if (entryIt == null)
- return;
+ assert entryIt != null || doneParts.size() == partMap.size() :
+ "Reached end of WAL but not all partitions are done";
}
}
}
@@ -1440,14 +1523,19 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
@GridToStringInclude
private final RootPage pendingTreeRoot;
+ /** */
+ @GridToStringInclude
+ private final RootPage partMetastoreReuseListRoot;
+
/**
* @param treeRoot Metadata storage root.
* @param reuseListRoot Reuse list root.
*/
- Metas(RootPage treeRoot, RootPage reuseListRoot, RootPage pendingTreeRoot) {
+ Metas(RootPage treeRoot, RootPage reuseListRoot, RootPage pendingTreeRoot, RootPage partMetastoreReuseListRoot) {
this.treeRoot = treeRoot;
this.reuseListRoot = reuseListRoot;
this.pendingTreeRoot = pendingTreeRoot;
+ this.partMetastoreReuseListRoot = partMetastoreReuseListRoot;
}
/** {@inheritDoc} */
@@ -1467,13 +1555,13 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
private String name;
/** */
- private volatile CacheFreeListImpl freeList;
+ private volatile AbstractFreeList<CacheDataRow> freeList;
/** */
private PendingEntriesTree pendingTree;
/** */
- private volatile CacheDataStore delegate;
+ private volatile CacheDataStoreImpl delegate;
/**
* Cache id which should be throttled.
@@ -1488,6 +1576,9 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
private volatile long nextStoreCleanTime;
/** */
+ private PartitionMetaStorage<SimpleDataRow> partStorage;
+
+ /** */
private final boolean exists;
/** */
@@ -1512,7 +1603,7 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
* @throws IgniteCheckedException If failed.
*/
private CacheDataStore init0(boolean checkExists) throws IgniteCheckedException {
- CacheDataStore delegate0 = delegate;
+ CacheDataStoreImpl delegate0 = delegate;
if (delegate0 != null)
return delegate0;
@@ -1532,7 +1623,9 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
if (PageIdUtils.partId(metas.reuseListRoot.pageId().pageId()) != partId ||
PageIdUtils.partId(metas.treeRoot.pageId().pageId()) != partId ||
- PageIdUtils.partId(metas.pendingTreeRoot.pageId().pageId()) != partId) {
+ PageIdUtils.partId(metas.pendingTreeRoot.pageId().pageId()) != partId ||
+ PageIdUtils.partId(metas.partMetastoreReuseListRoot.pageId().pageId()) != partId
+ ) {
throw new IgniteCheckedException("Invalid meta root allocated [" +
"cacheOrGroupName=" + grp.cacheOrGroupName() +
", partId=" + partId +
@@ -1558,6 +1651,25 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
};
+ RootPage partMetastoreReuseListRoot = metas.partMetastoreReuseListRoot;
+
+ partStorage = new PartitionMetaStorageImpl<SimpleDataRow>(
+ grp.groupId(),
+ grp.cacheOrGroupName() + "-partstore-" + partId,
+ grp.dataRegion().memoryMetrics(),
+ grp.dataRegion(),
+ freeList,
+ ctx.wal(),
+ partMetastoreReuseListRoot.pageId().pageId(),
+ partMetastoreReuseListRoot.isAllocated()) {
+ /** {@inheritDoc} */
+ @Override protected long allocatePageNoReuse() throws IgniteCheckedException {
+ assert grp.shared().database().checkpointLockIsHeldByThread();
+
+ return pageMem.allocatePage(grpId, partId, PageIdAllocator.FLAG_DATA);
+ }
+ };
+
CacheDataRowStore rowStore = new CacheDataRowStore(grp, freeList, partId);
RootPage treeRoot = metas.treeRoot;
@@ -1642,14 +1754,18 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
try {
if (PageIO.getType(pageAddr) != 0) {
- PagePartitionMetaIO io = PagePartitionMetaIO.VERSIONS.latest();
+ PagePartitionMetaIOV2 io = (PagePartitionMetaIOV2)PagePartitionMetaIO.VERSIONS.latest();
Map<Integer, Long> cacheSizes = null;
if (grp.sharedGroup())
cacheSizes = readSharedGroupCacheSizes(pageMem, grpId, io.getCountersPageId(pageAddr));
- delegate0.init(io.getSize(pageAddr), io.getUpdateCounter(pageAddr), cacheSizes);
+ long link = io.getGapsLink(pageAddr);
+
+ byte[] data = link == 0 ? null : partStorage.readRow(link);
+
+ delegate0.restoreState(io.getSize(pageAddr), io.getUpdateCounter(pageAddr), cacheSizes, data);
globalRemoveId().setIfGreater(io.getGlobalRemoveId(pageAddr));
}
@@ -1704,28 +1820,32 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
try {
boolean allocated = false;
boolean pendingTreeAllocated = false;
+ boolean partMetastoreReuseListAllocated = false;
long pageAddr = pageMem.writeLock(grpId, partMetaId, partMetaPage);
try {
- long treeRoot, reuseListRoot, pendingTreeRoot;
+ long treeRoot, reuseListRoot, pendingTreeRoot, partMetaStoreReuseListRoot;
// Initialize new page.
if (PageIO.getType(pageAddr) != PageIO.T_PART_META) {
- PagePartitionMetaIO io = PagePartitionMetaIO.VERSIONS.latest();
+ PagePartitionMetaIOV2 io = (PagePartitionMetaIOV2)PagePartitionMetaIO.VERSIONS.latest();
io.initNewPage(pageAddr, partMetaId, pageMem.realPageSize(grpId));
treeRoot = pageMem.allocatePage(grpId, partId, PageMemory.FLAG_DATA);
reuseListRoot = pageMem.allocatePage(grpId, partId, PageMemory.FLAG_DATA);
pendingTreeRoot = pageMem.allocatePage(grpId, partId, PageMemory.FLAG_DATA);
+ partMetaStoreReuseListRoot = pageMem.allocatePage(grpId, partId, PageMemory.FLAG_DATA);
assert PageIdUtils.flag(treeRoot) == PageMemory.FLAG_DATA;
assert PageIdUtils.flag(reuseListRoot) == PageMemory.FLAG_DATA;
assert PageIdUtils.flag(pendingTreeRoot) == PageMemory.FLAG_DATA;
+ assert PageIdUtils.flag(partMetaStoreReuseListRoot) == PageMemory.FLAG_DATA;
io.setTreeRoot(pageAddr, treeRoot);
io.setReuseListRoot(pageAddr, reuseListRoot);
io.setPendingTreeRoot(pageAddr, pendingTreeRoot);
+ io.setPartitionMetaStoreReuseListRoot(pageAddr, partMetaStoreReuseListRoot);
if (PageHandler.isWalDeltaRecordNeeded(pageMem, grpId, partMetaId, partMetaPage, wal, null)) {
wal.log(new PageSnapshot(new FullPageId(partMetaId, grpId), pageAddr,
@@ -1740,14 +1860,14 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
treeRoot = io.getTreeRoot(pageAddr);
reuseListRoot = io.getReuseListRoot(pageAddr);
- int pageVersion = PagePartitionMetaIO.getVersion(pageAddr);
+ int pageVer = PagePartitionMetaIO.getVersion(pageAddr);
- if (pageVersion < 2) {
- assert pageVersion == 1;
+ if (pageVer < 2) {
+ assert pageVer == 1;
if (log.isDebugEnabled())
log.info("Upgrade partition meta page version: [part=" + partId +
- ", grpId=" + grpId + ", oldVer=" + pageVersion +
+ ", grpId=" + grpId + ", oldVer=" + pageVer +
", newVer=" + io.getVersion()
);
@@ -1756,8 +1876,10 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
((PagePartitionMetaIOV2)io).upgradePage(pageAddr);
pendingTreeRoot = pageMem.allocatePage(grpId, partId, PageMemory.FLAG_DATA);
+ partMetaStoreReuseListRoot = pageMem.allocatePage(grpId, partId, PageMemory.FLAG_DATA);
io.setPendingTreeRoot(pageAddr, pendingTreeRoot);
+ io.setPartitionMetaStoreReuseListRoot(pageAddr, partMetaStoreReuseListRoot);
if (PageHandler.isWalDeltaRecordNeeded(pageMem, grpId, partMetaId, partMetaPage, wal,
null)) {
@@ -1765,10 +1887,24 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
pageMem.pageSize(), pageMem.realPageSize(grpId)));
}
- pendingTreeAllocated = true;
+ pendingTreeAllocated = partMetastoreReuseListAllocated = true;
}
- else
+ else {
pendingTreeRoot = io.getPendingTreeRoot(pageAddr);
+ partMetaStoreReuseListRoot = io.getPartitionMetaStoreReuseListRoot(pageAddr);
+
+ if (partMetaStoreReuseListRoot == 0) {
+ partMetaStoreReuseListRoot = pageMem.allocatePage(grpId, partId, PageMemory.FLAG_DATA);
+
+ if (PageHandler.isWalDeltaRecordNeeded(pageMem, grpId, partMetaId, partMetaPage, wal,
+ null)) {
+ wal.log(new PageSnapshot(new FullPageId(partMetaId, grpId), pageAddr,
+ pageMem.pageSize(), pageMem.realPageSize(grpId)));
+ }
+
+ partMetastoreReuseListAllocated = true;
+ }
+ }
if (PageIdUtils.flag(treeRoot) != PageMemory.FLAG_DATA)
throw new StorageException("Wrong tree root page id flag: treeRoot="
@@ -1781,15 +1917,21 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
if (PageIdUtils.flag(pendingTreeRoot) != PageMemory.FLAG_DATA)
throw new StorageException("Wrong pending tree root page id flag: reuseListRoot="
+ U.hexLong(reuseListRoot) + ", part=" + partId + ", grpId=" + grpId);
+
+ if (PageIdUtils.flag(partMetaStoreReuseListRoot) != PageMemory.FLAG_DATA)
+ throw new StorageException("Wrong partition meta store list root page id flag: partMetaStoreReuseListRoot="
+ + U.hexLong(partMetaStoreReuseListRoot) + ", part=" + partId + ", grpId=" + grpId);
}
return new Metas(
new RootPage(new FullPageId(treeRoot, grpId), allocated),
new RootPage(new FullPageId(reuseListRoot, grpId), allocated),
- new RootPage(new FullPageId(pendingTreeRoot, grpId), allocated || pendingTreeAllocated));
+ new RootPage(new FullPageId(pendingTreeRoot, grpId), allocated || pendingTreeAllocated),
+ new RootPage(new FullPageId(partMetaStoreReuseListRoot, grpId), allocated || partMetastoreReuseListAllocated));
}
finally {
- pageMem.writeUnlock(grpId, partMetaId, partMetaPage, null, allocated || pendingTreeAllocated);
+ pageMem.writeUnlock(grpId, partMetaId, partMetaPage, null,
+ allocated || pendingTreeAllocated || partMetastoreReuseListAllocated);
}
}
finally {
@@ -1798,6 +1940,16 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
/** {@inheritDoc} */
+ @Override public boolean init() {
+ try {
+ return init0(true) != null;
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
+ }
+
+ /** {@inheritDoc} */
@Override public int partId() {
return partId;
}
@@ -1888,10 +2040,34 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
/** {@inheritDoc} */
- @Override public long getAndIncrementUpdateCounter(long delta) {
+ @Override public long reservedCounter() {
+ try {
+ CacheDataStore delegate0 = init0(true);
+
+ return delegate0 == null ? 0 : delegate0.reservedCounter();
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public PartitionUpdateCounter partUpdateCounter() {
try {
CacheDataStore delegate0 = init0(true);
+ return delegate0 == null ? null : delegate0.partUpdateCounter();
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
+ }
+
+ /** {@inheritDoc} */
+ @Override public long getAndIncrementUpdateCounter(long delta) {
+ try {
+ CacheDataStore delegate0 = init0(false);
+
return delegate0 == null ? 0 : delegate0.getAndIncrementUpdateCounter(delta);
}
catch (IgniteCheckedException e) {
@@ -1900,8 +2076,18 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
/** {@inheritDoc} */
- @Override public void init(long size, long updCntr, @Nullable Map<Integer, Long> cacheSizes) {
- throw new IllegalStateException("Should be never called.");
+ @Override public long reserve(long delta) {
+ try {
+ CacheDataStore delegate0 = init0(false);
+
+ if (delegate0 == null)
+ throw new IllegalStateException("Should be never called.");
+
+ return delegate0.reserve(delta);
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
}
/** {@inheritDoc} */
@@ -1918,12 +2104,11 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
/** {@inheritDoc} */
- @Override public void updateCounter(long start, long delta) {
+ @Override public boolean updateCounter(long start, long delta) {
try {
CacheDataStore delegate0 = init0(false);
- if (delegate0 != null)
- delegate0.updateCounter(start, delta);
+ return delegate0 != null && delegate0.updateCounter(start, delta);
}
catch (IgniteCheckedException e) {
throw new IgniteException(e);
@@ -1947,7 +2132,10 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
try {
CacheDataStore delegate0 = init0(false);
- return delegate0 == null ? 0 : delegate0.nextUpdateCounter();
+ if (delegate0 == null)
+ throw new IllegalStateException("Should be never called.");
+
+ return delegate0.nextUpdateCounter();
}
catch (IgniteCheckedException e) {
throw new IgniteException(e);
@@ -1967,12 +2155,14 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
}
/** {@inheritDoc} */
- @Override public void updateInitialCounter(long cntr) {
+ @Override public void updateInitialCounter(long start, long delta) {
try {
CacheDataStore delegate0 = init0(true);
- if (delegate0 != null)
- delegate0.updateInitialCounter(cntr);
+ if (delegate0 == null)
+ throw new IllegalStateException("Should be never called.");
+
+ delegate0.updateInitialCounter(start, delta);
}
catch (IgniteCheckedException e) {
throw new IgniteException(e);
@@ -2495,6 +2685,25 @@ public class GridCacheOffheapManager extends IgniteCacheOffheapManagerImpl imple
if (delegate0 != null)
delegate0.preload();
}
+
+ /** {@inheritDoc} */
+ @Override public void resetUpdateCounter() {
+ try {
+ CacheDataStore delegate0 = init0(true);
+
+ if (delegate0 == null)
+ return;
+
+ delegate0.resetUpdateCounter();
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
+ }
+
+ @Override public PartitionMetaStorage partStorage() {
+ return partStorage;
+ }
}
/**
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/RowStore.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/RowStore.java
index 91fd207..9a3d9d2 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/RowStore.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/RowStore.java
@@ -18,6 +18,7 @@
package org.apache.ignite.internal.processors.cache.persistence;
import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.internal.pagemem.PageIdUtils;
import org.apache.ignite.internal.pagemem.PageMemory;
import org.apache.ignite.internal.processors.cache.CacheGroupContext;
import org.apache.ignite.internal.processors.cache.CacheObjectContext;
@@ -26,6 +27,7 @@ import org.apache.ignite.internal.processors.cache.persistence.freelist.FreeList
import org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler;
import org.apache.ignite.internal.processors.query.GridQueryRowCacheCleaner;
import org.apache.ignite.internal.stat.IoStatisticsHolder;
+import org.apache.ignite.internal.util.typedef.internal.U;
/**
* Data store for H2 rows.
@@ -109,6 +111,10 @@ public class RowStore {
ctx.database().checkpointReadUnlock();
}
}
+
+ assert row.key().partition() == PageIdUtils.partId(row.link()) :
+ "Constructed a link with invalid partition ID [partId=" + row.key().partition() +
+ ", link=" + U.hexLong(row.link()) + ']';
}
/**
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
index 133f0a1..d8fe3a4 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
@@ -18,6 +18,8 @@
package org.apache.ignite.internal.processors.cache.persistence;
import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
/**
* Simple interface for data, store in some RowStore.
@@ -49,4 +51,9 @@ public interface Storable {
* which is entirely available on the very first page followed by the row link.
*/
public int headerSize();
+
+ /**
+ * @return I/O for handling this storable.
+ */
+ public IOVersions<? extends AbstractDataPageIO> ioVersions();
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/CheckpointEntry.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/CheckpointEntry.java
index c26e0a3..e7bcb05 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/CheckpointEntry.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/checkpoint/CheckpointEntry.java
@@ -20,7 +20,6 @@ package org.apache.ignite.internal.processors.cache.persistence.checkpoint;
import java.lang.ref.SoftReference;
import java.util.Arrays;
import java.util.Collections;
-import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.CountDownLatch;
@@ -275,7 +274,7 @@ public class CheckpointEntry {
if (stateRec == null)
return Collections.emptyMap();
- Map<Integer, GroupState> grpStates = new HashMap<>(stateRec.size());
+ Map<Integer, GroupState> grpStates = U.newHashMap(stateRec.size());
for (Integer grpId : stateRec.keySet()) {
CacheState recState = stateRec.get(grpId);
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/AbstractFreeList.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/AbstractFreeList.java
index 958fb31..5742b74 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/AbstractFreeList.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/AbstractFreeList.java
@@ -34,7 +34,6 @@ import org.apache.ignite.internal.processors.cache.persistence.Storable;
import org.apache.ignite.internal.processors.cache.persistence.evict.PageEvictionTracker;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.DataPagePayload;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO;
import org.apache.ignite.internal.processors.cache.persistence.tree.reuse.LongListReuseBag;
import org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseBag;
@@ -484,23 +483,32 @@ public abstract class AbstractFreeList<T extends Storable> extends PagesList imp
long pageId = 0L;
- for (int b = remaining < MIN_SIZE_FOR_DATA_PAGE ? bucket(remaining, false) + 1 : REUSE_BUCKET; b < BUCKETS; b++) {
- pageId = takeEmptyPage(b, ioVersions(), statHolder);
+ if (remaining < MIN_SIZE_FOR_DATA_PAGE) {
+ for (int b = bucket(remaining, false) + 1; b < BUCKETS - 1; b++) {
+ pageId = takeEmptyPage(b, row.ioVersions(), statHolder);
- if (pageId != 0L)
- break;
+ if (pageId != 0L)
+ break;
+ }
+ }
+
+ if (pageId == 0L) { // Handle reuse bucket.
+ if (reuseList == this)
+ pageId = takeEmptyPage(REUSE_BUCKET, row.ioVersions(), statHolder);
+ else
+ pageId = reuseList.takeRecycledPage();
}
- AbstractDataPageIO<T> initIo = null;
+ AbstractDataPageIO initIo = null;
if (pageId == 0L) {
pageId = allocateDataPage(row.partition());
- initIo = ioVersions().latest();
+ initIo = row.ioVersions().latest();
}
- else if (PageIdUtils.tag(pageId) != PageIdAllocator.FLAG_DATA)
- pageId = initReusedPage(pageId, row.partition(), statHolder);
- else
+ else if (PageIdUtils.tag(pageId) != PageIdAllocator.FLAG_DATA) // Page is taken from reuse bucket.
+ pageId = initReusedPage(row, pageId, row.partition(), statHolder);
+ else // Page is taken from free space bucket. For in-memory mode partition must be changed.
pageId = PageIdUtils.changePartitionId(pageId, (row.partition()));
written = write(pageId, writeRow, initIo, row, written, FAIL_I, statHolder);
@@ -525,7 +533,7 @@ public abstract class AbstractFreeList<T extends Storable> extends PagesList imp
*
* @see PagesList#initReusedPage(long, long, long, int, byte, PageIO)
*/
- private long initReusedPage(long reusedPageId, int partId,
+ private long initReusedPage(T row, long reusedPageId, int partId,
IoStatisticsHolder statHolder) throws IgniteCheckedException {
long reusedPage = acquirePage(reusedPageId, statHolder);
try {
@@ -535,7 +543,7 @@ public abstract class AbstractFreeList<T extends Storable> extends PagesList imp
try {
return initReusedPage(reusedPageId, reusedPage, reusedPageAddr,
- partId, PageIdAllocator.FLAG_DATA, ioVersions().latest());
+ partId, PageIdAllocator.FLAG_DATA, row.ioVersions().latest());
}
finally {
writeUnlock(reusedPageId, reusedPage, reusedPageAddr, true);
@@ -694,11 +702,6 @@ public abstract class AbstractFreeList<T extends Storable> extends PagesList imp
}
}
- /**
- * @return IOVersions.
- */
- public abstract IOVersions<? extends AbstractDataPageIO<T>> ioVersions();
-
/** {@inheritDoc} */
@Override public String toString() {
return "FreeList [name=" + name + ']';
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/CacheFreeListImpl.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/CacheFreeListImpl.java
index 625c0b1..c1a58de 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/CacheFreeListImpl.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/CacheFreeListImpl.java
@@ -23,9 +23,6 @@ import org.apache.ignite.internal.pagemem.wal.IgniteWriteAheadLogManager;
import org.apache.ignite.internal.processors.cache.persistence.CacheDataRow;
import org.apache.ignite.internal.processors.cache.persistence.DataRegion;
import org.apache.ignite.internal.processors.cache.persistence.DataRegionMetricsImpl;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.DataPageIO;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
import org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseList;
import org.apache.ignite.internal.stat.IoStatisticsHolder;
import org.apache.ignite.internal.util.typedef.internal.U;
@@ -51,17 +48,12 @@ public class CacheFreeListImpl extends AbstractFreeList<CacheDataRow> {
}
/** {@inheritDoc} */
- @Override public IOVersions<? extends AbstractDataPageIO<CacheDataRow>> ioVersions() {
- return DataPageIO.VERSIONS;
- }
-
- /** {@inheritDoc} */
@Override public void insertDataRow(CacheDataRow row, IoStatisticsHolder statHolder) throws IgniteCheckedException {
super.insertDataRow(row, statHolder);
assert row.key().partition() == PageIdUtils.partId(row.link()) :
"Constructed a link with invalid partition ID [partId=" + row.key().partition() +
- ", link=" + U.hexLong(row.link()) + ']';
+ ", link=" + U.hexLong(row.link()) + ']';
}
/** {@inheritDoc} */
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/PagesList.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/PagesList.java
index e5cb3a6..1855578 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/PagesList.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/PagesList.java
@@ -660,7 +660,7 @@ public abstract class PagesList extends DataStructure {
* @throws IgniteCheckedException If failed.
*/
protected final void put(
- ReuseBag bag,
+ @Nullable ReuseBag bag,
final long dataId,
final long dataPage,
final long dataAddr,
@@ -669,10 +669,13 @@ public abstract class PagesList extends DataStructure {
throws IgniteCheckedException {
assert bag == null ^ dataAddr == 0L;
+ if (bag != null && bag.isEmpty()) // Skip allocating stripe for empty bag.
+ return;
+
for (int lockAttempt = 0; ;) {
Stripe stripe = getPageForPut(bucket, bag);
- // No need to continue if bag has been utilized at getPageForPut.
+ // No need to continue if bag has been utilized at getPageForPut (free page can be used for pagelist).
if (bag != null && bag.isEmpty())
return;
@@ -936,7 +939,6 @@ public abstract class PagesList extends DataStructure {
int idx = io.addPage(prevAddr, nextId, pageSize());
if (idx == -1) { // Attempt to add page failed: the node page is full.
-
final long nextPage = acquirePage(nextId, statHolder);
try {
@@ -1095,7 +1097,7 @@ public abstract class PagesList extends DataStructure {
* @return Removed page ID.
* @throws IgniteCheckedException If failed.
*/
- protected final long takeEmptyPage(int bucket, @Nullable IOVersions initIoVers,
+ protected long takeEmptyPage(int bucket, @Nullable IOVersions initIoVers,
IoStatisticsHolder statHolder) throws IgniteCheckedException {
for (int lockAttempt = 0; ;) {
Stripe stripe = getPageForTake(bucket);
@@ -1278,6 +1280,8 @@ public abstract class PagesList extends DataStructure {
}
/**
+ * Removes data page from bucket, merges bucket list if needed.
+ *
* @param dataId Data page ID.
* @param dataPage Data page pointer.
* @param dataAddr Data page address.
@@ -1642,7 +1646,7 @@ public abstract class PagesList extends DataStructure {
public volatile long tailId;
/** */
- volatile boolean empty;
+ public volatile boolean empty;
/**
* @param tailId Tail ID.
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageDataRow.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/SimpleDataRow.java
similarity index 52%
copy from modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageDataRow.java
copy to modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/SimpleDataRow.java
index 2d7b0a6..382869c 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageDataRow.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/freelist/SimpleDataRow.java
@@ -15,89 +15,89 @@
* limitations under the License.
*/
-package org.apache.ignite.internal.processors.cache.persistence.metastorage;
+package org.apache.ignite.internal.processors.cache.persistence.freelist;
import org.apache.ignite.IgniteCheckedException;
-import org.apache.ignite.internal.pagemem.PageIdAllocator;
import org.apache.ignite.internal.processors.cache.persistence.Storable;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.SimpleDataPageIO;
+import org.apache.ignite.internal.util.tostring.GridToStringExclude;
+import org.apache.ignite.internal.util.typedef.internal.S;
/**
- *
+ * Implementation of {@link Storable} which holds byte array of variable length.
*/
-public class MetastorageDataRow implements MetastorageSearchRow, Storable {
+public class SimpleDataRow implements Storable {
/** */
private long link;
/** */
- private String key;
+ private final int part;
/** */
- private byte[] value;
+ @GridToStringExclude
+ private final byte[] val;
- /** */
- public MetastorageDataRow(long link, String key, byte[] value) {
+ /**
+ * @param link Link.
+ * @param part Partition.
+ * @param val Value.
+ */
+ public SimpleDataRow(long link, int part, byte[] val) {
this.link = link;
- this.key = key;
- this.value = value;
- }
-
- /** */
- public MetastorageDataRow(String key, byte[] value) {
- this.key = key;
- this.value = value;
+ this.part = part;
+ this.val = val;
}
/**
- * @return Key.
+ * @param part Partition.
+ * @param val Value.
*/
- @Override public String key() {
- return key;
- }
-
- /** {@inheritDoc} */
- @Override
- public int hash() {
- return key.hashCode();
+ public SimpleDataRow(int part, byte[] val) {
+ this.part = part;
+ this.val = val;
}
/** {@inheritDoc} */
- @Override
- public int partition() {
- return MetaStorage.PRESERVE_LEGACY_METASTORAGE_PARTITION_ID ? PageIdAllocator.OLD_METASTORE_PARTITION: PageIdAllocator.METASTORE_PARTITION;
+ @Override public void link(long link) {
+ this.link = link;
}
/** {@inheritDoc} */
- @Override public int size() throws IgniteCheckedException {
- return 4 + value().length;
+ @Override public long link() {
+ return link;
}
/** {@inheritDoc} */
- @Override public int headerSize() {
- return 0;
+ @Override public int partition() {
+ return part;
}
/** {@inheritDoc} */
- @Override
- public void link(long link) {
- this.link = link;
+ @Override public int size() throws IgniteCheckedException {
+ return 2 /** Fragment size */ + 2 /** Row size */ + value().length;
}
/** {@inheritDoc} */
- @Override
- public long link() {
- return link;
+ @Override public int headerSize() {
+ return 0;
}
/**
* @return Value.
*/
public byte[] value() {
- return value;
+ return val;
+ }
+
+ /** {@inheritDoc} */
+ @Override public IOVersions<? extends AbstractDataPageIO> ioVersions() {
+ return SimpleDataPageIO.VERSIONS;
}
/** {@inheritDoc} */
- @Override
- public String toString() {
- return "key=" + key;
+ @Override public String toString() {
+ return S.toString(SimpleDataRow.class, this, "len", val.length);
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetaStorage.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetaStorage.java
index cfab287..331da92 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetaStorage.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetaStorage.java
@@ -50,7 +50,6 @@ import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageInitRecord;
import org.apache.ignite.internal.processors.cache.CacheGroupContext;
import org.apache.ignite.internal.processors.cache.GridCacheProcessor;
import org.apache.ignite.internal.processors.cache.GridCacheSharedContext;
-import org.apache.ignite.internal.processors.cache.IncompleteObject;
import org.apache.ignite.internal.processors.cache.persistence.DataRegion;
import org.apache.ignite.internal.processors.cache.persistence.DataRegionMetricsImpl;
import org.apache.ignite.internal.processors.cache.persistence.DbCheckpointListener;
@@ -58,15 +57,10 @@ import org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabase
import org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager;
import org.apache.ignite.internal.processors.cache.persistence.RootPage;
import org.apache.ignite.internal.processors.cache.persistence.StorageException;
-import org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList;
import org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryEx;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.DataPagePayload;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
+import org.apache.ignite.internal.processors.cache.persistence.partstorage.PartitionMetaStorageImpl;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PagePartitionMetaIO;
-import org.apache.ignite.internal.processors.cache.persistence.tree.io.SimpleDataPageIO;
-import org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseList;
import org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler;
import org.apache.ignite.internal.processors.failure.FailureProcessor;
import org.apache.ignite.internal.util.lang.GridCursor;
@@ -79,8 +73,6 @@ import org.jetbrains.annotations.NotNull;
import static org.apache.ignite.internal.pagemem.PageIdAllocator.FLAG_DATA;
import static org.apache.ignite.internal.pagemem.PageIdAllocator.OLD_METASTORE_PARTITION;
-import static org.apache.ignite.internal.pagemem.PageIdUtils.itemId;
-import static org.apache.ignite.internal.pagemem.PageIdUtils.pageId;
/**
* General purpose key-value local-only storage.
@@ -104,7 +96,6 @@ public class MetaStorage implements DbCheckpointListener, ReadWriteMetastorage {
/** Temporary metastorage buffer size (file). */
private static final int TEMPORARY_METASTORAGE_BUFFER_SIZE = 1024 * 1024;
-
/** */
private final IgniteWriteAheadLogManager wal;
@@ -136,7 +127,7 @@ public class MetaStorage implements DbCheckpointListener, ReadWriteMetastorage {
private RootPage reuseListRoot;
/** */
- private FreeListImpl freeList;
+ private PartitionMetaStorageImpl<MetastorageDataRow> partStorage;
/** */
private SortedMap<String, byte[]> lastUpdates;
@@ -245,14 +236,18 @@ public class MetaStorage implements DbCheckpointListener, ReadWriteMetastorage {
getOrAllocateMetas(partId = PageIdAllocator.METASTORE_PARTITION);
if (!empty) {
- freeList = new FreeListImpl(METASTORAGE_CACHE_ID, "metastorage",
+ partStorage = new PartitionMetaStorageImpl<MetastorageDataRow>(METASTORAGE_CACHE_ID, "metastorage",
regionMetrics, dataRegion, null, wal, reuseListRoot.pageId().pageId(),
- reuseListRoot.isAllocated());
+ reuseListRoot.isAllocated()) {
+ @Override protected long allocatePageNoReuse() throws IgniteCheckedException {
+ return pageMem.allocatePage(grpId, partId, FLAG_DATA);
+ }
+ };
- MetastorageRowStore rowStore = new MetastorageRowStore(freeList, db);
+ MetastorageRowStore rowStore = new MetastorageRowStore(partStorage, db);
tree = new MetastorageTree(METASTORAGE_CACHE_ID, dataRegion.pageMemory(), wal, rmvId,
- freeList, rowStore, treeRoot.pageId().pageId(), treeRoot.isAllocated(), failureProcessor, partId);
+ partStorage, rowStore, treeRoot.pageId().pageId(), treeRoot.isAllocated(), failureProcessor, partId);
if (!readOnly)
((GridCacheDatabaseSharedManager)db).addCheckpointListener(this);
@@ -578,14 +573,14 @@ public class MetaStorage implements DbCheckpointListener, ReadWriteMetastorage {
Executor executor = ctx.executor();
if (executor == null) {
- freeList.saveMetadata();
+ partStorage.saveMetadata();
saveStoreMetadata();
}
else {
executor.execute(() -> {
try {
- freeList.saveMetadata();
+ partStorage.saveMetadata();
}
catch (IgniteCheckedException e) {
throw new IgniteException(e);
@@ -605,7 +600,7 @@ public class MetaStorage implements DbCheckpointListener, ReadWriteMetastorage {
/** {@inheritDoc} */
@Override public void beforeCheckpointBegin(Context ctx) throws IgniteCheckedException {
- freeList.saveMetadata();
+ partStorage.saveMetadata();
}
/** {@inheritDoc} */
@@ -663,108 +658,6 @@ public class MetaStorage implements DbCheckpointListener, ReadWriteMetastorage {
}
}
- /** */
- public class FreeListImpl extends AbstractFreeList<MetastorageDataRow> {
- /** {@inheritDoc} */
- FreeListImpl(int cacheId, String name, DataRegionMetricsImpl regionMetrics, DataRegion dataRegion,
- ReuseList reuseList,
- IgniteWriteAheadLogManager wal, long metaPageId, boolean initNew) throws IgniteCheckedException {
- super(cacheId, name, regionMetrics, dataRegion, reuseList, wal, metaPageId, initNew);
- }
-
- /** {@inheritDoc} */
- @Override public IOVersions<? extends AbstractDataPageIO<MetastorageDataRow>> ioVersions() {
- return SimpleDataPageIO.VERSIONS;
- }
-
- /** {@inheritDoc} */
- @Override protected long allocatePageNoReuse() throws IgniteCheckedException {
- return pageMem.allocatePage(grpId, partId, FLAG_DATA);
- }
-
- /**
- * Read row from data pages.
- */
- final MetastorageDataRow readRow(String key, long link)
- throws IgniteCheckedException {
- assert link != 0 : "link";
-
- long nextLink = link;
- IncompleteObject incomplete = null;
- int size = 0;
-
- boolean first = true;
-
- do {
- final long pageId = pageId(nextLink);
-
- final long page = pageMem.acquirePage(grpId, pageId);
-
- try {
- long pageAddr = pageMem.readLock(grpId, pageId, page); // Non-empty data page must not be recycled.
-
- assert pageAddr != 0L : nextLink;
-
- try {
- SimpleDataPageIO io = (SimpleDataPageIO)ioVersions().forPage(pageAddr);
-
- //MetaStorage never encrypted so realPageSize == pageSize.
- DataPagePayload data = io.readPayload(pageAddr, itemId(nextLink), pageMem.pageSize());
-
- nextLink = data.nextLink();
-
- if (first) {
- if (nextLink == 0) {
- // Fast path for a single page row.
- return new MetastorageDataRow(link, key, SimpleDataPageIO.readPayload(pageAddr + data.offset()));
- }
-
- first = false;
- }
-
- ByteBuffer buf = pageMem.pageBuffer(pageAddr);
-
- buf.position(data.offset());
- buf.limit(data.offset() + data.payloadSize());
-
- if (size == 0) {
- if (buf.remaining() >= 4 && incomplete == null) {
- // Just read size.
- size = buf.getInt();
- incomplete = new IncompleteObject(new byte[size]);
- }
- else {
- if (incomplete == null)
- incomplete = new IncompleteObject(new byte[4]);
-
- incomplete.readData(buf);
-
- if (incomplete.isReady()) {
- size = ByteBuffer.wrap(incomplete.data()).order(buf.order()).getInt();
- incomplete = new IncompleteObject(new byte[size]);
- }
- }
- }
-
- if (size != 0 && buf.remaining() > 0)
- incomplete.readData(buf);
- }
- finally {
- pageMem.readUnlock(grpId, pageId, page);
- }
- }
- finally {
- pageMem.releasePage(grpId, pageId, page);
- }
- }
- while (nextLink != 0);
-
- assert incomplete.isReady();
-
- return new MetastorageDataRow(link, key, incomplete.data());
- }
- }
-
/**
* Temporary storage internal
*/
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageDataRow.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageDataRow.java
index 2d7b0a6..a84924a 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageDataRow.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageDataRow.java
@@ -17,34 +17,28 @@
package org.apache.ignite.internal.processors.cache.persistence.metastorage;
-import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.internal.pagemem.PageIdAllocator;
-import org.apache.ignite.internal.processors.cache.persistence.Storable;
+import org.apache.ignite.internal.processors.cache.persistence.freelist.SimpleDataRow;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
/**
*
*/
-public class MetastorageDataRow implements MetastorageSearchRow, Storable {
- /** */
- private long link;
-
+public class MetastorageDataRow extends SimpleDataRow implements MetastorageSearchRow {
/** */
private String key;
/** */
- private byte[] value;
-
- /** */
- public MetastorageDataRow(long link, String key, byte[] value) {
- this.link = link;
+ public MetastorageDataRow(long link, String key, byte[] val) {
+ super(link, MetaStorage.PRESERVE_LEGACY_METASTORAGE_PARTITION_ID ?
+ PageIdAllocator.OLD_METASTORE_PARTITION: PageIdAllocator.METASTORE_PARTITION, val);
this.key = key;
- this.value = value;
}
/** */
- public MetastorageDataRow(String key, byte[] value) {
- this.key = key;
- this.value = value;
+ public MetastorageDataRow(String key, byte[] val) {
+ this(0, key, val);
}
/**
@@ -55,49 +49,17 @@ public class MetastorageDataRow implements MetastorageSearchRow, Storable {
}
/** {@inheritDoc} */
- @Override
- public int hash() {
+ @Override public int hash() {
return key.hashCode();
}
/** {@inheritDoc} */
- @Override
- public int partition() {
- return MetaStorage.PRESERVE_LEGACY_METASTORAGE_PARTITION_ID ? PageIdAllocator.OLD_METASTORE_PARTITION: PageIdAllocator.METASTORE_PARTITION;
- }
-
- /** {@inheritDoc} */
- @Override public int size() throws IgniteCheckedException {
- return 4 + value().length;
- }
-
- /** {@inheritDoc} */
- @Override public int headerSize() {
- return 0;
- }
-
- /** {@inheritDoc} */
- @Override
- public void link(long link) {
- this.link = link;
- }
-
- /** {@inheritDoc} */
- @Override
- public long link() {
- return link;
- }
-
- /**
- * @return Value.
- */
- public byte[] value() {
- return value;
+ @Override public IOVersions<? extends AbstractDataPageIO> ioVersions() {
+ return MetastoreDataPageIO.VERSIONS;
}
/** {@inheritDoc} */
- @Override
- public String toString() {
+ @Override public String toString() {
return "key=" + key;
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageRowStore.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageRowStore.java
index c482a94..ccc55dc 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageRowStore.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastorageRowStore.java
@@ -19,7 +19,7 @@ package org.apache.ignite.internal.processors.cache.persistence.metastorage;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager;
-import org.apache.ignite.internal.processors.cache.persistence.freelist.FreeList;
+import org.apache.ignite.internal.processors.cache.persistence.partstorage.PartitionMetaStorage;
import org.apache.ignite.internal.stat.IoStatisticsHolderNoOp;
/**
@@ -27,14 +27,14 @@ import org.apache.ignite.internal.stat.IoStatisticsHolderNoOp;
*/
public class MetastorageRowStore {
/** */
- private final FreeList freeList;
+ private final PartitionMetaStorage<MetastorageDataRow> partStorage;
/** */
protected final IgniteCacheDatabaseSharedManager db;
/** */
- public MetastorageRowStore(FreeList freeList, IgniteCacheDatabaseSharedManager db) {
- this.freeList = freeList;
+ public MetastorageRowStore(PartitionMetaStorage<MetastorageDataRow> partStorage, IgniteCacheDatabaseSharedManager db) {
+ this.partStorage = partStorage;
this.db = db;
}
@@ -43,7 +43,7 @@ public class MetastorageRowStore {
* @return Data row.
*/
public MetastorageDataRow dataRow(String key, long link) throws IgniteCheckedException {
- return ((MetaStorage.FreeListImpl)freeList).readRow(key, link);
+ return new MetastorageDataRow(link, key, partStorage.readRow(link));
}
/**
@@ -55,7 +55,7 @@ public class MetastorageRowStore {
db.checkpointReadLock();
try {
- freeList.removeDataRowByLink(link, IoStatisticsHolderNoOp.INSTANCE);
+ partStorage.removeDataRowByLink(link, IoStatisticsHolderNoOp.INSTANCE);
}
finally {
db.checkpointReadUnlock();
@@ -70,28 +70,10 @@ public class MetastorageRowStore {
db.checkpointReadLock();
try {
- freeList.insertDataRow(row, IoStatisticsHolderNoOp.INSTANCE);
+ partStorage.insertDataRow(row, IoStatisticsHolderNoOp.INSTANCE);
}
finally {
db.checkpointReadUnlock();
}
}
-
- /**
- * @param link Row link.
- * @param row New row data.
- * @return {@code True} if was able to update row.
- * @throws IgniteCheckedException If failed.
- */
- public boolean updateRow(long link, MetastorageDataRow row) throws IgniteCheckedException {
- return freeList.updateDataRow(link, row, IoStatisticsHolderNoOp.INSTANCE);
- }
-
- /**
- * @return Free list.
- */
- public FreeList freeList() {
- return freeList;
- }
-
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastoreDataPageIO.java
similarity index 52%
copy from modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
copy to modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastoreDataPageIO.java
index 133f0a1..0ac0915 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/metastorage/MetastoreDataPageIO.java
@@ -15,38 +15,23 @@
* limitations under the License.
*/
-package org.apache.ignite.internal.processors.cache.persistence;
+package org.apache.ignite.internal.processors.cache.persistence.metastorage;
-import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.SimpleDataPageIO;
/**
- * Simple interface for data, store in some RowStore.
*/
-public interface Storable {
- /**
- * @param link Link for this row.
- */
- public void link(long link);
-
- /**
- * @return Link for this row.
- */
- public long link();
-
- /**
- * @return Partition.
- */
- public int partition();
-
- /**
- * @return Row size in page.
- * @throws IgniteCheckedException If failed.
- */
- public int size() throws IgniteCheckedException;
+public class MetastoreDataPageIO extends SimpleDataPageIO {
+ /** */
+ public static final IOVersions<MetastoreDataPageIO> VERSIONS = new IOVersions<>(
+ new MetastoreDataPageIO(1)
+ );
/**
- * @return Row header size in page. Header is indivisible part of row
- * which is entirely available on the very first page followed by the row link.
+ * @param ver Version.
*/
- public int headerSize();
+ public MetastoreDataPageIO(int ver) {
+ super(T_DATA_METASTORAGE, ver);
+ }
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/partstorage/PartitionMetaStorage.java
similarity index 51%
copy from modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
copy to modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/partstorage/PartitionMetaStorage.java
index 133f0a1..0dbd5f4 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/Storable.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/partstorage/PartitionMetaStorage.java
@@ -15,38 +15,37 @@
* limitations under the License.
*/
-package org.apache.ignite.internal.processors.cache.persistence;
+package org.apache.ignite.internal.processors.cache.persistence.partstorage;
import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.internal.processors.cache.persistence.Storable;
+import org.apache.ignite.internal.stat.IoStatisticsHolder;
/**
- * Simple interface for data, store in some RowStore.
+ * Provides a way to associate any {@link Storable} implementation as partition metadata.
*/
-public interface Storable {
+public interface PartitionMetaStorage<T extends Storable> {
/**
- * @param link Link for this row.
- */
- public void link(long link);
-
- /**
- * @return Link for this row.
+ * Read row data by link as byte array.
+ * @param link Link.
+ * @throws IgniteCheckedException If failed.
*/
- public long link();
+ public byte[] readRow(long link) throws IgniteCheckedException;
/**
- * @return Partition.
+ * @param row Row.
+ * @param statHolder Stat holder.
*/
- public int partition();
+ public void insertDataRow(T row, IoStatisticsHolder statHolder) throws IgniteCheckedException;
/**
- * @return Row size in page.
+ * @param link Row link.
* @throws IgniteCheckedException If failed.
*/
- public int size() throws IgniteCheckedException;
+ public void removeDataRowByLink(long link, IoStatisticsHolder statHolder) throws IgniteCheckedException;
/**
- * @return Row header size in page. Header is indivisible part of row
- * which is entirely available on the very first page followed by the row link.
+ * Saves storage metadata.
*/
- public int headerSize();
+ public void saveMetadata() throws IgniteCheckedException;
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/partstorage/PartitionMetaStorageImpl.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/partstorage/PartitionMetaStorageImpl.java
new file mode 100644
index 0000000..f078f2c
--- /dev/null
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/partstorage/PartitionMetaStorageImpl.java
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.cache.persistence.partstorage;
+
+import java.nio.ByteBuffer;
+import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.internal.pagemem.PageUtils;
+import org.apache.ignite.internal.pagemem.wal.IgniteWriteAheadLogManager;
+import org.apache.ignite.internal.processors.cache.IncompleteObject;
+import org.apache.ignite.internal.processors.cache.persistence.DataRegion;
+import org.apache.ignite.internal.processors.cache.persistence.DataRegionMetricsImpl;
+import org.apache.ignite.internal.processors.cache.persistence.Storable;
+import org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.DataPagePayload;
+import org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO;
+import org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseList;
+
+import static org.apache.ignite.internal.pagemem.PageIdUtils.itemId;
+import static org.apache.ignite.internal.pagemem.PageIdUtils.pageId;
+
+/**
+ */
+public class PartitionMetaStorageImpl<T extends Storable> extends AbstractFreeList<T> implements PartitionMetaStorage<T> {
+ /**
+ * @param cacheId Cache id.
+ * @param name Name.
+ * @param memMetrics Mem metrics.
+ * @param memPlc Mem policy.
+ * @param reuseList Reuse list.
+ * @param wal Wal.
+ * @param metaPageId Meta page id.
+ * @param initNew Initialize new.
+ */
+ public PartitionMetaStorageImpl(int cacheId, String name,
+ DataRegionMetricsImpl memMetrics,
+ DataRegion memPlc,
+ ReuseList reuseList,
+ IgniteWriteAheadLogManager wal, long metaPageId, boolean initNew) throws IgniteCheckedException {
+ super(cacheId, name, memMetrics, memPlc, reuseList, wal, metaPageId, initNew);
+ }
+
+ /**
+ * Read row as byte array from data pages.
+ */
+ @Override public final byte[] readRow(long link) throws IgniteCheckedException {
+ assert link != 0 : "link";
+
+ long nextLink = link;
+ IncompleteObject incomplete = null;
+ int size = 0;
+
+ boolean first = true;
+
+ do {
+ final long pageId = pageId(nextLink);
+
+ final long page = pageMem.acquirePage(grpId, pageId);
+
+ try {
+ long pageAddr = pageMem.readLock(grpId, pageId, page); // Non-empty data page must not be recycled.
+
+ assert pageAddr != 0L : nextLink;
+
+ try {
+ AbstractDataPageIO io = PageIO.getPageIO(pageAddr);
+
+ //MetaStorage never encrypted so realPageSize == pageSize.
+ DataPagePayload data = io.readPayload(pageAddr, itemId(nextLink), pageMem.pageSize());
+
+ nextLink = data.nextLink();
+
+ if (first) {
+ if (nextLink == 0) {
+ long payloadAddr = pageAddr + data.offset();
+
+ // Fast path for a single page row.
+ return PageUtils.getBytes(payloadAddr, 4, PageUtils.getInt(payloadAddr, 0));
+ }
+
+ first = false;
+ }
+
+ ByteBuffer buf = pageMem.pageBuffer(pageAddr);
+
+ buf.position(data.offset());
+ buf.limit(data.offset() + data.payloadSize());
+
+ if (size == 0) {
+ if (buf.remaining() >= 4 && incomplete == null) {
+ // Just read size.
+ size = buf.getInt();
+ incomplete = new IncompleteObject(new byte[size]);
+ }
+ else {
+ if (incomplete == null)
+ incomplete = new IncompleteObject(new byte[4]);
+
+ incomplete.readData(buf);
+
+ if (incomplete.isReady()) {
+ size = ByteBuffer.wrap(incomplete.data()).order(buf.order()).getInt();
+ incomplete = new IncompleteObject(new byte[size]);
+ }
+ }
+ }
+
+ if (size != 0 && buf.remaining() > 0)
+ incomplete.readData(buf);
+ }
+ finally {
+ pageMem.readUnlock(grpId, pageId, page);
+ }
+ }
+ finally {
+ pageMem.releasePage(grpId, pageId, page);
+ }
+ }
+ while (nextLink != 0);
+
+ assert incomplete.isReady();
+
+ return incomplete.data();
+ }
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/BPlusTree.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/BPlusTree.java
index 54d9816..c131a1e 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/BPlusTree.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/BPlusTree.java
@@ -3775,6 +3775,7 @@ public abstract class BPlusTree<L, T extends L> extends DataStructure implements
break;
case NOOP:
+ case IN_PLACE:
return;
default:
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java
index 78752bb..a2e03e9 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/AbstractDataPageIO.java
@@ -39,7 +39,6 @@ import static org.apache.ignite.internal.util.GridUnsafe.bufferAddress;
* Data pages IO.
*/
public abstract class AbstractDataPageIO<T extends Storable> extends PageIO implements CompactablePageIO {
-
/** */
private static final int SHOW_ITEM = 0b0001;
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PageIO.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PageIO.java
index dcf1acd..fafea22 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PageIO.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PageIO.java
@@ -31,6 +31,7 @@ import org.apache.ignite.internal.processors.cache.persistence.IndexStorageImpl;
import org.apache.ignite.internal.processors.cache.persistence.freelist.io.PagesListMetaIO;
import org.apache.ignite.internal.processors.cache.persistence.freelist.io.PagesListNodeIO;
import org.apache.ignite.internal.processors.cache.persistence.metastorage.MetastorageTree;
+import org.apache.ignite.internal.processors.cache.persistence.metastorage.MetastoreDataPageIO;
import org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler;
import org.apache.ignite.internal.processors.cache.persistence.tree.util.PageLockListener;
import org.apache.ignite.internal.processors.cache.tree.CacheIdAwareDataInnerIO;
@@ -251,6 +252,9 @@ public abstract class PageIO {
/** */
public static final short T_TX_LOG_INNER = 31;
+ /** */
+ public static final short T_DATA_PART = 32;
+
/** Index for payload == 1. */
public static final short T_H2_EX_REF_LEAF_START = 10_000;
@@ -667,6 +671,9 @@ public abstract class PageIO {
return (Q)TrackingPageIO.VERSIONS.forVersion(ver);
case T_DATA_METASTORAGE:
+ return (Q)MetastoreDataPageIO.VERSIONS.forVersion(ver);
+
+ case T_DATA_PART:
return (Q)SimpleDataPageIO.VERSIONS.forVersion(ver);
default:
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIO.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIO.java
index 6a81057..c86a64a 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIO.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIO.java
@@ -194,6 +194,40 @@ public class PagePartitionMetaIO extends PageMetaIO {
"this PagePartitionMetaIO version: ver=" + getVersion());
}
+ /**
+ * @param pageAddr Page address.
+ */
+ public long getPartitionMetaStoreReuseListRoot(long pageAddr) {
+ throw new UnsupportedOperationException("Partition metastore is not supported by " +
+ "this PagePartitionMetaIO version: ver=" + getVersion());
+ }
+
+ /**
+ * @param pageAddr Page address.
+ * @param listRoot List root.
+ */
+ public void setPartitionMetaStoreReuseListRoot(long pageAddr, long listRoot) {
+ throw new UnsupportedOperationException("Partition metastore is not supported by " +
+ "this PagePartitionMetaIO version: ver=" + getVersion());
+ }
+
+ /**
+ * @param pageAddr Page address.
+ */
+ public long getGapsLink(long pageAddr) {
+ throw new UnsupportedOperationException("Gaps link is not supported by " +
+ "this PagePartitionMetaIO version: ver=" + getVersion());
+ }
+
+ /**
+ * @param pageAddr Page address.
+ * @param link Link.
+ */
+ public boolean setGapsLink(long pageAddr, long link) {
+ throw new UnsupportedOperationException("Gaps link is not supported by " +
+ "this PagePartitionMetaIO version: ver=" + getVersion());
+ }
+
/** {@inheritDoc} */
@Override protected void printPage(long pageAddr, int pageSize, GridStringBuilder sb) throws IgniteCheckedException {
super.printPage(pageAddr, pageSize, sb);
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIOV2.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIOV2.java
index 220ff83..37b7243 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIOV2.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/PagePartitionMetaIOV2.java
@@ -31,6 +31,12 @@ public class PagePartitionMetaIOV2 extends PagePartitionMetaIO {
/** */
private static final int PENDING_TREE_ROOT_OFF = PagePartitionMetaIO.END_OF_PARTITION_PAGE_META;
+ /** */
+ private static final int PART_META_REUSE_LIST_ROOT_OFF = PENDING_TREE_ROOT_OFF + 8;
+
+ /** */
+ private static final int GAPS_LINK = PART_META_REUSE_LIST_ROOT_OFF + 8;
+
/**
* @param ver Version.
*/
@@ -43,6 +49,8 @@ public class PagePartitionMetaIOV2 extends PagePartitionMetaIO {
super.initNewPage(pageAddr, pageId, pageSize);
setPendingTreeRoot(pageAddr, 0L);
+ setPartitionMetaStoreReuseListRoot(pageAddr, 0L);
+ setGapsLink(pageAddr, 0);
}
/** {@inheritDoc} */
@@ -51,8 +59,44 @@ public class PagePartitionMetaIOV2 extends PagePartitionMetaIO {
}
/** {@inheritDoc} */
- @Override public void setPendingTreeRoot(long pageAddr, long treeRoot) {
- PageUtils.putLong(pageAddr, PENDING_TREE_ROOT_OFF, treeRoot);
+ @Override public void setPendingTreeRoot(long pageAddr, long listRoot) {
+ PageUtils.putLong(pageAddr, PENDING_TREE_ROOT_OFF, listRoot);
+ }
+
+ /**
+ * @param pageAddr Page address.
+ */
+ @Override public long getPartitionMetaStoreReuseListRoot(long pageAddr) {
+ return PageUtils.getLong(pageAddr, PART_META_REUSE_LIST_ROOT_OFF);
+ }
+
+ /**
+ * @param pageAddr Page address.
+ * @param listRoot List root.
+ */
+ @Override public void setPartitionMetaStoreReuseListRoot(long pageAddr, long listRoot) {
+ PageUtils.putLong(pageAddr, PART_META_REUSE_LIST_ROOT_OFF, listRoot);
+ }
+
+ /**
+ * @param pageAddr Page address.
+ * @return Partition size.
+ */
+ public long getGapsLink(long pageAddr) {
+ return PageUtils.getLong(pageAddr, GAPS_LINK);
+ }
+
+ /**
+ * @param pageAddr Page address.
+ * @param link Link.
+ */
+ public boolean setGapsLink(long pageAddr, long link) {
+ if (getGapsLink(pageAddr) == link)
+ return false;
+
+ PageUtils.putLong(pageAddr, GAPS_LINK, link);
+
+ return true;
}
/** {@inheritDoc} */
@@ -72,6 +116,7 @@ public class PagePartitionMetaIOV2 extends PagePartitionMetaIO {
sb.a(",\n\tglobalRemoveId=").a(getGlobalRemoveId(pageAddr));
sb.a(",\n\tpartitionState=").a(state).a("(").a(GridDhtPartitionState.fromOrdinal(state)).a(")");
sb.a(",\n\tcountersPageId=").a(getCountersPageId(pageAddr));
+ sb.a(",\n\tcntrUpdDataPageId=").a(getGapsLink(pageAddr));
sb.a("\n]");
}
@@ -86,5 +131,7 @@ public class PagePartitionMetaIOV2 extends PagePartitionMetaIO {
PageIO.setVersion(pageAddr, getVersion());
setPendingTreeRoot(pageAddr, 0);
+ setPartitionMetaStoreReuseListRoot(pageAddr, 0);
+ setGapsLink(pageAddr, 0);
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/SimpleDataPageIO.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/SimpleDataPageIO.java
index 14489b7..716f8be 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/SimpleDataPageIO.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/io/SimpleDataPageIO.java
@@ -20,13 +20,13 @@ package org.apache.ignite.internal.processors.cache.persistence.tree.io;
import java.nio.ByteBuffer;
import org.apache.ignite.IgniteCheckedException;
import org.apache.ignite.internal.pagemem.PageUtils;
-import org.apache.ignite.internal.processors.cache.persistence.metastorage.MetastorageDataRow;
+import org.apache.ignite.internal.processors.cache.persistence.freelist.SimpleDataRow;
import org.apache.ignite.internal.util.GridStringBuilder;
/**
- * Data pages IO for Metastorage.
+ * Data pages IO for writing binary arrays.
*/
-public class SimpleDataPageIO extends AbstractDataPageIO<MetastorageDataRow> {
+public class SimpleDataPageIO extends AbstractDataPageIO<SimpleDataRow> {
/** */
public static final IOVersions<SimpleDataPageIO> VERSIONS = new IOVersions<>(
new SimpleDataPageIO(1)
@@ -36,13 +36,21 @@ public class SimpleDataPageIO extends AbstractDataPageIO<MetastorageDataRow> {
* @param ver Page format version.
*/
public SimpleDataPageIO(int ver) {
- super(T_DATA_METASTORAGE, ver);
+ super(T_DATA_PART, ver);
+ }
+
+ /**
+ * Constructor is intended for extending types.
+ * @param type IO type.
+ * @param ver Page format version.
+ */
+ public SimpleDataPageIO(int type, int ver) {
+ super(type, ver);
}
/** {@inheritDoc} */
- @Override
- protected void writeFragmentData(
- final MetastorageDataRow row,
+ @Override protected void writeFragmentData(
+ final SimpleDataRow row,
final ByteBuffer buf,
final int rowOff,
final int payloadSize
@@ -65,7 +73,7 @@ public class SimpleDataPageIO extends AbstractDataPageIO<MetastorageDataRow> {
}
/** */
- private int writeSizeFragment(final MetastorageDataRow row, final ByteBuffer buf, final int rowOff,
+ private int writeSizeFragment(final SimpleDataRow row, final ByteBuffer buf, final int rowOff,
final int payloadSize) {
final int size = 4;
@@ -89,12 +97,11 @@ public class SimpleDataPageIO extends AbstractDataPageIO<MetastorageDataRow> {
}
/** {@inheritDoc} */
- @Override
- protected void writeRowData(
+ @Override protected void writeRowData(
long pageAddr,
int dataOff,
int payloadSize,
- MetastorageDataRow row,
+ SimpleDataRow row,
boolean newRow
) throws IgniteCheckedException {
long addr = pageAddr + dataOff;
@@ -106,12 +113,6 @@ public class SimpleDataPageIO extends AbstractDataPageIO<MetastorageDataRow> {
PageUtils.putBytes(addr, 6, row.value());
}
- public static byte[] readPayload(long link) {
- int size = PageUtils.getInt(link, 0);
-
- return PageUtils.getBytes(link, 4, size);
- }
-
/** {@inheritDoc} */
@Override protected void printPage(long addr, int pageSize, GridStringBuilder sb) throws IgniteCheckedException {
sb.a("SimpleDataPageIO [\n");
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/util/PageHandler.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/util/PageHandler.java
index 5ab1bf3..ff980af 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/util/PageHandler.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/tree/util/PageHandler.java
@@ -26,6 +26,7 @@ import org.apache.ignite.internal.pagemem.wal.record.delta.InitNewPageRecord;
import org.apache.ignite.internal.processors.cache.persistence.tree.io.PageIO;
import org.apache.ignite.internal.stat.IoStatisticsHolder;
import org.apache.ignite.internal.util.GridUnsafe;
+import org.jetbrains.annotations.Nullable;
import static java.lang.Boolean.FALSE;
import static java.lang.Boolean.TRUE;
@@ -463,7 +464,7 @@ public abstract class PageHandler<X, R> {
long pageId,
long page,
IgniteWriteAheadLogManager wal,
- Boolean walPlc) {
+ @Nullable Boolean walPlc) {
// If the page is clean, then it is either newly allocated or just after checkpoint.
// In both cases we have to write full page contents to WAL.
return wal != null && !wal.isAlwaysWriteFullPages() && walPlc != TRUE && !wal.disabled(cacheId) &&
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java
index 86924c6..2831d87 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java
@@ -917,7 +917,7 @@ public class FileWriteAheadLogManager extends GridCacheSharedManagerAdapter impl
if (hnd != null)
end = hnd.position();
- return new RecordsIterator(
+ RecordsIterator iter = new RecordsIterator(
cctx,
walArchiveDir,
walWorkDir,
@@ -932,6 +932,17 @@ public class FileWriteAheadLogManager extends GridCacheSharedManagerAdapter impl
segmentAware,
segmentRouter,
lockedSegmentFileInputFactory);
+
+ try {
+ iter.init(); // Make sure iterator is closed on any error.
+ }
+ catch (Throwable t) {
+ iter.close();
+
+ throw t;
+ }
+
+ return iter;
}
/** {@inheritDoc} */
@@ -2673,10 +2684,6 @@ public class FileWriteAheadLogManager extends GridCacheSharedManagerAdapter impl
this.decompressor = decompressor;
this.segmentRouter = segmentRouter;
this.segmentAware = segmentAware;
-
- init();
-
- advance();
}
/** {@inheritDoc} */
@@ -2760,6 +2767,8 @@ public class FileWriteAheadLogManager extends GridCacheSharedManagerAdapter impl
if (log.isDebugEnabled())
log.debug("Initialized WAL cursor [start=" + start + ", end=" + end + ", curWalSegmIdx=" + curWalSegmIdx + ']');
+
+ advance();
}
/** {@inheritDoc} */
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/record/RecordTypes.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/record/RecordTypes.java
index 65f0aae..b6d20dc 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/record/RecordTypes.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/record/RecordTypes.java
@@ -60,6 +60,7 @@ public final class RecordTypes {
DELTA_TYPE_SET.add(WALRecord.RecordType.PAGES_LIST_REMOVE_PAGE);
DELTA_TYPE_SET.add(WALRecord.RecordType.META_PAGE_INIT);
DELTA_TYPE_SET.add(WALRecord.RecordType.PARTITION_META_PAGE_UPDATE_COUNTERS);
+ DELTA_TYPE_SET.add(WALRecord.RecordType.PARTITION_META_PAGE_UPDATE_COUNTERS_V2);
DELTA_TYPE_SET.add(WALRecord.RecordType.TRACKING_PAGE_DELTA);
DELTA_TYPE_SET.add(WALRecord.RecordType.META_PAGE_UPDATE_LAST_SUCCESSFUL_SNAPSHOT_ID);
DELTA_TYPE_SET.add(WALRecord.RecordType.META_PAGE_UPDATE_LAST_SUCCESSFUL_FULL_SNAPSHOT_ID);
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV1Serializer.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV1Serializer.java
index 705cc38..ec68972 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV1Serializer.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV1Serializer.java
@@ -70,6 +70,7 @@ import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageUpdateLastSuc
import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageUpdateLastSuccessfulSnapshotId;
import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageUpdateNextSnapshotId;
import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageUpdatePartitionDataRecord;
+import org.apache.ignite.internal.pagemem.wal.record.delta.MetaPageUpdatePartitionDataRecordV2;
import org.apache.ignite.internal.pagemem.wal.record.delta.NewRootInitRecord;
import org.apache.ignite.internal.pagemem.wal.record.delta.PageListMetaResetCountRecord;
import org.apache.ignite.internal.pagemem.wal.record.delta.PagesListAddPageRecord;
@@ -374,6 +375,10 @@ public class RecordDataV1Serializer implements RecordDataSerializer {
return /*cache ID*/4 + /*page ID*/8 + /*upd cntr*/8 + /*rmv id*/8 + /*part size*/4 + /*counters page id*/8 + /*state*/ 1
+ /*allocatedIdxCandidate*/ 4;
+ case PARTITION_META_PAGE_UPDATE_COUNTERS_V2:
+ return /*cache ID*/4 + /*page ID*/8 + /*upd cntr*/8 + /*rmv id*/8 + /*part size*/4 + /*counters page id*/8 + /*state*/ 1
+ + /*allocatedIdxCandidate*/ 4 + /*link*/ 8;
+
case MEMORY_RECOVERY:
return 8;
@@ -597,17 +602,12 @@ public class RecordDataV1Serializer implements RecordDataSerializer {
break;
case PARTITION_META_PAGE_UPDATE_COUNTERS:
- cacheId = in.readInt();
- pageId = in.readLong();
+ res = new MetaPageUpdatePartitionDataRecord(in);
- long updCntr = in.readLong();
- long rmvId = in.readLong();
- int partSize = in.readInt();
- long countersPageId = in.readLong();
- byte state = in.readByte();
- int allocatedIdxCandidate = in.readInt();
+ break;
- res = new MetaPageUpdatePartitionDataRecord(cacheId, pageId, updCntr, rmvId, partSize, countersPageId, state, allocatedIdxCandidate);
+ case PARTITION_META_PAGE_UPDATE_COUNTERS_V2:
+ res = new MetaPageUpdatePartitionDataRecordV2(in);
break;
@@ -950,7 +950,7 @@ public class RecordDataV1Serializer implements RecordDataSerializer {
int dstIdx = in.readUnsignedShort();
long srcPageId = in.readLong();
int srcIdx = in.readUnsignedShort();
- rmvId = in.readLong();
+ long rmvId = in.readLong();
res = new InnerReplaceRecord<>(cacheId, pageId, dstIdx, srcPageId, srcIdx, rmvId);
@@ -1112,7 +1112,7 @@ public class RecordDataV1Serializer implements RecordDataSerializer {
partId = in.readInt();
- state = in.readByte();
+ byte state = in.readByte();
long updateCntr = in.readLong();
@@ -1201,17 +1201,8 @@ public class RecordDataV1Serializer implements RecordDataSerializer {
break;
case PARTITION_META_PAGE_UPDATE_COUNTERS:
- MetaPageUpdatePartitionDataRecord partDataRec = (MetaPageUpdatePartitionDataRecord)rec;
-
- buf.putInt(partDataRec.groupId());
- buf.putLong(partDataRec.pageId());
-
- buf.putLong(partDataRec.updateCounter());
- buf.putLong(partDataRec.globalRemoveId());
- buf.putInt(partDataRec.partitionSize());
- buf.putLong(partDataRec.countersPageId());
- buf.put(partDataRec.state());
- buf.putInt(partDataRec.allocatedIndexCandidate());
+ case PARTITION_META_PAGE_UPDATE_COUNTERS_V2:
+ ((MetaPageUpdatePartitionDataRecord)rec).toBytes(buf);
break;
@@ -1690,7 +1681,7 @@ public class RecordDataV1Serializer implements RecordDataSerializer {
buf.put(partMetaStateRecord.state());
- buf.putLong(partMetaStateRecord.updateCounter());
+ buf.putLong(0);
break;
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV2Serializer.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV2Serializer.java
index ec45a06..3063a25 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV2Serializer.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/serializer/RecordDataV2Serializer.java
@@ -38,6 +38,7 @@ import org.apache.ignite.internal.pagemem.wal.record.MvccDataEntry;
import org.apache.ignite.internal.pagemem.wal.record.MvccDataRecord;
import org.apache.ignite.internal.pagemem.wal.record.MvccTxRecord;
import org.apache.ignite.internal.pagemem.wal.record.PageSnapshot;
+import org.apache.ignite.internal.pagemem.wal.record.RollbackRecord;
import org.apache.ignite.internal.pagemem.wal.record.SnapshotRecord;
import org.apache.ignite.internal.pagemem.wal.record.TxRecord;
import org.apache.ignite.internal.pagemem.wal.record.WALRecord;
@@ -111,6 +112,9 @@ public class RecordDataV2Serializer extends RecordDataV1Serializer {
case MVCC_TX_RECORD:
return txRecordSerializer.size((MvccTxRecord)rec);
+ case ROLLBACK_TX_RECORD:
+ return 4 + 4 + 8 + 8;
+
default:
return super.plainSize(rec);
}
@@ -206,6 +210,14 @@ public class RecordDataV2Serializer extends RecordDataV1Serializer {
case MVCC_TX_RECORD:
return txRecordSerializer.readMvccTx(in);
+ case ROLLBACK_TX_RECORD:
+ int grpId = in.readInt();
+ int partId = in.readInt();
+ long start = in.readLong();
+ long range = in.readLong();
+
+ return new RollbackRecord(grpId, partId, start, range);
+
default:
return super.readPlainRecord(type, in, encrypted, recordSize);
}
@@ -288,6 +300,16 @@ public class RecordDataV2Serializer extends RecordDataV1Serializer {
break;
+ case ROLLBACK_TX_RECORD:
+ RollbackRecord rb = (RollbackRecord)rec;
+
+ buf.putInt(rb.groupId());
+ buf.putInt(rb.partitionId());
+ buf.putLong(rb.start());
+ buf.putLong(rb.range());
+
+ break;
+
default:
super.writePlainRecord(rec, buf);
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryEventBuffer.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryEventBuffer.java
index c707c72..82400f2 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryEventBuffer.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryEventBuffer.java
@@ -114,9 +114,10 @@ public class CacheContinuousQueryEventBuffer {
}
/**
+ * @param backup {@code True} if backup context.
* @return Initial partition counter.
*/
- protected long currentPartitionCounter() {
+ protected long currentPartitionCounter(boolean backup) {
return 0;
}
@@ -153,7 +154,7 @@ public class CacheContinuousQueryEventBuffer {
Object res = null;
for (;;) {
- batch = initBatch(entry.topologyVersion());
+ batch = initBatch(entry.topologyVersion(), backup);
if (batch == null || cntr < batch.startCntr) {
if (backup) {
@@ -185,7 +186,7 @@ public class CacheContinuousQueryEventBuffer {
res = processPending(res, batch, backup);
- batch0 = initBatch(entry.topologyVersion());
+ batch0 = initBatch(entry.topologyVersion(), backup);
}
while (batch != batch0);
}
@@ -195,16 +196,17 @@ public class CacheContinuousQueryEventBuffer {
/**
* @param topVer Current event topology version.
+ * @param backup {@code True} if backup entry.
* @return Current batch.
*/
- @Nullable private Batch initBatch(AffinityTopologyVersion topVer) {
+ private Batch initBatch(AffinityTopologyVersion topVer, boolean backup) {
Batch batch = curBatch.get();
if (batch != null)
return batch;
for (;;) {
- long curCntr = currentPartitionCounter();
+ long curCntr = currentPartitionCounter(backup);
if (curCntr == -1)
return null;
@@ -483,4 +485,4 @@ public class CacheContinuousQueryEventBuffer {
return res;
}
}
-}
\ No newline at end of file
+}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java
index 7f9b918..d8737bd 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/query/continuous/CacheContinuousQueryHandler.java
@@ -54,6 +54,7 @@ import org.apache.ignite.internal.processors.cache.GridCacheAdapter;
import org.apache.ignite.internal.processors.cache.GridCacheAffinityManager;
import org.apache.ignite.internal.processors.cache.GridCacheContext;
import org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager;
+import org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition;
import org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicAbstractUpdateFuture;
import org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition;
import org.apache.ignite.internal.processors.cache.query.CacheQueryType;
@@ -1106,13 +1107,14 @@ public class CacheContinuousQueryHandler<K, V> implements GridContinuousHandler
if (buf == null) {
buf = new CacheContinuousQueryEventBuffer(part) {
- @Override protected long currentPartitionCounter() {
+ @Override protected long currentPartitionCounter(boolean backup) {
GridDhtLocalPartition locPart = cctx.topology().localPartition(part, null, false);
if (locPart == null)
return -1L;
- return locPart.updateCounter();
+ // Use HWM for primary, LWM for backup.
+ return backup ? locPart.updateCounter() : locPart.reservedCounter();
}
};
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxEntry.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxEntry.java
index 7d0f988..a105a2f 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxEntry.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxEntry.java
@@ -40,6 +40,7 @@ import org.apache.ignite.internal.processors.cache.GridCacheSharedContext;
import org.apache.ignite.internal.processors.cache.KeyCacheObject;
import org.apache.ignite.internal.processors.cache.distributed.IgniteExternalizableExpiryPolicy;
import org.apache.ignite.internal.processors.cache.version.GridCacheVersion;
+import org.apache.ignite.internal.util.lang.GridAbsClosureX;
import org.apache.ignite.internal.util.lang.GridPeerDeployAware;
import org.apache.ignite.internal.util.tostring.GridToStringBuilder;
import org.apache.ignite.internal.util.tostring.GridToStringExclude;
@@ -212,6 +213,11 @@ public class IgniteTxEntry implements GridPeerDeployAware, Message {
/** */
private GridCacheVersion serReadVer;
+ /** */
+ @GridDirectTransient
+ @GridToStringExclude
+ private transient @Nullable GridAbsClosureX cqNotifyC;
+
/**
* Required by {@link Externalizable}
*/
@@ -1262,6 +1268,19 @@ public class IgniteTxEntry implements GridPeerDeployAware, Message {
key.getClass() : val != null ? val.getClass() : getClass();
}
+ /**
+ */
+ public GridAbsClosureX cqNotifyClosure() {
+ return cqNotifyC;
+ }
+
+ /**
+ * @param clo Clo.
+ */
+ public void cqNotifyClosure(GridAbsClosureX clo) {
+ cqNotifyC = clo;
+ }
+
/** {@inheritDoc} */
@Override public ClassLoader classLoader() {
return deployClass().getClassLoader();
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxHandler.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxHandler.java
index 719bf04..787fb97 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxHandler.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxHandler.java
@@ -23,6 +23,7 @@ import java.util.List;
import java.util.UUID;
import javax.cache.processor.EntryProcessor;
import org.apache.ignite.IgniteCheckedException;
+import org.apache.ignite.IgniteException;
import org.apache.ignite.IgniteLogger;
import org.apache.ignite.cluster.ClusterNode;
import org.apache.ignite.failure.FailureContext;
@@ -30,6 +31,7 @@ import org.apache.ignite.failure.FailureType;
import org.apache.ignite.internal.IgniteFutureTimeoutCheckedException;
import org.apache.ignite.internal.IgniteInternalFuture;
import org.apache.ignite.internal.cluster.ClusterTopologyCheckedException;
+import org.apache.ignite.internal.pagemem.wal.record.RollbackRecord;
import org.apache.ignite.internal.processors.affinity.AffinityTopologyVersion;
import org.apache.ignite.internal.processors.cache.CacheEntryInfoCollection;
import org.apache.ignite.internal.processors.cache.CacheObject;
@@ -1352,8 +1354,14 @@ public class IgniteTxHandler {
if (dhtTx != null)
finish(nodeId, dhtTx, req);
- else
- applyPartitionsUpdatesCounters(req.updateCounters());
+ else {
+ try {
+ applyPartitionsUpdatesCounters(req.updateCounters(), !req.commit(), false);
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
+ }
if (nearTx != null)
finish(nodeId, nearTx, req);
@@ -1422,9 +1430,6 @@ public class IgniteTxHandler {
// Complete remote candidates.
tx.doneRemote(req.baseVersion(), null, null, null);
- tx.setPartitionUpdateCounters(
- req.partUpdateCounters() != null ? req.partUpdateCounters().array() : null);
-
tx.commitRemoteTx();
}
else {
@@ -1698,7 +1703,7 @@ public class IgniteTxHandler {
if (log.isDebugEnabled())
log.debug("Attempt to start a completed transaction (will ignore): " + tx);
- applyPartitionsUpdatesCounters(req.updateCounters());
+ applyPartitionsUpdatesCounters(req.updateCounters(), true, false);
return null;
}
@@ -1710,7 +1715,7 @@ public class IgniteTxHandler {
ctx.tm().uncommitTx(tx);
- applyPartitionsUpdatesCounters(req.updateCounters());
+ applyPartitionsUpdatesCounters(req.updateCounters(), true, false);
return null;
}
@@ -1720,8 +1725,13 @@ public class IgniteTxHandler {
tx.transactionNodes(req.transactionNodes());
}
- if (req.updateCounters() != null)
- tx.txCounters(true).updateCounters(req.updateCounters());
+ TxCounters txCounters = null;
+
+ if (req.updateCounters() != null) {
+ txCounters = tx.txCounters(true);
+
+ txCounters.updateCounters(req.updateCounters());
+ }
if (!tx.isSystemInvalidate()) {
int idx = 0;
@@ -1739,6 +1749,13 @@ public class IgniteTxHandler {
try {
tx.addWrite(entry, ctx.deploy().globalLoader());
+ if (txCounters != null) {
+ Long cntr = txCounters.generateNextCounter(entry.cacheId(), entry.cached().partition());
+
+ if (cntr != null) // Counter is null if entry is no-op.
+ entry.updateCounter(cntr);
+ }
+
if (isNearEnabled(cacheCtx) && req.invalidateNearEntry(idx))
invalidateNearEntry(cacheCtx, entry.key(), req.version());
@@ -2198,7 +2215,12 @@ public class IgniteTxHandler {
* @param req Request.
*/
private void processPartitionCountersRequest(UUID nodeId, PartitionCountersNeighborcastRequest req) {
- applyPartitionsUpdatesCounters(req.updateCounters());
+ try {
+ applyPartitionsUpdatesCounters(req.updateCounters(), true, false);
+ }
+ catch (IgniteCheckedException e) {
+ throw new IgniteException(e);
+ }
try {
ctx.io().send(nodeId, new PartitionCountersNeighborcastResponse(req.futId()), SYSTEM_POOL);
@@ -2230,21 +2252,32 @@ public class IgniteTxHandler {
}
/**
- * Applies partition counter updates for mvcc transactions.
+ * @param counters Counters.
+ */
+ public void applyPartitionsUpdatesCounters(Iterable<PartitionUpdateCountersMessage> counters)
+ throws IgniteCheckedException {
+ applyPartitionsUpdatesCounters(counters, false, false);
+ }
+
+ /**
+ * Applies partition counter updates for transactions.
*
* @param counters Counter values to be updated.
+ * @param rollback {@code True} if applied from rollbacks.
*/
- public void applyPartitionsUpdatesCounters(Iterable<PartitionUpdateCountersMessage> counters) {
+ public void applyPartitionsUpdatesCounters(Iterable<PartitionUpdateCountersMessage> counters,
+ boolean rollback,
+ boolean rollbackOnPrimary) throws IgniteCheckedException {
if (counters == null)
return;
for (PartitionUpdateCountersMessage counter : counters) {
GridCacheContext ctx0 = ctx.cacheContext(counter.cacheId());
- assert ctx0.mvccEnabled();
-
GridDhtPartitionTopology top = ctx0.topology();
+ AffinityTopologyVersion topVer = top.readyTopologyVersion();
+
assert top != null;
for (int i = 0; i < counter.size(); i++) {
@@ -2255,8 +2288,29 @@ public class IgniteTxHandler {
if (part != null && part.reserve()) {
try {
- if (part.state() != GridDhtPartitionState.RENTING)
- part.updateCounter(counter.initialCounter(i), counter.updatesCount(i));
+ if (part.state() != GridDhtPartitionState.RENTING) { // Check is actual only for backup node.
+ long start = counter.initialCounter(i);
+ long delta = counter.updatesCount(i);
+
+ boolean updated = part.updateCounter(start, delta);
+
+ // Need to log rolled back range for logical recovery.
+ if (updated && rollback) {
+ if (part.group().persistenceEnabled() &&
+ part.group().walEnabled() &&
+ !part.group().mvccEnabled()) {
+ RollbackRecord rec = new RollbackRecord(part.group().groupId(), part.id(),
+ start, delta);
+
+ ctx.wal().log(rec);
+ }
+
+ for (int cntr = 1; cntr <= delta; cntr++) {
+ ctx0.continuousQueries().skipUpdateCounter(null, part.id(), start + cntr,
+ topVer, rollbackOnPrimary);
+ }
+ }
+ }
else
invalid = true;
}
@@ -2272,7 +2326,7 @@ public class IgniteTxHandler {
}
if (invalid && log.isDebugEnabled())
- log.debug("Received partition update counters message for invalid partition: " +
+ log.debug("Received partition update counters message for invalid partition, ignoring: " +
"[cacheId=" + counter.cacheId() + ", part=" + counter.partition(i) + "]");
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxLocalAdapter.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxLocalAdapter.java
index 26350b6..55376f9 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxLocalAdapter.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxLocalAdapter.java
@@ -57,7 +57,6 @@ import org.apache.ignite.internal.processors.cache.IgniteCacheExpiryPolicy;
import org.apache.ignite.internal.processors.cache.KeyCacheObject;
import org.apache.ignite.internal.processors.cache.distributed.dht.PartitionUpdateCountersMessage;
import org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition;
-import org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionState;
import org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionTopology;
import org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal;
import org.apache.ignite.internal.processors.cache.store.CacheStoreManager;
@@ -95,6 +94,8 @@ import static org.apache.ignite.internal.processors.cache.GridCacheOperation.REA
import static org.apache.ignite.internal.processors.cache.GridCacheOperation.RELOAD;
import static org.apache.ignite.internal.processors.cache.GridCacheOperation.TRANSFORM;
import static org.apache.ignite.internal.processors.cache.GridCacheOperation.UPDATE;
+import static org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionState.LOST;
+import static org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionState.OWNING;
import static org.apache.ignite.internal.processors.dr.GridDrType.DR_NONE;
import static org.apache.ignite.internal.processors.dr.GridDrType.DR_PRIMARY;
import static org.apache.ignite.transactions.TransactionState.COMMITTED;
@@ -431,7 +432,8 @@ public abstract class IgniteTxLocalAdapter extends IgniteTxAdapter implements Ig
try {
cctx.tm().prepareTx(this, entries);
- calculatePartitionUpdateCounters();
+ if (txState().mvccEnabled())
+ calculatePartitionUpdateCounters();
}
catch (IgniteCheckedException e) {
throw e;
@@ -451,7 +453,7 @@ public abstract class IgniteTxLocalAdapter extends IgniteTxAdapter implements Ig
* pair (init, delta) values, where init - initial update counter, and delta - updates count made
* by current transaction for a given partition.
*/
- private void calculatePartitionUpdateCounters() {
+ public void calculatePartitionUpdateCounters() {
TxCounters counters = txCounters(false);
if (counters != null && F.isEmpty(counters.updateCounters())) {
@@ -468,9 +470,8 @@ public abstract class IgniteTxLocalAdapter extends IgniteTxAdapter implements Ig
continue;
PartitionUpdateCountersMessage msg = new PartitionUpdateCountersMessage(cacheId, partToCntrs.size());
- GridCacheContext ctx0 = cctx.cacheContext(cacheId);
- assert ctx0 != null && ctx0.mvccEnabled();
+ GridCacheContext ctx0 = cctx.cacheContext(cacheId);
GridDhtPartitionTopology top = ctx0.topology();
@@ -490,7 +491,11 @@ public abstract class IgniteTxLocalAdapter extends IgniteTxAdapter implements Ig
GridDhtLocalPartition part = top.localPartition(p);
- assert part != null && part.state() == GridDhtPartitionState.OWNING;
+ // LOST state is possible if tx is started over LOST partition.
+ assert part != null && (part.state() == OWNING || part.state() == LOST):
+ part == null ?
+ "map=" + top.partitionMap(false) + ", lost=" + top.lostPartitions() :
+ "part=" + part.toString();
msg.add(p, part.getAndIncrementUpdateCounter(cntr), cntr);
}
@@ -577,6 +582,8 @@ public abstract class IgniteTxLocalAdapter extends IgniteTxAdapter implements Ig
AffinityTopologyVersion topVer = topologyVersion();
+ TxCounters txCounters = txCounters(false);
+
/*
* Commit to cache. Note that for 'near' transaction we loop through all the entries.
*/
@@ -893,7 +900,15 @@ public abstract class IgniteTxLocalAdapter extends IgniteTxAdapter implements Ig
txEntry.cached(entryEx(cacheCtx, txEntry.txKey(), topologyVersion()));
}
}
+ }
+
+ if (!txState.mvccEnabled() && txCounters != null) {
+ cctx.tm().txHandler().applyPartitionsUpdatesCounters(txCounters.updateCounters());
+ for (IgniteTxEntry entry : commitEntries) {
+ if (entry.cqNotifyClosure() != null)
+ entry.cqNotifyClosure().applyx();
+ }
}
// Apply cache sizes only for primary nodes. Update counters were applied on prepare state.
@@ -1072,6 +1087,13 @@ public abstract class IgniteTxLocalAdapter extends IgniteTxAdapter implements Ig
}
if (DONE_FLAG_UPD.compareAndSet(this, 0, 1)) {
+ if (!txState.mvccEnabled()) {
+ TxCounters txCounters = txCounters(false);
+
+ if (txCounters != null)
+ cctx.tm().txHandler().applyPartitionsUpdatesCounters(txCounters.updateCounters(), true, true);
+ }
+
cctx.tm().rollbackTx(this, clearThreadMap, skipCompletedVersions());
cctx.mvccCaching().onTxFinished(this, false);
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxManager.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxManager.java
index c88c863..6f2d594 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxManager.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/IgniteTxManager.java
@@ -2128,8 +2128,8 @@ public class IgniteTxManager extends GridCacheSharedManagerAdapter {
if (commit)
tx.commitAsync().listen(new CommitListener(tx));
- else if (tx.mvccSnapshot() != null && !tx.local())
- // remote (backup) mvcc transaction sends partition counters to other backup transaction
+ else if (!tx.local())
+ // remote (backup) transaction sends partition counters to other backup transaction on recovery rollback
// in order to keep counters consistent
neighborcastPartitionCountersAndRollback(tx);
else
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/TxCounters.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/TxCounters.java
index 515aeef..15c92f9 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/TxCounters.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/transactions/TxCounters.java
@@ -24,6 +24,7 @@ import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import org.apache.ignite.internal.processors.cache.distributed.dht.PartitionUpdateCountersMessage;
+import org.apache.ignite.internal.util.typedef.internal.U;
import org.jetbrains.annotations.Nullable;
/**
@@ -37,7 +38,7 @@ public class TxCounters {
private final Map<Integer, Map<Integer, AtomicLong>> updCntrsAcc = new HashMap<>();
/** Final update counters for cache partitions in the end of transaction */
- private volatile Collection<PartitionUpdateCountersMessage> updCntrs;
+ private volatile Map<Integer, PartitionUpdateCountersMessage> updCntrs;
/** Counter tracking number of entries locked by tx. */
private final AtomicInteger lockCntr = new AtomicInteger();
@@ -68,14 +69,17 @@ public class TxCounters {
* @param updCntrs Final update counters.
*/
public void updateCounters(Collection<PartitionUpdateCountersMessage> updCntrs) {
- this.updCntrs = updCntrs;
+ this.updCntrs = U.newHashMap(updCntrs.size());
+
+ for (PartitionUpdateCountersMessage cntr : updCntrs)
+ this.updCntrs.put(cntr.cacheId(), cntr);
}
/**
* @return Final update counters.
*/
@Nullable public Collection<PartitionUpdateCountersMessage> updateCounters() {
- return updCntrs;
+ return updCntrs == null ? null : updCntrs.values();
}
/**
@@ -145,4 +149,19 @@ public class TxCounters {
public int lockCounter() {
return lockCntr.get();
}
+
+ /**
+ * @param cacheId Cache id.
+ * @param partId Partition id.
+ *
+ * @return Counter or {@code null} if cache partition has not updates.
+ */
+ public Long generateNextCounter(int cacheId, int partId) {
+ PartitionUpdateCountersMessage msg = updCntrs.get(cacheId);
+
+ if (msg == null)
+ return null;
+
+ return msg.nextCounter(partId);
+ }
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java b/modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java
index f05e0e5..17d4be0 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java
@@ -44,6 +44,7 @@ import org.apache.ignite.internal.GridKernalContext;
import org.apache.ignite.internal.IgniteEx;
import org.apache.ignite.internal.IgniteFeatures;
import org.apache.ignite.internal.IgniteInternalFuture;
+import org.apache.ignite.internal.NodeStoppingException;
import org.apache.ignite.internal.cluster.ClusterGroupAdapter;
import org.apache.ignite.internal.cluster.ClusterTopologyCheckedException;
import org.apache.ignite.internal.cluster.DistributedBaselineConfiguration;
@@ -385,7 +386,7 @@ public class GridClusterStateProcessor extends GridProcessorAdapter implements I
GridChangeGlobalStateFuture fut = this.stateChangeFut.get();
if (fut != null)
- fut.onDone(new IgniteCheckedException("Failed to wait for cluster state change, node is stopping."));
+ fut.onDone(new NodeStoppingException("Failed to wait for cluster state change, node is stopping."));
super.onKernalStop(cancel);
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/stat/IoStatisticsHolder.java b/modules/core/src/main/java/org/apache/ignite/internal/stat/IoStatisticsHolder.java
index 0c36820..5b7a57f 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/stat/IoStatisticsHolder.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/stat/IoStatisticsHolder.java
@@ -24,7 +24,6 @@ import java.util.Map;
* Holder of IO statistics.
*/
public interface IoStatisticsHolder {
-
/**
* Track logical read of given page.
*
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteTree.java b/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteTree.java
index 9e854d2..140349a 100644
--- a/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteTree.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteTree.java
@@ -141,6 +141,9 @@ public interface IgniteTree<L, T> {
REMOVE,
/** */
- PUT
+ PUT,
+
+ /** */
+ IN_PLACE
}
}
diff --git a/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java b/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java
index 395705c..5d9d8da 100755
--- a/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java
+++ b/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java
@@ -9519,6 +9519,60 @@ public abstract class IgniteUtils {
}
/**
+ * Create a map with single key-value pair.
+ *
+ * @param k Key.
+ * @param v Value.
+ * @return Map.
+ */
+ public static <K, V> Map<K, V> map(K k, V v) {
+ GridLeanMap<K, V> map = new GridLeanMap<>(1);
+
+ map.put(k, v);
+
+ return map;
+ }
+
+ /**
+ * Create a map with two key-value pairs.
+ *
+ * @param k1 Key 1.
+ * @param v1 Value 1.
+ * @param k2 Key 2.
+ * @param v2 Value 2.
+ * @return Map.
+ */
+ public static <K, V> Map<K, V> map(K k1, V v1, K k2, V v2) {
+ GridLeanMap<K, V> map = new GridLeanMap<>(2);
+
+ map.put(k1, v1);
+ map.put(k2, v2);
+
+ return map;
+ }
+
+ /**
+ * Create a map with three key-value pairs.
+ *
+ * @param k1 Key 1.
+ * @param v1 Value 1.
+ * @param k2 Key 2.
+ * @param v2 Value 2.
+ * @param k3 Key 3.
+ * @param v3 Value 3.
+ * @return Map.
+ */
+ public static <K, V> Map<K, V> map(K k1, V v1, K k2, V v2, K k3, V v3) {
+ GridLeanMap<K, V> map = new GridLeanMap<>(3);
+
+ map.put(k1, v1);
+ map.put(k2, v2);
+ map.put(k3, v3);
+
+ return map;
+ }
+
+ /**
* @param col non-null collection with one element
* @return a SingletonList containing the element in the original collection
*/
@@ -9699,7 +9753,7 @@ public abstract class IgniteUtils {
public static <T extends R, R> List<R> arrayList(Collection<T> c, @Nullable IgnitePredicate<? super T>... p) {
assert c != null;
- return arrayList(c, c.size(), p);
+ return arrayList(c.iterator(), c.size(), p);
}
/**
@@ -9718,14 +9772,16 @@ public abstract class IgniteUtils {
* @param p Optional filters.
* @return Resulting array list.
*/
- public static <T extends R, R> List<R> arrayList(Iterable<T> c, int cap,
+ public static <T extends R, R> List<R> arrayList(Iterator<T> c, int cap,
@Nullable IgnitePredicate<? super T>... p) {
assert c != null;
assert cap >= 0;
List<R> list = new ArrayList<>(cap);
- for (T t : c) {
+ while (c.hasNext()) {
+ T t = c.next();
+
if (F.isAll(t, p))
list.add(t);
}
diff --git a/modules/core/src/test/config/log4j-test.xml b/modules/core/src/test/config/log4j-test.xml
index bb5f94a..7206b4e 100755
--- a/modules/core/src/test/config/log4j-test.xml
+++ b/modules/core/src/test/config/log4j-test.xml
@@ -97,6 +97,15 @@
</category>
-->
+ <!--
+ Uncomment to enable debugging of partition counters.
+ -->
+ <!--
+ <category name="org.apache.ignite.internal.processors.cache.PartitionTxUpdateCounterDebugWrapper">
+ <level value="DEBUG"/>
+ </category>
+ -->
+
<!-- Disable all open source debugging. -->
<category name="org">
<level value="INFO"/>
diff --git a/modules/core/src/test/java/org/apache/ignite/cache/ResetLostPartitionTest.java b/modules/core/src/test/java/org/apache/ignite/cache/ResetLostPartitionTest.java
index 8d318dc..3e1568a 100644
--- a/modules/core/src/test/java/org/apache/ignite/cache/ResetLostPartitionTest.java
+++ b/modules/core/src/test/java/org/apache/ignite/cache/ResetLostPartitionTest.java
@@ -36,6 +36,7 @@ import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
import org.junit.Test;
+import static org.apache.ignite.configuration.WALMode.LOG_ONLY;
import static org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionState.LOST;
import static org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtPartitionState.OWNING;
import static org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.DFLT_STORE_DIR;
@@ -75,6 +76,8 @@ public class ResetLostPartitionTest extends GridCommonAbstractTest {
DataStorageConfiguration storageCfg = new DataStorageConfiguration();
+ storageCfg.setPageSize(1024).setWalMode(LOG_ONLY).setWalSegmentSize(8 * 1024 * 1024);
+
storageCfg.getDefaultDataRegionConfiguration()
.setPersistenceEnabled(true)
.setMaxSize(500L * 1024 * 1024);
@@ -104,7 +107,7 @@ public class ResetLostPartitionTest extends GridCommonAbstractTest {
.setBackups(1)
.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC)
.setPartitionLossPolicy(PartitionLossPolicy.READ_ONLY_SAFE)
- .setAffinity(new RendezvousAffinityFunction(false, 1024))
+ .setAffinity(new RendezvousAffinityFunction(false, 64))
.setIndexedTypes(String.class, String.class);
}
diff --git a/modules/core/src/test/java/org/apache/ignite/internal/TestRecordingCommunicationSpi.java b/modules/core/src/test/java/org/apache/ignite/internal/TestRecordingCommunicationSpi.java
index 988395f..5969d25 100644
--- a/modules/core/src/test/java/org/apache/ignite/internal/TestRecordingCommunicationSpi.java
+++ b/modules/core/src/test/java/org/apache/ignite/internal/TestRecordingCommunicationSpi.java
@@ -18,10 +18,10 @@
package org.apache.ignite.internal;
import java.util.ArrayList;
-import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -30,7 +30,7 @@ import org.apache.ignite.IgniteException;
import org.apache.ignite.cluster.ClusterNode;
import org.apache.ignite.internal.cluster.ClusterTopologyCheckedException;
import org.apache.ignite.internal.managers.communication.GridIoMessage;
-import org.apache.ignite.internal.util.typedef.F;
+import org.apache.ignite.internal.util.typedef.G;
import org.apache.ignite.internal.util.typedef.T2;
import org.apache.ignite.internal.util.typedef.internal.U;
import org.apache.ignite.lang.IgniteBiInClosure;
@@ -300,7 +300,7 @@ public class TestRecordingCommunicationSpi extends TcpCommunicationSpi {
* Stops block messages and sends all already blocked messages.
*/
public void stopBlock() {
- stopBlock(true, null);
+ stopBlock(true, null, true, true);
}
/**
@@ -309,7 +309,7 @@ public class TestRecordingCommunicationSpi extends TcpCommunicationSpi {
* @param sndMsgs {@code True} to send blocked messages.
*/
public void stopBlock(boolean sndMsgs) {
- stopBlock(sndMsgs, null);
+ stopBlock(sndMsgs, null, true, true);
}
/**
@@ -320,15 +320,37 @@ public class TestRecordingCommunicationSpi extends TcpCommunicationSpi {
* @param unblockPred If not null unblocks only messages allowed by predicate.
*/
public void stopBlock(boolean sndMsgs, @Nullable IgnitePredicate<T2<ClusterNode, GridIoMessage>> unblockPred) {
+ stopBlock(sndMsgs, unblockPred, true, true);
+ }
+
+ /**
+ * Stops block messages and sends all already blocked messages if sndMsgs is 'true' optionally filtered by
+ * unblockPred.
+ *
+ * @param sndMsgs If {@code true} sends blocked messages.
+ * @param unblockPred If not null unblocks only messages allowed by predicate.
+ * @param clearFilters {@code true} to clear filters.
+ * @param rmvBlockedMsgs {@code true} to remove blocked messages. Sometimes useful in conjunction with {@code
+ * sndMsgs=false}.
+ */
+ public void stopBlock(boolean sndMsgs, @Nullable IgnitePredicate<T2<ClusterNode, GridIoMessage>> unblockPred,
+ boolean clearFilters, boolean rmvBlockedMsgs) {
synchronized (this) {
- blockCls.clear();
- blockP = null;
+ if (clearFilters) {
+ blockCls.clear();
+ blockP = null;
+ }
- Collection<T2<ClusterNode, GridIoMessage>> msgs =
- unblockPred == null ? blockedMsgs : F.view(blockedMsgs, unblockPred);
+ Iterator<T2<ClusterNode, GridIoMessage>> iter = blockedMsgs.iterator();
- if (sndMsgs) {
- for (T2<ClusterNode, GridIoMessage> msg : msgs) {
+ while (iter.hasNext()) {
+ T2<ClusterNode, GridIoMessage> msg = iter.next();
+
+ // It is important what predicate if called only once for each message.
+ if (unblockPred != null && !unblockPred.apply(msg))
+ continue;
+
+ if (sndMsgs) {
try {
ignite.log().info("Send blocked message [node=" + msg.get1().id() +
", order=" + msg.get1().order() +
@@ -340,9 +362,18 @@ public class TestRecordingCommunicationSpi extends TcpCommunicationSpi {
U.error(ignite.log(), "Failed to send blocked message: " + msg, e);
}
}
- }
- msgs.clear();
+ if (rmvBlockedMsgs)
+ iter.remove();
+ }
}
}
+
+ /**
+ * Stop blocking all messages.
+ */
+ public static void stopBlockAll() {
+ for (Ignite ignite : G.allGrids())
+ spi(ignite).stopBlock(true);
+ }
}
diff --git a/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCacheGroupsTest.java b/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCacheGroupsTest.java
index 222b161..2f02e10 100644
--- a/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCacheGroupsTest.java
+++ b/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/IgniteCacheGroupsTest.java
@@ -115,7 +115,8 @@ import static org.apache.ignite.transactions.TransactionIsolation.REPEATABLE_REA
import static org.apache.ignite.transactions.TransactionIsolation.SERIALIZABLE;
/**
- *
+ * TODO FIXME Mixed atomic-tx cache groups currently are broken if tx cache is dynamically started after atomic.
+ * TODO FIXME See https://issues.apache.org/jira/browse/IGNITE-11797
*/
@SuppressWarnings({"unchecked", "ThrowableNotThrown"})
public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
@@ -2291,7 +2292,7 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
for (final int i : sequence(loaders)) {
final IgniteDataStreamer ldr = clientNode.dataStreamer(cache.getName());
-
+ ldr.allowOverwrite(true); // TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11793
ldr.autoFlushFrequency(0);
cls.add(new Callable<Void>() {
@@ -2985,6 +2986,7 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
}
/**
+ * TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11797
* @throws Exception If failed.
*/
@Test
@@ -3090,6 +3092,7 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
}
/**
+ * TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11797
* @throws Exception If failed.
*/
@Test
@@ -3196,7 +3199,7 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
IgniteCache cache = node.createCache(cacheConfiguration(grp, "tmpCache-" + cntr++,
PARTITIONED,
- rnd.nextBoolean() ? ATOMIC : TRANSACTIONAL,
+ grp.equals(GROUP1) ? ATOMIC : TRANSACTIONAL,
backups,
rnd.nextBoolean()));
@@ -3816,6 +3819,7 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
}
/**
+ * TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11797
* @throws Exception If failed.
*/
@Test
@@ -3824,6 +3828,7 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
}
/**
+ * TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11797
* @throws Exception If failed.
*/
@Test
@@ -3846,8 +3851,8 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
client.createCache(cacheConfiguration(GROUP1, "c2", PARTITIONED, TRANSACTIONAL, 1, false));
client.createCache(cacheConfiguration(GROUP1, "c3", PARTITIONED, ATOMIC, 1, false));
- client.createCache(cacheConfiguration(GROUP2, "c4", PARTITIONED, TRANSACTIONAL, 1, false));
- client.createCache(cacheConfiguration(GROUP2, "c5", PARTITIONED, ATOMIC, 1, false));
+ client.createCache(cacheConfiguration(GROUP2, "c4", PARTITIONED, ATOMIC, 1, false));
+ client.createCache(cacheConfiguration(GROUP2, "c5", PARTITIONED, TRANSACTIONAL, 1, false));
client.createCache(cacheConfiguration(GROUP2, "c6", PARTITIONED, TRANSACTIONAL, 1, false));
client.createCache(cacheConfiguration(null, "c7", PARTITIONED, ATOMIC, 1, false));
@@ -3946,6 +3951,7 @@ public class IgniteCacheGroupsTest extends GridCommonAbstractTest {
}
/**
+ * TODO FIXME https://issues.apache.org/jira/browse/IGNITE-11797
* @throws Exception If failed.
*/
@Test
diff --git a/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/distributed/CachePartitionLossDetectionOnNodeLeftTest.java b/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/distributed/CachePartitionLossDetectionOnNodeLeftTest.java
new file mode 100644
index 0000000..61b024c
--- /dev/null
+++ b/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/distributed/CachePartitionLossDetectionOnNodeLeftTest.java
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
... 6304 lines suppressed ...