You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@geode.apache.org by "Nabarun Nag (Jira)" <ji...@apache.org> on 2021/09/03 02:23:09 UTC
[jira] [Closed] (GEODE-8827) The DiskRegionStats bytesOnlyOnDisk
stat is not incremented during persistent region recovery
[ https://issues.apache.org/jira/browse/GEODE-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nabarun Nag closed GEODE-8827.
------------------------------
> The DiskRegionStats bytesOnlyOnDisk stat is not incremented during persistent region recovery
> ---------------------------------------------------------------------------------------------
>
> Key: GEODE-8827
> URL: https://issues.apache.org/jira/browse/GEODE-8827
> Project: Geode
> Issue Type: Bug
> Components: persistence, statistics
> Reporter: Barrett Oglesby
> Assignee: Barrett Oglesby
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
> Attachments: DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_gets.gif, DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_gets_with_change.gif, DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_restart.gif, DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_restart_with_change.gif, DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_no_eviction.gif
>
>
> With a test like:
> - 2 servers with partitioned region configured like:
> ** persistence enabled
> ** heap eviction with overflow enabled
> - load enough entries to cause overflow
> - shut down the servers
> - restart the servers
> - execute a function to get all entries in each server
> After the step to restart the servers, the bytesOnlyOnDisk stat is 0.
> After the step to get all entries, the bytesOnlyOnDisk stat is negative.
> The entriesInVM and entriesOnlyOnDisk stats are incremented as BucketRegions are recovered from disk in LocalRegion.initializeStats here:
> {noformat}
> java.lang.Exception: Stack trace
> at java.lang.Thread.dumpStack(Thread.java:1333)
> at org.apache.geode.internal.cache.LocalRegion.initializeStats(LocalRegion.java:10222)
> at org.apache.geode.internal.cache.BucketRegion.initializeStats(BucketRegion.java:2163)
> at org.apache.geode.internal.cache.AbstractDiskRegion.copyExistingRegionMap(AbstractDiskRegion.java:775)
> at org.apache.geode.internal.cache.DiskStoreImpl.initializeOwner(DiskStoreImpl.java:631)
> at org.apache.geode.internal.cache.DiskRegion.initializeOwner(DiskRegion.java:239)
> at org.apache.geode.internal.cache.DistributedRegion.initialize(DistributedRegion.java:1081)
> at org.apache.geode.internal.cache.BucketRegion.initialize(BucketRegion.java:262)
> at org.apache.geode.internal.cache.LocalRegion.createSubregion(LocalRegion.java:981)
> at org.apache.geode.internal.cache.PartitionedRegionDataStore.createBucketRegion(PartitionedRegionDataStore.java:785)
> at org.apache.geode.internal.cache.PartitionedRegionDataStore.grabFreeBucket(PartitionedRegionDataStore.java:460)
> at org.apache.geode.internal.cache.PartitionedRegionDataStore.grabFreeBucketRecursively(PartitionedRegionDataStore.java:319)
> at org.apache.geode.internal.cache.PartitionedRegionDataStore.grabBucket(PartitionedRegionDataStore.java:2896)
> at org.apache.geode.internal.cache.ProxyBucketRegion.recoverFromDisk(ProxyBucketRegion.java:441)
> at org.apache.geode.internal.cache.ProxyBucketRegion.recoverFromDiskRecursively(ProxyBucketRegion.java:407)
> at org.apache.geode.internal.cache.PRHARedundancyProvider$2.run2(PRHARedundancyProvider.java:1640)
> at org.apache.geode.internal.cache.partitioned.RecoveryRunnable.run(RecoveryRunnable.java:60)
> at org.apache.geode.internal.cache.PRHARedundancyProvider$2.run(PRHARedundancyProvider.java:1630)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The current LocalRegion.initializeStats method implementation is:
> {noformat}
> public void initializeStats(long numEntriesInVM, long numOverflowOnDisk,
> long numOverflowBytesOnDisk) {
> getDiskRegion().getStats().incNumEntriesInVM(numEntriesInVM);
> getDiskRegion().getStats().incNumOverflowOnDisk(numOverflowOnDisk);
> }
> {noformat}
> Even though numOverflowBytesOnDisk is passed into this method, it is ignored as this logging shows:
> {noformat}
> [warn 2021/01/12 11:19:11.785 PST <Recovery thread for bucket _B__data_3> tid=0x49] XXX LocalRegion.initializeStats numOverflowBytesOnDisk=4546560; bytesOnlyOnDiskFromStats=0
> [warn 2021/01/12 11:19:11.791 PST <Recovery thread for bucket _B__data_9> tid=0x4f] XXX LocalRegion.initializeStats numOverflowBytesOnDisk=4536320; bytesOnlyOnDiskFromStats=0
> [warn 2021/01/12 11:19:11.797 PST <Recovery thread for bucket _B__data_6> tid=0x4c] XXX LocalRegion.initializeStats numOverflowBytesOnDisk=4526080; bytesOnlyOnDiskFromStats=0
> [warn 2021/01/12 11:19:11.800 PST <Recovery thread for bucket _B__data_2> tid=0x48] XXX LocalRegion.initializeStats numOverflowBytesOnDisk=4546560; bytesOnlyOnDiskFromStats=0
> [warn 2021/01/12 11:19:11.801 PST <Recovery thread for bucket _B__data_8> tid=0x4e] XXX LocalRegion.initializeStats numOverflowBytesOnDisk=4526080; bytesOnlyOnDiskFromStats=0
> {noformat}
> The attached DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_restart.gif chart shows the behavior after server restart.
> The attached DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_gets.gif chart shows the behavior after the gets.
> If I make this change, I see correct behavior:
> {noformat}
> public void initializeStats(long numEntriesInVM, long numOverflowOnDisk,
> long numOverflowBytesOnDisk) {
> getDiskRegion().getStats().incNumEntriesInVM(numEntriesInVM);
> getDiskRegion().getStats().incNumOverflowOnDisk(numOverflowOnDisk);
> -> getDiskRegion().getStats().incNumOverflowBytesOnDisk(numOverflowBytesOnDisk);
> }
> {noformat}
> The attached DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_restart_with_change.gif chart shows the behavior with this change after server restart.
> The attached DiskRegionStats_entriesOnlyOnDisk_bytesOnlyOnDisk_after_gets_with_change.gif chart shows the behavior with this change after the gets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)