You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Vikas Saurabh (JIRA)" <ji...@apache.org> on 2015/07/22 13:05:04 UTC
[jira] [Commented] (OAK-3070) Use a lower bound in VersionGC query
to avoid checking unmodified once deleted docs
[ https://issues.apache.org/jira/browse/OAK-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14636722#comment-14636722 ]
Vikas Saurabh commented on OAK-3070:
------------------------------------
BTW, even without this optimization, one of test setups reports following stats:
{noformat}
# grep " Version garbage collected in" error.log*
error.log:22.07.2015 02:51:10.195 *INFO* [pool-5-thread-16] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 51.16 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=621383, splitDocGCCount=2448, intermediateSplitDocGCCount=265, timeToCollectDeletedDocs=3.496 min, timeTakenToDeleteDocs=47.54 min}
error.log.2015-07-15:15.07.2015 02:34:46.895 *INFO* [pool-5-thread-9] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 34.77 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=557486, splitDocGCCount=2364, intermediateSplitDocGCCount=257, timeToCollectDeletedDocs=1.735 min, timeTakenToDeleteDocs=32.74 min}
error.log.2015-07-16:16.07.2015 02:31:47.559 *INFO* [pool-5-thread-11] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 31.79 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=559512, splitDocGCCount=2302, intermediateSplitDocGCCount=249, timeToCollectDeletedDocs=1.124 min, timeTakenToDeleteDocs=30.27 min}
error.log.2015-07-17:17.07.2015 02:35:16.096 *INFO* [pool-5-thread-10] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 35.26 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=560226, splitDocGCCount=2351, intermediateSplitDocGCCount=257, timeToCollectDeletedDocs=1.537 min, timeTakenToDeleteDocs=33.54 min}
error.log.2015-07-18:18.07.2015 02:52:02.542 *INFO* [pool-5-thread-11] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 42.19 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=556254, splitDocGCCount=2304, intermediateSplitDocGCCount=245, timeToCollectDeletedDocs=2.341 min, timeTakenToDeleteDocs=39.02 min}
error.log.2015-07-19:19.07.2015 02:41:59.809 *INFO* [pool-5-thread-14] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 41.98 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=556036, splitDocGCCount=2285, intermediateSplitDocGCCount=252, timeToCollectDeletedDocs=2.637 min, timeTakenToDeleteDocs=39.05 min}
error.log.2015-07-20:20.07.2015 02:46:04.002 *INFO* [pool-5-thread-5] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 46.06 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=555721, splitDocGCCount=2320, intermediateSplitDocGCCount=249, timeToCollectDeletedDocs=2.836 min, timeTakenToDeleteDocs=42.57 min}
error.log.2015-07-21:21.07.2015 02:45:08.122 *INFO* [pool-5-thread-15] org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector Version garbage collected in 45.12 min. VersionGCStats{ignoredGCDueToCheckPoint=false, deletedDocGCCount=552140, splitDocGCCount=2267, intermediateSplitDocGCCount=245, timeToCollectDeletedDocs=3.238 min, timeTakenToDeleteDocs=41.72 min}
{noformat}
Number of docs getting collected everyday is around half million (there aren't stats do see how many candidates got fetched). Fetching candidate docs ranged from ~1min to ~3.5 minutes.
This setup is a 2 node cluster with Mongo backend. It doesn't have {{oak.mongo.disableVersionGCIndexHint}} set, so index hint to use deleted once is being sent.
(cc [~chetanm], [~mreutegg])
> Use a lower bound in VersionGC query to avoid checking unmodified once deleted docs
> -----------------------------------------------------------------------------------
>
> Key: OAK-3070
> URL: https://issues.apache.org/jira/browse/OAK-3070
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk, rdbmk
> Reporter: Chetan Mehrotra
> Fix For: 1.3.5
>
> Attachments: OAK-3070.patch
>
>
> As part of OAK-3062 [~mreutegg] suggested
> {quote}
> As a further optimization we could also limit the lower bound of the _modified
> range. The revision GC does not need to check documents with a _deletedOnce
> again if they were not modified after the last successful GC run. If they
> didn't change and were considered existing during the last run, then they
> must still exist in the current GC run. To make this work, we'd need to
> track the last successful revision GC run.
> {quote}
> Lowest last validated _modified can be possibly saved in settings collection and reused for next run
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)