[jira] [Commented] (OAK-5970) (Re-)Indexing: estimate progress / ETA

["Chetan Mehrotra (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-5970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022709#comment-16022709 ] 

Chetan Mehrotra commented on OAK-5970:
--------------------------------------

Done initial implementation with 1796042 which emits the rate of traversal like \[3648.62 nodes/s/ 13135037.84 nodes/hr\]

{noformat}
14:00:20.463 INFO  [main]     /oak:index/lucene => Indexed 110000 nodes in 1.330 s ...
14:00:21.349 INFO  [main]     Reindexing Traversed #230000 /libs/dam/gui/components/admin/collections/createcollection/selfcollectionmember/selfcollectionmember.jsp/jcr:content [3593.72 nodes/s/ 12937387.50 nodes/hr]
14:00:21.961 INFO  [main]     Reindexing Traversed #240000 /libs/dam/gui/content/assets/jcr:content/actions/selection/share/items/macshare/granite:rendercondition [3636.33 nodes/s/ 13090800.00 nodes/hr]
14:00:21.961 INFO  [main]     /oak:index/lucene => Indexed 120000 nodes in 1.498 s ...
14:00:23.215 INFO  [main]     Reindexing Traversed #250000 /libs/wcm/core/components/timewarp/timeline/ajax/images/bubble-left.png [3676.44 nodes/s/ 13235188.24 nodes/hr]
14:00:24.723 INFO  [main]     Reindexing Traversed #260000 /libs/granite/operations/content/queryperformance/query-performance-main-panel/items/explainResultSpinnerModal/footer/close [3611.08 nodes/s/ 12999900.00 nodes/hr]
14:00:24.724 INFO  [main]     /oak:index/lucene => Indexed 130000 nodes in 2.763 s ...
14:00:25.747 INFO  [main]     Reindexing Traversed #270000 /libs/granite/distribution/content/distribution-agent/content/items [3648.62 nodes/s/ 13135037.84 nodes/hr]
{noformat}

It uses a TraversalRateEstimator which estimates the current rate. This has 2 implementations

* SimpleRateEstimator - Does a simple mean rate calculation based on time spent and number of nodes traversed so far
* MetricRateEstimator - Uses metrics api and makes use of 5 min moving average providing a better result. This is used if MetricsStatisticsProvider is in use

> (Re-)Indexing: estimate progress / ETA
> --------------------------------------
>
>                 Key: OAK-5970
>                 URL: https://issues.apache.org/jira/browse/OAK-5970
>             Project: Jackrabbit Oak
>          Issue Type: New Feature
>          Components: indexing
>            Reporter: Thomas Mueller
>            Assignee: Chetan Mehrotra
>             Fix For: 1.8
>
>
> Reindexing can take a long time, so it would be good if we can estimate where we are at (for example in percent of the relevant number of nodes). It might also be possible to estimate when indexing will be done, and the current path.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)