[jira] Commented: (COUCHDB-532) Have requests for views that use 'stale=ok' return whats in the index immediately, but still kickoff background indexing process.

["Jan Lehnardt (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/COUCHDB-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766904#action_12766904 ] 

Jan Lehnardt commented on COUCHDB-532:
--------------------------------------

I think we have plans for a facility that fixes the underlying issue here without the proposed way of solving it. Adam and I are dreaming up a "task manager" sorta like cron, that lets you define arbitrary actions to be run on events, they could be time related, based on stats counters or just continuous. for example continuous replication that is supposed to survive a server restart is one of these things. or scheduling compaction to run each night at 2am. another part is keeping views up to date asynchronously, so clients can use stale=ok all the time. I'm pretty sure we a ticket for this, but I couldn't find it just now.

> Have requests for views that use 'stale=ok' return whats in the index immediately, but still kickoff background indexing process.
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: COUCHDB-532
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-532
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>         Environment: All
>            Reporter: Glenn Rempe
>
> Currently a common use case for querying indexes from external applications is to pass in the optional parameter 'stale=ok' which instructs the view to return what is has in its index immediately without waiting for the index to first catch up.
> This is a *very* important feature as it allows applications to eliminate the poor user experience of potential HTTP client timeouts or long'ish waits for a query to return while the index is updated.
> However, applications that make extensive (or in my case exlusive) use of this feature are now essentially *required* to build some kind of script which will do the indexing in the background since no user requests will *ever* initiate indexing.  This would be for example a cron script that makes an HTTP call to the view every N hours.  This becomes an operational issue as you now have to write, maintain and schedule an external script which calls each view in each of your DBs periodically.
> I propose to the team, after some discussion with Paul Joseph Davis, that view requests with the 'stale=ok' param continue to return results in the as-is index to users immediately, but *also* kick off the background indexing process.  This would have the effect of removing the need for any external maintenance scripts (aside from DB/View compaction which is a related but separate issue).
> In some use cases it might be true that indexing *must* occur outside of 'normal business hours' due to processing load.  If this is a requirement the behavior of view indexing on 'stale' requests could be configurable (e.g. as a boolean to always/never do this additional background indexing, or force BG indexing every N requests/minutes using 'stale').
> Cheers.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.