You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by tsutsu <gi...@git.apache.org> on 2016/02/27 03:06:57 UTC
[GitHub] couchdb-couch-index pull request: Continue to reopen DB and enum c...
GitHub user tsutsu opened a pull request:
https://github.com/apache/couchdb-couch-index/pull/14
Continue to reopen DB and enum changes until DB reopens up-to-date
Currently, indexes make an initial read of the database state and the index state, and then work to bring the index up-to-date with that sequence-point of the database. Once the index is caught up with this sequence-point, the indexing process halts (excepting the case where another consumer has requested the index state while the index updater was running; in that case, the indexing process finishes, then notices that another reader is waiting and starts again, reading off another sequence-point and working toward that.)
With this PR, the index updater will now repeatedly reopen the DB and check to see whether it has truly caught up, getting a new sequence-point to work against each time, all within the context of a single index update operation. This means that a synchronous index update (as is used when querying the index state) will only return when the index has been made fully current, rather than just having caught up to the point the DB was at when the index-update began.
COUCHDB-2955
BugzID: 14036
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloudant/couchdb-couch-index 14036-loop-updater-until-caught-up
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/couchdb-couch-index/pull/14.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14
----
commit 5d534a4ea7ccea9aa77b6b25a851e99f65e06807
Author: Levi McAuley <le...@leviaul.com>
Date: 2016-02-27T01:35:46Z
Continue to reopen DB and enum changes until DB reopens up-to-date
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch-index pull request: Continue to reopen DB and enum c...
Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the pull request:
https://github.com/apache/couchdb-couch-index/pull/14#issuecomment-189603301
Hi! Thanks for contribution. This looks important, but how to test that it works and right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch-index pull request: Continue to reopen DB and enum c...
Posted by janl <gi...@git.apache.org>.
Github user janl commented on the pull request:
https://github.com/apache/couchdb-couch-index/pull/14#issuecomment-190815415
Doesn’t this also change fundamentally how views work? And doesn’t this introduce the same race condition we have for compaction, where heavy database writes can stall out the compaction swap?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] couchdb-couch-index pull request: Continue to reopen DB and enum c...
Posted by davisp <gi...@git.apache.org>.
Github user davisp commented on the pull request:
https://github.com/apache/couchdb-couch-index/pull/14#issuecomment-216579991
This turned out to be an awfully big change. In hindsight I don't think we're going to get much benefit out of reopening the database (I made that assumption almost four years ago so forgive me). Though what might be useful is a check in the handle_info clause when we get the updated index state. Something like this maybe: https://gist.github.com/davisp/153c239cfcd8a4dc53be2e43e84d280d
@janl I'm not sure what you mean by fundamentally changes how views work. This change just makes it so that we restart the indexer rather than wait for the message that a client is still waiting. The technical difference would be that if you had a single view read during db updates, it would update until all active updates were handled instead of handling an arbitrary number of them and waiting for another view read.
Views have always been able to stall the compaction swap. I guess this would lessen the read load to hit that condition though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---