You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Henrik Hofmeister (Created) (JIRA)" <ji...@apache.org> on 2011/12/18 22:26:30 UTC
[jira] [Created] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
--------------------------------------------------------------------------------------------------------------------------
Key: COUCHDB-1367
URL: https://issues.apache.org/jira/browse/COUCHDB-1367
Project: CouchDB
Issue Type: Bug
Components: HTTP Interface
Affects Versions: 1.1.1
Environment: Any
Reporter: Henrik Hofmeister
Priority: Minor
If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (COUCHDB-1367) update_seq does not always
reflect the seq of the latest document update
Posted by Jason Smith <jh...@iriscouch.com>.
On Fri, Dec 30, 2011 at 10:46 PM, Robert Newson (Commented) (JIRA)
<ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177798#comment-13177798 ]
>
> Robert Newson commented on COUCHDB-1367:
> ----------------------------------------
>
> I still don't see why we need do anything. My early mistaken understanding of this value should not be used as motivation here. At the time, I was not a "couchdb committer" so the earlier implication that it escaped even my awesome knowledge is to impute omniscience where none is warranted.
That was my oversight. Please forgive the error.
--
Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Jason Smith <jh...@iriscouch.com>.
On Tue, Dec 20, 2011 at 7:39 AM, Robert Newson (Commented) (JIRA)
<ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172791#comment-13172791 ]
>
> Robert Newson commented on COUCHDB-1367:
> ----------------------------------------
>
> Oh, ye of little faith. couchdb-lucene certainly can handle the last_seq output (and has since approximately forever). It's just so rarely encountered that it doesn't help with respect to the issue at hand.
First of all, faith is the most overrated virtue. It is, in fact, a vice.
But no, sorry. I wasn't speaking about couchdb-lucene, but rather
making a minor correction to Bob D.'s comment that a continuous
listener hasn't access to the last_seq object.
--
Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Randall Leeds <ra...@gmail.com>.
On Mon, Dec 26, 2011 at 22:30, Jason Smith <jh...@iriscouch.com> wrote:
> On Tue, Dec 27, 2011 at 9:02 AM, Randall Leeds <ra...@gmail.com> wrote:
>> Awesome. I'm glad you testing descending. Sounds like "last_seq" is a
>> poor name, because it applies to the particular changes request.
>>
>> So then we have this other thing floating around "the sequence number
>> of the last replicable document change".
>> Interestingly, updates to _local/<id> documents don't affect update_seq.
>>
>> Looking into the code, I see all the places where it's bumped
>> artificially. It's quite obvious, actually. Search for "update_seq+1"
>> in couch_db_updater.erl.
>>
>> 1) On setting _revs_limit
>> 2) On setting _security
>> 3) On some call to increment it (who knows why) that has an HTTP POST
>> handler in couch_httpd_misc_handlers that is not exposed by
>> etc/couchdb/default.ini.
>>
>> I don't see any reason why (1) and (2) need to be bumping this number.
>>
>> (3) has been there for a long time but doesn't seem to be part of the
>> default public API.
>> It appears to have been introduced by Damien in May of 2008 (333d18cf)
>> with the commit message:
>>
>> Experimental functionality to increment database update seq, might
>> go away, use at own risk.
>>
>> I propose we just get rid of all these and then update_seq becomes
>> what everyone expects it to be.
>
> It sounds like update_seq an infallible indicator of when a backup is
> necessary--better than the filesystem timestamp for the .couch file.
> Just compare its value to the one from your previous backup. You'll be
> sure to catch not only document updates, but also per-db configuration
> settings like _security and _revs_limit.
>
> I'm not sure if that's 100% correct but if it is, it's useful, or at
> least not useless.
>
> --
> Iris Couch
Thanks a ton for pointing out the operations issue of backup here; I
hadn't considered it. It's worth noting, as I did before, that _local
docs do not affect update_seq, though that's probably not a disaster
for backup scenarios unless _local is being used by something other
than the replicator.
-R
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Jason Smith <jh...@iriscouch.com>.
On Tue, Dec 27, 2011 at 9:02 AM, Randall Leeds <ra...@gmail.com> wrote:
> Awesome. I'm glad you testing descending. Sounds like "last_seq" is a
> poor name, because it applies to the particular changes request.
>
> So then we have this other thing floating around "the sequence number
> of the last replicable document change".
> Interestingly, updates to _local/<id> documents don't affect update_seq.
>
> Looking into the code, I see all the places where it's bumped
> artificially. It's quite obvious, actually. Search for "update_seq+1"
> in couch_db_updater.erl.
>
> 1) On setting _revs_limit
> 2) On setting _security
> 3) On some call to increment it (who knows why) that has an HTTP POST
> handler in couch_httpd_misc_handlers that is not exposed by
> etc/couchdb/default.ini.
>
> I don't see any reason why (1) and (2) need to be bumping this number.
>
> (3) has been there for a long time but doesn't seem to be part of the
> default public API.
> It appears to have been introduced by Damien in May of 2008 (333d18cf)
> with the commit message:
>
> Experimental functionality to increment database update seq, might
> go away, use at own risk.
>
> I propose we just get rid of all these and then update_seq becomes
> what everyone expects it to be.
It sounds like update_seq an infallible indicator of when a backup is
necessary--better than the filesystem timestamp for the .couch file.
Just compare its value to the one from your previous backup. You'll be
sure to catch not only document updates, but also per-db configuration
settings like _security and _revs_limit.
I'm not sure if that's 100% correct but if it is, it's useful, or at
least not useless.
--
Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Paul Davis <pa...@gmail.com>.
On Wed, Dec 28, 2011 at 7:02 PM, Damien Katz <da...@apache.org> wrote:
>
> On Dec 28, 2011, at 4:21 PM, Jason Smith wrote:
>
>> On Thu, Dec 29, 2011 at 1:44 AM, Randall Leeds <ra...@gmail.com> wrote:
>>> On Tue, Dec 27, 2011 at 22:37, Jason Smith <jh...@iriscouch.com> wrote:
>>>> On Wed, Dec 28, 2011 at 9:38 AM, Randall Leeds <ra...@gmail.com> wrote:
>>>>> On Tue, Dec 27, 2011 at 05:22, Jason Smith <jh...@iriscouch.com> wrote:
>>>>>> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <ra...@gmail.com> wrote:
>>>> Your idea improves consistency and orthogonality. It also solves the
>>>> problem of how to enumerate _local docs. (AFAIK there is no way to
>>>> list them all, not via _all_docs, or _changes, or a view).
>>>>
>>>> But it doesn't solve the larger problem: How to follow a _changes feed
>>>> and know when you have caught up. Both Bob N. and I independently did
>>>> the following for our projects:
>>>>
>>>> 1. GET /db and wrongly assume update_seq will appear in the changes feed
>>>> 2. GET /db/_changes?feed=continuous
>>>> 3. Break when a change has .seq >= update_seq
>>>>
>>>> Suppose you have step 0: Update _security or _revs_limit. The loop
>>>> will never break.
>>>>
>>>> You propose (WLOG) _changes?comprehensive=true which guarantees a
>>>> change equal or greater than update_seq. That's cool, but IMO app
>>>> developers now have to add code to ignore irrelevant changes like
>>>> those containing replication checkpoints.
>>>
>>> All great points.
>>>
>>>>
>>>> I propose (WLOG) update_sikh in the db header which is the seq id of
>>>> the latest *document* update. App developers modify their step 1 to
>>>> use update_sikh instead of update_seq.
>>>>
>>>> Is that an accurate synopsis?
>>>
>>> Yes. If we decide to go this route I would rather see _revs_limit and
>>> _security stop bumping update_seq because I find it confusing that
>>> update_seq is then not directly related to the by_seq tree.
>>
>> "update_seq is not directly related to the by_seq tree." That is a
>> very powerful observation. My idea has less orthogonality and
>> conceptual simplicity.
>>
>> The only mystery is why update_seq did this in the first place. Did
>> Damien have a reason, from hard-won experience?
>
> I wanted to give an easy way to monitor other changes to the database header, for things like automated backup and admin tool UIs, though I don't know if anything uses it. It was easiest to do it this way, and I ddn't think about the problems it might cause with users expecting it to correspond with the by_seq index. With all the confusion it's caused, I'd be fine with taking it out. Or maybe put in a db version_seq header, that's like a changes feed but for all types of updates to the db.
>
Having a _changes and _events feed I think makes good sense. It could
get us away from the db update notifier stuff as well if we
consolidated things in there.
Though as a note we need to remember to fix the triggering mechanism
for _view updates because of _purge. I think the simplest way would be
to just check both the _update and _purge seq's instead of just
relying on _purge to have changed the _update seq. Luckily we store
both of these thigns already so it should just be a matter of updating
the logic in the guards when grabbing a view group.
>>
>>> The one exception raised so far is your observation that
>>> it may be useful for doing backups. However, if that's the only real
>>> exception, perhaps we should surface some other thing more directly
>>> analogous to the UNIX crime.
>>
>> Meh, the backups argument is okay, but weak. If you depend on it,
>> you'll miss _local docs (replication checkpoints, so that's no small
>> error). You still have to back up .ini files, and perhaps logs and
>> view files. So my suggestion for Unix crime is just that: OS
>> timestamps.
>>
>> --
>> Iris Couch
>
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
Posted by Damien Katz <da...@apache.org>.
On Dec 28, 2011, at 4:21 PM, Jason Smith wrote:
> On Thu, Dec 29, 2011 at 1:44 AM, Randall Leeds <ra...@gmail.com> wrote:
>> On Tue, Dec 27, 2011 at 22:37, Jason Smith <jh...@iriscouch.com> wrote:
>>> On Wed, Dec 28, 2011 at 9:38 AM, Randall Leeds <ra...@gmail.com> wrote:
>>>> On Tue, Dec 27, 2011 at 05:22, Jason Smith <jh...@iriscouch.com> wrote:
>>>>> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <ra...@gmail.com> wrote:
>>> Your idea improves consistency and orthogonality. It also solves the
>>> problem of how to enumerate _local docs. (AFAIK there is no way to
>>> list them all, not via _all_docs, or _changes, or a view).
>>>
>>> But it doesn't solve the larger problem: How to follow a _changes feed
>>> and know when you have caught up. Both Bob N. and I independently did
>>> the following for our projects:
>>>
>>> 1. GET /db and wrongly assume update_seq will appear in the changes feed
>>> 2. GET /db/_changes?feed=continuous
>>> 3. Break when a change has .seq >= update_seq
>>>
>>> Suppose you have step 0: Update _security or _revs_limit. The loop
>>> will never break.
>>>
>>> You propose (WLOG) _changes?comprehensive=true which guarantees a
>>> change equal or greater than update_seq. That's cool, but IMO app
>>> developers now have to add code to ignore irrelevant changes like
>>> those containing replication checkpoints.
>>
>> All great points.
>>
>>>
>>> I propose (WLOG) update_sikh in the db header which is the seq id of
>>> the latest *document* update. App developers modify their step 1 to
>>> use update_sikh instead of update_seq.
>>>
>>> Is that an accurate synopsis?
>>
>> Yes. If we decide to go this route I would rather see _revs_limit and
>> _security stop bumping update_seq because I find it confusing that
>> update_seq is then not directly related to the by_seq tree.
>
> "update_seq is not directly related to the by_seq tree." That is a
> very powerful observation. My idea has less orthogonality and
> conceptual simplicity.
>
> The only mystery is why update_seq did this in the first place. Did
> Damien have a reason, from hard-won experience?
I wanted to give an easy way to monitor other changes to the database header, for things like automated backup and admin tool UIs, though I don't know if anything uses it. It was easiest to do it this way, and I ddn't think about the problems it might cause with users expecting it to correspond with the by_seq index. With all the confusion it's caused, I'd be fine with taking it out. Or maybe put in a db version_seq header, that's like a changes feed but for all types of updates to the db.
>
>> The one exception raised so far is your observation that
>> it may be useful for doing backups. However, if that's the only real
>> exception, perhaps we should surface some other thing more directly
>> analogous to the UNIX crime.
>
> Meh, the backups argument is okay, but weak. If you depend on it,
> you'll miss _local docs (replication checkpoints, so that's no small
> error). You still have to back up .ini files, and perhaps logs and
> view files. So my suggestion for Unix crime is just that: OS
> timestamps.
>
> --
> Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Jason Smith <jh...@iriscouch.com>.
On Thu, Dec 29, 2011 at 1:44 AM, Randall Leeds <ra...@gmail.com> wrote:
> On Tue, Dec 27, 2011 at 22:37, Jason Smith <jh...@iriscouch.com> wrote:
>> On Wed, Dec 28, 2011 at 9:38 AM, Randall Leeds <ra...@gmail.com> wrote:
>>> On Tue, Dec 27, 2011 at 05:22, Jason Smith <jh...@iriscouch.com> wrote:
>>>> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <ra...@gmail.com> wrote:
>> Your idea improves consistency and orthogonality. It also solves the
>> problem of how to enumerate _local docs. (AFAIK there is no way to
>> list them all, not via _all_docs, or _changes, or a view).
>>
>> But it doesn't solve the larger problem: How to follow a _changes feed
>> and know when you have caught up. Both Bob N. and I independently did
>> the following for our projects:
>>
>> 1. GET /db and wrongly assume update_seq will appear in the changes feed
>> 2. GET /db/_changes?feed=continuous
>> 3. Break when a change has .seq >= update_seq
>>
>> Suppose you have step 0: Update _security or _revs_limit. The loop
>> will never break.
>>
>> You propose (WLOG) _changes?comprehensive=true which guarantees a
>> change equal or greater than update_seq. That's cool, but IMO app
>> developers now have to add code to ignore irrelevant changes like
>> those containing replication checkpoints.
>
> All great points.
>
>>
>> I propose (WLOG) update_sikh in the db header which is the seq id of
>> the latest *document* update. App developers modify their step 1 to
>> use update_sikh instead of update_seq.
>>
>> Is that an accurate synopsis?
>
> Yes. If we decide to go this route I would rather see _revs_limit and
> _security stop bumping update_seq because I find it confusing that
> update_seq is then not directly related to the by_seq tree.
"update_seq is not directly related to the by_seq tree." That is a
very powerful observation. My idea has less orthogonality and
conceptual simplicity.
The only mystery is why update_seq did this in the first place. Did
Damien have a reason, from hard-won experience?
> The one exception raised so far is your observation that
> it may be useful for doing backups. However, if that's the only real
> exception, perhaps we should surface some other thing more directly
> analogous to the UNIX crime.
Meh, the backups argument is okay, but weak. If you depend on it,
you'll miss _local docs (replication checkpoints, so that's no small
error). You still have to back up .ini files, and perhaps logs and
view files. So my suggestion for Unix crime is just that: OS
timestamps.
--
Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Randall Leeds <ra...@gmail.com>.
On Tue, Dec 27, 2011 at 22:37, Jason Smith <jh...@iriscouch.com> wrote:
> On Wed, Dec 28, 2011 at 9:38 AM, Randall Leeds <ra...@gmail.com> wrote:
>> On Tue, Dec 27, 2011 at 05:22, Jason Smith <jh...@iriscouch.com> wrote:
>>> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <ra...@gmail.com> wrote:
> Your idea improves consistency and orthogonality. It also solves the
> problem of how to enumerate _local docs. (AFAIK there is no way to
> list them all, not via _all_docs, or _changes, or a view).
>
> But it doesn't solve the larger problem: How to follow a _changes feed
> and know when you have caught up. Both Bob N. and I independently did
> the following for our projects:
>
> 1. GET /db and wrongly assume update_seq will appear in the changes feed
> 2. GET /db/_changes?feed=continuous
> 3. Break when a change has .seq >= update_seq
>
> Suppose you have step 0: Update _security or _revs_limit. The loop
> will never break.
>
> You propose (WLOG) _changes?comprehensive=true which guarantees a
> change equal or greater than update_seq. That's cool, but IMO app
> developers now have to add code to ignore irrelevant changes like
> those containing replication checkpoints.
All great points.
>
> I propose (WLOG) update_sikh in the db header which is the seq id of
> the latest *document* update. App developers modify their step 1 to
> use update_sikh instead of update_seq.
>
> Is that an accurate synopsis?
Yes. If we decide to go this route I would rather see _revs_limit and
_security stop bumping update_seq because I find it confusing that
update_seq is then not directly related to the by_seq tree. Granted,
app developers don't need to know anything about the by_seq tree, but
I don't see a compelling reason to keep the old definition of
update_seq. The one exception raised so far is your observation that
it may be useful for doing backups. However, if that's the only real
exception, perhaps we should surface some other thing more directly
analogous to the UNIX ctime.
> Is it really true that shards need the same _revs_limit as the
> simulated whole? Maybe they really want _revs_limit /
> number_of_shards?
>
> Is it really necessary that _security be identical in each shard?
> Actually, yes it is, because validate_doc_update uses it. But still...
>
> How are you computing doc_count in the /db response? You have to sum
> doc_count from each shard. But every shard needs a copy of every
> design doc for validation. So you have to subtract those back out? My
> broader point is, sharding applications already do lots of magic. I'm
> not sure if replicating _security and _local docs buys you much.
Good observations as well.
-Randall
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Jason Smith <jh...@iriscouch.com>.
On Wed, Dec 28, 2011 at 9:38 AM, Randall Leeds <ra...@gmail.com> wrote:
> On Tue, Dec 27, 2011 at 05:22, Jason Smith <jh...@iriscouch.com> wrote:
>> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <ra...@gmail.com> wrote:
>>> Either
>>> these things have a proper place in the sequential history of a
>>> database or they do not. That there are things which affect update_seq
>>> but do not appear in the by_seq index and _changes feed feels like a
>>> mistake.
>>
>> The first sentence is, well, a tautology actually, but it asks the
>> right question and the answer is they DO NOT belong. _changes shows
>> data, not metadata. By definition, _changes is anything worth
>> replicating.
>
> That strikes me as incorrect. The _changes feed is purely metadata
> unless ?include_docs=true is specified.
Yes, "data" and "metadata" are problematic words. I'll stop using them.
Do you agree that _changes is, by definition, anything worth replicating?
>> But I hope my filesystem example above shows why it is okay to
>> increment update_seq but not change by_seq.
>
> You show a nice precedent for separating metadata and data, but
> CouchDB has a decent precedent of avoiding this same thing. For
> example, _id and _rev are in the returned document body rather than
> part of the HTTP request (it could have been just URL and entity tag
> headers only for this).
Yeah that's a good point.
>> The bug with update_seq is not that it it is too eager (increments for
>> _security, _revs_limit), but it is not eager enough (it should bump
>> for _local too).
>>
>
> I agree, but for different reasons. I think _local docs may have a
> place in by_seq even if the default _changes request still only shows
> the default, replicable documents.
That's an interesting idea.
IMO, _security, _revs_limit, apply to a specific database and URL, and
consequently must never replicate. _local docs are those which don't
replicate. If _local would replicate, I'd worry about spurious
checkpoints spreading to where they don't belong; and unchecked
_security replication is even worse.
Your idea improves consistency and orthogonality. It also solves the
problem of how to enumerate _local docs. (AFAIK there is no way to
list them all, not via _all_docs, or _changes, or a view).
But it doesn't solve the larger problem: How to follow a _changes feed
and know when you have caught up. Both Bob N. and I independently did
the following for our projects:
1. GET /db and wrongly assume update_seq will appear in the changes feed
2. GET /db/_changes?feed=continuous
3. Break when a change has .seq >= update_seq
Suppose you have step 0: Update _security or _revs_limit. The loop
will never break.
You propose (WLOG) _changes?comprehensive=true which guarantees a
change equal or greater than update_seq. That's cool, but IMO app
developers now have to add code to ignore irrelevant changes like
those containing replication checkpoints.
I propose (WLOG) update_sikh in the db header which is the seq id of
the latest *document* update. App developers modify their step 1 to
use update_sikh instead of update_seq.
Is that an accurate synopsis?
>> 2. As a frequent consumer of _changes, I would prefer *not* to see
>> _local documents, nor _security or other updates in there. They are
>> metadata, not data. Maybe I misunderstood, but nobody wants to
>> *replicate* _security objects or _local docs; they just want MVCC
>> semantics (Adam on _security, IIRC) and a simplified API (me, on
>> making all metadata a _local doc, and making _local docs full MVCC).
>
> I think you misunderstand, maybe. In the case of BigCouch, MVCC is all
> that's needed because the replication does not go over HTTP. I see no
> reason to require that special care be taken to copy these objects
> when a flag on the _changes feed might cause them to be transferred
> very naturally. In particular, I would use this feature in a
> hypothetical Lounge 3.0. It also means that with admin privileges we
> could do full backup replications.
If couch could do this, then cool. But consider that both examples are
the same special-case: sharding and simulating a normal database API
when there are actually multiple parts. That sounds like an
application concern.
Is it really true that shards need the same _revs_limit as the
simulated whole? Maybe they really want _revs_limit /
number_of_shards?
Is it really necessary that _security be identical in each shard?
Actually, yes it is, because validate_doc_update uses it. But still...
How are you computing doc_count in the /db response? You have to sum
doc_count from each shard. But every shard needs a copy of every
design doc for validation. So you have to subtract those back out? My
broader point is, sharding applications already do lots of magic. I'm
not sure if replicating _security and _local docs buys you much.
But you've definitely persuaded me that your idea works. It is the
second-best proposal I've seen in this thread :)
--
Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Randall Leeds <ra...@gmail.com>.
On Tue, Dec 27, 2011 at 05:22, Jason Smith <jh...@iriscouch.com> wrote:
> On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <ra...@gmail.com> wrote:
>> Either
>> these things have a proper place in the sequential history of a
>> database or they do not. That there are things which affect update_seq
>> but do not appear in the by_seq index and _changes feed feels like a
>> mistake.
>
> The first sentence is, well, a tautology actually, but it asks the
> right question and the answer is they DO NOT belong. _changes shows
> data, not metadata. By definition, _changes is anything worth
> replicating.
That strikes me as incorrect. The _changes feed is purely metadata
unless ?include_docs=true is specified.
>
> But I hope my filesystem example above shows why it is okay to
> increment update_seq but not change by_seq.
You show a nice precedent for separating metadata and data, but
CouchDB has a decent precedent of avoiding this same thing. For
example, _id and _rev are in the returned document body rather than
part of the HTTP request (it could have been just URL and entity tag
headers only for this).
>
> The bug with update_seq is not that it it is too eager (increments for
> _security, _revs_limit), but it is not eager enough (it should bump
> for _local too).
>
I agree, but for different reasons. I think _local docs may have a
place in by_seq even if the default _changes request still only shows
the default, replicable documents.
> 2. As a frequent consumer of _changes, I would prefer *not* to see
> _local documents, nor _security or other updates in there. They are
> metadata, not data. Maybe I misunderstood, but nobody wants to
> *replicate* _security objects or _local docs; they just want MVCC
> semantics (Adam on _security, IIRC) and a simplified API (me, on
> making all metadata a _local doc, and making _local docs full MVCC).
I think you misunderstand, maybe. In the case of BigCouch, MVCC is all
that's needed because the replication does not go over HTTP. I see no
reason to require that special care be taken to copy these objects
when a flag on the _changes feed might cause them to be transferred
very naturally. In particular, I would use this feature in a
hypothetical Lounge 3.0. It also means that with admin privileges we
could do full backup replications.
-R
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Jason Smith <jh...@iriscouch.com>.
On Tue, Dec 27, 2011 at 5:04 AM, Randall Leeds <ra...@gmail.com> wrote:
> Yes it does. There is mostly consistent relationship between update
> sequence (seq, update_seq, last_seq, committed_seq) and the by_seq
> index. It seems entirely too confusing that there are things which
> affect update_seq but do not appear in the by_seq btree. That is just
> plain wrong, else a massive confusion of vocabulary.
I think it is confused definitions.
If by_seq is defined as "the sequence ids of regular documents" then
it is implemented correctly.
> Bear with me for I believe ther is a related discussion about
> replicability for _security, _local docs, etc. It's clear that there
> are clustering and operational motivations for making this information
> replicable, thus making them proper documents with a place in the
> by_seq index, in the _changes feed, and affecting update_seq.
It is common to track changes to data vs. changes to metadata
independently. Changing Unix file permissions updates ctime but not
mtime. Changing Unix files updates both ctime and mtime. In CouchDB,
update_seq plays the role of ctime (data or metadata updates), and
nobody's been cast for mtime (metadata-only updates).
> Either
> these things have a proper place in the sequential history of a
> database or they do not. That there are things which affect update_seq
> but do not appear in the by_seq index and _changes feed feels like a
> mistake.
The first sentence is, well, a tautology actually, but it asks the
right question and the answer is they DO NOT belong. _changes shows
data, not metadata. By definition, _changes is anything worth
replicating.
But I hope my filesystem example above shows why it is okay to
increment update_seq but not change by_seq.
The bug with update_seq is not that it it is too eager (increments for
_security, _revs_limit), but it is not eager enough (it should bump
for _local too).
2. As a frequent consumer of _changes, I would prefer *not* to see
_local documents, nor _security or other updates in there. They are
metadata, not data. Maybe I misunderstood, but nobody wants to
*replicate* _security objects or _local docs; they just want MVCC
semantics (Adam on _security, IIRC) and a simplified API (me, on
making all metadata a _local doc, and making _local docs full MVCC).
> Placing additional metadata in the db header feels like
> rubbing salt in this wound.
On the contrary, IMHO, we want
1. A new value: the sequence id of the most recent document update (pretty sure)
2. Available to the client alongside existing values like doc_count
doc_del_count (somewhat sure)
> Right now only replicable documents surface in the _changes feed and
> are added to the by_seq btree but some other things affect the
> update_seq. I've just gone and checked, as described in my previous
> email, that none of these appear to require a change to update_seq for
> any technical reason, though Jason properly points out that it is
> perhaps useful for operational tasks such as knowing when to back up a
> .couch file.
Here is where get into migrating to more _local docs. I am actually
not sure if that's good for this discussion. But anyway, my basic
feeling is
* All metadata that clients can change is _local docs, with MVCC, *not
in* the by_seq tree
* update_seq counts changes to data or metadata
* update_sikh (WLOG) counts changes to documents only (changes to the
by_seq tree)
Doable? It bears mentioning that I haven't any idea what I am talking about.
> Thoughts, concerns, emotions and relevant, famous quotations encouraged.
WELCOME TO THE PARTY, PAL!
--
Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
Posted by Robert Dionne <di...@dionne-associates.com>.
What's interesting is that modulo one edge case, last_seq in the changes feed and update_seq in the db_info record are exactly as defined on the WIKI.
update_seq:
Current number of updates to the database (int)
last_seq:
last_seq is the sequence number of the last update returned. (Currently it will always be the same as the seq of the last item in results.)
this holds true also when there are no changes to documents, the value of last_seq is zero. The one edge case (which is a bit odd) is seen when you retrieve last_seq using ?descending=true&limit=1. If there are no changes the value will still be zero unless you call _set_revs_limit first in which case the value will be one. The value will still be zero if the normal _changes is called with no args. What makes it odd is that calling _changes?descending... after a call to _set_revs_limit does not impact the value of last_seq. This is a bug.
So yes it's a bit weird but it does pretty much agree with the documentation. The quote I'm looking for is the one about angels on the head of a pin.
I guess it needs more thought. In general I don't like metadata because I think it creates more things that need to be handled differently, adding complexity for the sake of something that doesn't exist (metadata).
Do you have any more swatches in magenta?
On Dec 27, 2011, at 12:04 AM, Randall Leeds wrote:
> On Mon, Dec 26, 2011 at 08:49, Jason Smith <jh...@iriscouch.com> wrote:
>> Hi, Bob. Thanks for your feedback.
>>
>> On Mon, Dec 26, 2011 at 12:24 PM, Robert Dionne
>> <di...@dionne-associates.com> wrote:
>>> Jason,
>>>
>>> After looking into this a bit I do not think it's a bug, at most poor documentation. update_seq != last_seq
>>
>> Nobody knows what update_seq means. Even a CouchDB committer got it wrong.
>>
>> Fine. It is "poor documentation."
>>
>> Adding last_seq into db_info is not helpful because last_seq also does
>> not mean what we think it means. My last email demonstrates that
>> last_seq is in fact incoherent.
>
> <snip>
>
> On Mon, Dec 26, 2011 at 23:03, Benoit Chesneau <bc...@gmail.com> wrote:
>> Mmm right that confusing (maybe except if you consider update_seq as a
>> way to know the numbers of updates in the databases but in this case
>> the wording is confiusing) . Imo changes seq & commited_seq should be
>> quites the same. At least a changes seq should only happen when there
>> is a doc update ie each time and only if a revision is created. Does
>> that make sense?
>>
>> - benoiît
>
> Yes it does. There is mostly consistent relationship between update
> sequence (seq, update_seq, last_seq, committed_seq) and the by_seq
> index. It seems entirely too confusing that there are things which
> affect update_seq but do not appear in the by_seq btree. That is just
> plain wrong, else a massive confusion of vocabulary. Benoit, I believe
> you are right to suggest that none of these sequences-related things
> should change unless a revision is created.
>
> Bear with me for I believe ther is a related discussion about
> replicability for _security, _local docs, etc. It's clear that there
> are clustering and operational motivations for making this information
> replicable, thus making them proper documents with a place in the
> by_seq index, in the _changes feed, and affecting update_seq. Either
> these things have a proper place in the sequential history of a
> database or they do not. That there are things which affect update_seq
> but do not appear in the by_seq index and _changes feed feels like a
> mistake. Placing additional metadata in the db header feels like
> rubbing salt in this wound.
>
> Right now only replicable documents surface in the _changes feed and
> are added to the by_seq btree but some other things affect the
> update_seq. I've just gone and checked, as described in my previous
> email, that none of these appear to require a change to update_seq for
> any technical reason, though Jason properly points out that it is
> perhaps useful for operational tasks such as knowing when to back up a
> .couch file.
>
> I see two reasonable ways forward.
>
> 1) Stop incrementing update_seq for anything but replicable document changes
> 2) Make things which already affect update_seq but do not appear in
> _changes appear there, likely by turning them into proper MVCC
> documents.
>
> Regarding option 1:
> This is easy. I already outlined how to do this. It requires removing
> about 3 characters from our codebase. However, it spits at Jason's
> operations concerns, which I think are quite valid, and misses an
> opportunity for great improvement.
>
> Regarding option 2:
> There is a cluster-aware use case, an operations use case, and, I
> think, a purity argument here. As for how to accomplish this feat
> without terrible API breakage, we get a lot of help from our URL
> structure. We have reserved paths which cannot conflict with documents
> so it does not create ambiguity if '{"seq":20,"id":"_security", ...}'
> appears in a changes feed. However, I think _security is a bad name
> for this document because it requires that /_security API
> compatibility is broken.
>
> One solution I like right now is to add a _meta (without loss of
> generality -- insert your own preferred name) document, with the
> normal MVCC document API, referenced by the by_seq index and appearing
> in the _changes feed, which contains both _revs_limit and _security
> while preserving the legacy, cloberring, MVCC-oblivious APIs. Voila!
> No breaking changes. Keep a pointer latest revision of this document
> in the database header for fast access (and perhaps cache it in
> memory).
>
> It would probably be acceptable to keep these out of a vanilla changes
> request (after all, they require db admin credentials to modify, and
> in the case of _security to view). Opening the door to additional
> flags for _changes also allows us to provide a natural extension of
> this idea to replicable _local docs for the clustering use case.
>
> Thoughts, concerns, emotions and relevant, famous quotations encouraged.
>
> -Randall
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Randall Leeds <ra...@gmail.com>.
On Mon, Dec 26, 2011 at 08:49, Jason Smith <jh...@iriscouch.com> wrote:
> Hi, Bob. Thanks for your feedback.
>
> On Mon, Dec 26, 2011 at 12:24 PM, Robert Dionne
> <di...@dionne-associates.com> wrote:
>> Jason,
>>
>> After looking into this a bit I do not think it's a bug, at most poor documentation. update_seq != last_seq
>
> Nobody knows what update_seq means. Even a CouchDB committer got it wrong.
>
> Fine. It is "poor documentation."
>
> Adding last_seq into db_info is not helpful because last_seq also does
> not mean what we think it means. My last email demonstrates that
> last_seq is in fact incoherent.
<snip>
On Mon, Dec 26, 2011 at 23:03, Benoit Chesneau <bc...@gmail.com> wrote:
> Mmm right that confusing (maybe except if you consider update_seq as a
> way to know the numbers of updates in the databases but in this case
> the wording is confiusing) . Imo changes seq & commited_seq should be
> quites the same. At least a changes seq should only happen when there
> is a doc update ie each time and only if a revision is created. Does
> that make sense?
>
> - benoiît
Yes it does. There is mostly consistent relationship between update
sequence (seq, update_seq, last_seq, committed_seq) and the by_seq
index. It seems entirely too confusing that there are things which
affect update_seq but do not appear in the by_seq btree. That is just
plain wrong, else a massive confusion of vocabulary. Benoit, I believe
you are right to suggest that none of these sequences-related things
should change unless a revision is created.
Bear with me for I believe ther is a related discussion about
replicability for _security, _local docs, etc. It's clear that there
are clustering and operational motivations for making this information
replicable, thus making them proper documents with a place in the
by_seq index, in the _changes feed, and affecting update_seq. Either
these things have a proper place in the sequential history of a
database or they do not. That there are things which affect update_seq
but do not appear in the by_seq index and _changes feed feels like a
mistake. Placing additional metadata in the db header feels like
rubbing salt in this wound.
Right now only replicable documents surface in the _changes feed and
are added to the by_seq btree but some other things affect the
update_seq. I've just gone and checked, as described in my previous
email, that none of these appear to require a change to update_seq for
any technical reason, though Jason properly points out that it is
perhaps useful for operational tasks such as knowing when to back up a
.couch file.
I see two reasonable ways forward.
1) Stop incrementing update_seq for anything but replicable document changes
2) Make things which already affect update_seq but do not appear in
_changes appear there, likely by turning them into proper MVCC
documents.
Regarding option 1:
This is easy. I already outlined how to do this. It requires removing
about 3 characters from our codebase. However, it spits at Jason's
operations concerns, which I think are quite valid, and misses an
opportunity for great improvement.
Regarding option 2:
There is a cluster-aware use case, an operations use case, and, I
think, a purity argument here. As for how to accomplish this feat
without terrible API breakage, we get a lot of help from our URL
structure. We have reserved paths which cannot conflict with documents
so it does not create ambiguity if '{"seq":20,"id":"_security", ...}'
appears in a changes feed. However, I think _security is a bad name
for this document because it requires that /_security API
compatibility is broken.
One solution I like right now is to add a _meta (without loss of
generality -- insert your own preferred name) document, with the
normal MVCC document API, referenced by the by_seq index and appearing
in the _changes feed, which contains both _revs_limit and _security
while preserving the legacy, cloberring, MVCC-oblivious APIs. Voila!
No breaking changes. Keep a pointer latest revision of this document
in the database header for fast access (and perhaps cache it in
memory).
It would probably be acceptable to keep these out of a vanilla changes
request (after all, they require db admin credentials to modify, and
in the case of _security to view). Opening the door to additional
flags for _changes also allows us to provide a natural extension of
this idea to replicable _local docs for the clustering use case.
Thoughts, concerns, emotions and relevant, famous quotations encouraged.
-Randall
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Benoit Chesneau <bc...@gmail.com>.
On Tue, Dec 27, 2011 at 3:02 AM, Randall Leeds <ra...@gmail.com> wrote:
> On Mon, Dec 26, 2011 at 08:49, Jason Smith <jh...@iriscouch.com> wrote:
>> Hi, Bob. Thanks for your feedback.
>>
>> On Mon, Dec 26, 2011 at 12:24 PM, Robert Dionne
>> <di...@dionne-associates.com> wrote:
>>> Jason,
>>>
>>> After looking into this a bit I do not think it's a bug, at most poor documentation. update_seq != last_seq
>>
>> Nobody knows what update_seq means. Even a CouchDB committer got it wrong.
>>
>> Fine. It is "poor documentation."
>>
>> Adding last_seq into db_info is not helpful because last_seq also does
>> not mean what we think it means. My last email demonstrates that
>> last_seq is in fact incoherent.
>
> Awesome. I'm glad you testing descending. Sounds like "last_seq" is a
> poor name, because it applies to the particular changes request.
>
> So then we have this other thing floating around "the sequence number
> of the last replicable document change".
> Interestingly, updates to _local/<id> documents don't affect update_seq.
>
> Looking into the code, I see all the places where it's bumped
> artificially. It's quite obvious, actually. Search for "update_seq+1"
> in couch_db_updater.erl.
>
> 1) On setting _revs_limit
> 2) On setting _security
> 3) On some call to increment it (who knows why) that has an HTTP POST
> handler in couch_httpd_misc_handlers that is not exposed by
> etc/couchdb/default.ini.
>
> I don't see any reason why (1) and (2) need to be bumping this number.
>
> (3) has been there for a long time but doesn't seem to be part of the
> default public API.
> It appears to have been introduced by Damien in May of 2008 (333d18cf)
> with the commit message:
>
> Experimental functionality to increment database update seq, might
> go away, use at own risk.
>
> I propose we just get rid of all these and then update_seq becomes
> what everyone expects it to be.
>
> -Randall
Mmm right that confusing (maybe except if you consider update_seq as a
way to know the numbers of updates in the databases but in this case
the wording is confiusing) . Imo changes seq & commited_seq should be
quites the same. At least a changes seq should only happen when there
is a doc update ie each time and only if a revision is created. Does
that make sense?
- benoiît
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Randall Leeds <ra...@gmail.com>.
On Mon, Dec 26, 2011 at 08:49, Jason Smith <jh...@iriscouch.com> wrote:
> Hi, Bob. Thanks for your feedback.
>
> On Mon, Dec 26, 2011 at 12:24 PM, Robert Dionne
> <di...@dionne-associates.com> wrote:
>> Jason,
>>
>> After looking into this a bit I do not think it's a bug, at most poor documentation. update_seq != last_seq
>
> Nobody knows what update_seq means. Even a CouchDB committer got it wrong.
>
> Fine. It is "poor documentation."
>
> Adding last_seq into db_info is not helpful because last_seq also does
> not mean what we think it means. My last email demonstrates that
> last_seq is in fact incoherent.
Awesome. I'm glad you testing descending. Sounds like "last_seq" is a
poor name, because it applies to the particular changes request.
So then we have this other thing floating around "the sequence number
of the last replicable document change".
Interestingly, updates to _local/<id> documents don't affect update_seq.
Looking into the code, I see all the places where it's bumped
artificially. It's quite obvious, actually. Search for "update_seq+1"
in couch_db_updater.erl.
1) On setting _revs_limit
2) On setting _security
3) On some call to increment it (who knows why) that has an HTTP POST
handler in couch_httpd_misc_handlers that is not exposed by
etc/couchdb/default.ini.
I don't see any reason why (1) and (2) need to be bumping this number.
(3) has been there for a long time but doesn't seem to be part of the
default public API.
It appears to have been introduced by Damien in May of 2008 (333d18cf)
with the commit message:
Experimental functionality to increment database update seq, might
go away, use at own risk.
I propose we just get rid of all these and then update_seq becomes
what everyone expects it to be.
-Randall
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Jason Smith <jh...@iriscouch.com>.
Hi, Bob. Thanks for your feedback.
On Mon, Dec 26, 2011 at 12:24 PM, Robert Dionne
<di...@dionne-associates.com> wrote:
> Jason,
>
> After looking into this a bit I do not think it's a bug, at most poor documentation. update_seq != last_seq
Nobody knows what update_seq means. Even a CouchDB committer got it wrong.
Fine. It is "poor documentation."
Adding last_seq into db_info is not helpful because last_seq also does
not mean what we think it means. My last email demonstrates that
last_seq is in fact incoherent.
What would be marvelous in the db_info is this:
The sequence id of the most recent document update, or zero if
there has not been one.
If you add it, that would be great but you might first consider what
its relationship is to last_seq, why the values differ sometimes, and
whether that is good for users.
Thanks again!
--
Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
Posted by Robert Dionne <di...@dionne-associates.com>.
Jason,
After looking into this a bit I do not think it's a bug, at most poor documentation. update_seq != last_seq Most of the time it does but as we know now sometimes it doesn't. It's a different thing. I"m not sure where else in the code we depend on update_seq reflecting all the changes to the database, perhaps as Randall suggests we might be able to *not* bump it in those other calls.
Another way to handle this is hang on to the last_seq when a changes call is made and use that as a since parameter in the next call. This to me seems like what's needed in this use case anyway.
In any event it's likely easy to add last_seq to the db_info record, and I'm more than happy to do that, we should open a new ticket for that.
Cheers,
Bob
On Dec 26, 2011, at 4:10 AM, Jason Smith wrote:
> Hi, Randall. Thanks for inviting me to argue a bit more. I hope you'll
> be persuaded that, if -1367 is not a bug, at least there is *some*
> bug.
>
> tl;dr summary:
>
> This is a real bug--a paper cut with a workaround, but still a real bug.
>
> 1. Apps want a changes feed since 0, but they want to know when
> they've "caught up" (defined below)
> 2. These apps (and robust apps generally) probably start out by
> pinging the /db anyway. Bob N. and I independently did so.
> 3. update_seq looks deceptively like the sequence id of the latest
> change, and people assume so. They define "caught up" as receiving a
> change at or above this value. They expect to "catch up" in finite
> time, and even if the db receives no subsequent updates.
> 4. In fact, CouchDB does not disclose the sequence id of the latest
> change in the /db response. To know that value:
> 4a. If you want to process every change anyway, just get _changes
> and use last_seq
> 4b. If you just want the last sequence id, query
> _changes?descending=true&limit=1
> 4b(1). If the response has a change, use its last_seq value
> 4b(2). If the response has no changes, ignore the last_seq value
> (it is really the update_seq) and use 0
>
> Step 3 is the major paper cut. That step 4 exists and is complicated
> is the minor paper cut.
>
> On Mon, Dec 26, 2011 at 5:36 AM, Randall Leeds (Commented) (JIRA)
> <ji...@apache.org> wrote:
>>
>> [ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175892#comment-13175892 ]
>>
>> Randall Leeds commented on COUCHDB-1367:
>> ----------------------------------------
>>
>>> Wait a second. Robert, you are not fixing a bug in C-L, you are working around a deficiency in CouchDB.
>>
>> Can't both be true?
>
> Only in the trivial sense. This ticket reveals that app
> developers--Henrik and me, but also a committer--misunderstand
> update_seq, thinking it is last_seq. last_seq is not easy to learn.
>
>> Nope. You can not ever know. You always know the latest sequence number at some arbitrarily recent point in time.
>
> Sorry, I cut corners and was not clear. Of course, nobody ever really
> knows anything except events in the very recent past. But I mean in
> the context of a _changes query one-two punch: get the last_seq, then
> begin a continuous feed since that value.
>
> The bug is that users cannot readily know the id of the most recent
> change. In fact, "the id of the most recent change" has no explicit
> label or name in the CouchDB interface. Neither update_seq nor
> last_seq mean exactly that.
>
>>> What if I want to see the most recent five changes? What if there are a hundred million documents? What if 99% of the time, update_seq equals last_seq and so developers assume it means something it doesn't?
>>
>> In order:
>> * /_changes?descending=true&limit=5
>
> I stand corrected. I had forgotten about a descending changes query.
> That resolves the hundred-million-docs problem. (My erroneous point
> was, 100M docs makes it too expensive to learn last_seq.)
>
> But that response looks bizarre.
>
> GET /db/_changes?descending=true\&limit=5
> {"results":[
> {"seq":22,"id":"after_3","changes":[{"rev":"1-0785e9eb543380151003dc452c3a001a"}]},
> {"seq":21,"id":"after_2","changes":[{"rev":"1-0785e9eb543380151003dc452c3a001a"}]},
> {"seq":20,"id":"after_1","changes":[{"rev":"1-0785e9eb543380151003dc452c3a001a"}]},
> {"seq":19,"id":"conc","changes":[{"rev":"2-584a4a504a97009241d2587fee8b5eb8"}]},
> {"seq":17,"id":"preload_create","changes":[{"rev":"1-28bf6cd8af83c40c6e3fb82b608ce98f"}]}
> ],
> "last_seq":17}
>
> last_seq is the *least recent* change. If you query with &limit=1 then
> they will be equal, and that is nice. *Except* if there were no
> changes yet.
>
> $ curl -X PUT localhost:5984/x
> {"ok":true}
>
> $ curl -X PUT localhost:5984/x/_revs_limit -d $RANDOM
> {"ok":true}
> $ curl -X PUT localhost:5984/x/_revs_limit -d $RANDOM
> {"ok":true}
> $ curl -X PUT localhost:5984/x/_revs_limit -d $RANDOM
> {"ok":true}
>
> $ curl localhost:5984/x/_changes
> {"results":[
>
> ],
> "last_seq":0}
>
> $ curl localhost:5984/x/_changes?descending=true
> {"results":[
>
> ],
> "last_seq":3}
>
> Weird.
>
>> * Add additional information to the changes feed, perhaps with a query parameter (almost the reverse of include docs)
>> * Stop incrementing the update sequence on certain kinds of non-document changes
>> * Add more information to the db information response
>
> A commonly-needed and valuable piece of data like this seems most
> appropriate cached in the db header and served in the db information.
>
> --
> Iris Couch
Re: [jira] [Commented] (COUCHDB-1367) When settings revs_limit on db
- the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by Jason Smith <jh...@iriscouch.com>.
Hi, Randall. Thanks for inviting me to argue a bit more. I hope you'll
be persuaded that, if -1367 is not a bug, at least there is *some*
bug.
tl;dr summary:
This is a real bug--a paper cut with a workaround, but still a real bug.
1. Apps want a changes feed since 0, but they want to know when
they've "caught up" (defined below)
2. These apps (and robust apps generally) probably start out by
pinging the /db anyway. Bob N. and I independently did so.
3. update_seq looks deceptively like the sequence id of the latest
change, and people assume so. They define "caught up" as receiving a
change at or above this value. They expect to "catch up" in finite
time, and even if the db receives no subsequent updates.
4. In fact, CouchDB does not disclose the sequence id of the latest
change in the /db response. To know that value:
4a. If you want to process every change anyway, just get _changes
and use last_seq
4b. If you just want the last sequence id, query
_changes?descending=true&limit=1
4b(1). If the response has a change, use its last_seq value
4b(2). If the response has no changes, ignore the last_seq value
(it is really the update_seq) and use 0
Step 3 is the major paper cut. That step 4 exists and is complicated
is the minor paper cut.
On Mon, Dec 26, 2011 at 5:36 AM, Randall Leeds (Commented) (JIRA)
<ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175892#comment-13175892 ]
>
> Randall Leeds commented on COUCHDB-1367:
> ----------------------------------------
>
>> Wait a second. Robert, you are not fixing a bug in C-L, you are working around a deficiency in CouchDB.
>
> Can't both be true?
Only in the trivial sense. This ticket reveals that app
developers--Henrik and me, but also a committer--misunderstand
update_seq, thinking it is last_seq. last_seq is not easy to learn.
> Nope. You can not ever know. You always know the latest sequence number at some arbitrarily recent point in time.
Sorry, I cut corners and was not clear. Of course, nobody ever really
knows anything except events in the very recent past. But I mean in
the context of a _changes query one-two punch: get the last_seq, then
begin a continuous feed since that value.
The bug is that users cannot readily know the id of the most recent
change. In fact, "the id of the most recent change" has no explicit
label or name in the CouchDB interface. Neither update_seq nor
last_seq mean exactly that.
>> What if I want to see the most recent five changes? What if there are a hundred million documents? What if 99% of the time, update_seq equals last_seq and so developers assume it means something it doesn't?
>
> In order:
> * /_changes?descending=true&limit=5
I stand corrected. I had forgotten about a descending changes query.
That resolves the hundred-million-docs problem. (My erroneous point
was, 100M docs makes it too expensive to learn last_seq.)
But that response looks bizarre.
GET /db/_changes?descending=true\&limit=5
{"results":[
{"seq":22,"id":"after_3","changes":[{"rev":"1-0785e9eb543380151003dc452c3a001a"}]},
{"seq":21,"id":"after_2","changes":[{"rev":"1-0785e9eb543380151003dc452c3a001a"}]},
{"seq":20,"id":"after_1","changes":[{"rev":"1-0785e9eb543380151003dc452c3a001a"}]},
{"seq":19,"id":"conc","changes":[{"rev":"2-584a4a504a97009241d2587fee8b5eb8"}]},
{"seq":17,"id":"preload_create","changes":[{"rev":"1-28bf6cd8af83c40c6e3fb82b608ce98f"}]}
],
"last_seq":17}
last_seq is the *least recent* change. If you query with &limit=1 then
they will be equal, and that is nice. *Except* if there were no
changes yet.
$ curl -X PUT localhost:5984/x
{"ok":true}
$ curl -X PUT localhost:5984/x/_revs_limit -d $RANDOM
{"ok":true}
$ curl -X PUT localhost:5984/x/_revs_limit -d $RANDOM
{"ok":true}
$ curl -X PUT localhost:5984/x/_revs_limit -d $RANDOM
{"ok":true}
$ curl localhost:5984/x/_changes
{"results":[
],
"last_seq":0}
$ curl localhost:5984/x/_changes?descending=true
{"results":[
],
"last_seq":3}
Weird.
> * Add additional information to the changes feed, perhaps with a query parameter (almost the reverse of include docs)
> * Stop incrementing the update sequence on certain kinds of non-document changes
> * Add more information to the db information response
A commonly-needed and valuable piece of data like this seems most
appropriate cached in the db header and served in the db information.
--
Iris Couch
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Jason Smith (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172752#comment-13172752 ]
Jason Smith commented on COUCHDB-1367:
--------------------------------------
Hi, Bob. You said,
> couchdb-lucene uses continuous changes feeds so it doesn't have access to the last_seq value of a normal changes feed.
If the database is deleted during a continuous _changes request, CouchDB will send {"last_seq":N} and disconnect the client. Perhaps it's not relevant to this bug. But a robust changes listener should be prepared to get a last_seq message.
I learned this from a bug report in Follow https://github.com/iriscouch/follow/issues/6
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175162#comment-13175162 ]
Randall Leeds commented on COUCHDB-1367:
----------------------------------------
I'd propose making the heartbeat option send the last_seq row, but that'd probably break clients that expect a disconnect at that point.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175357#comment-13175357 ]
Robert Newson commented on COUCHDB-1367:
----------------------------------------
On reflection, it's couchdb-lucene's bug, not couchdb's. Let me explain.
CouchDB-Lucene (to give it its grown-up name) compares the update_seq from a GET /dbname to the sequences a background process is indexing through. It then unblocks searcher threads as that process reaches or exceeds the required update_seq. This is, in fact, just silly.
Instead, a search query should cause a GET /dbname/_changes?since=<latest index checkpoint>. It should block until it consumes the entire response, passing the updates to the indexing process. It can then return a non-stale search result. In the case that the index is fresh, the _changes response contains no rows, and serves only to confirm that the index is fresh. If, as planned, CouchDB-Lucene *also* runs a _changes?feed=continuous to keep indexes fresh in the background then indexes will simply be fresher than they would be in the CouchDB case.
I repeat, CouchDB-Lucene's *mistake* is to *only* use the feed=continuous variety of the changes feed. This prevents it from knowing when its own index is fresh.
I will make this change next week and I suggest that this ticket be closed with no further action taken.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175217#comment-13175217 ]
Randall Leeds commented on COUCHDB-1367:
----------------------------------------
My gut says it's more confusing to put update_seq and last_seq in the db info. If clients want to be sure they're up to date, they need only pass a heartbeat parameter to the changes feed. If the heartbeats are coming in but no updates are, the feed is up to date.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) update_seq does not always
reflect the seq of the latest document update
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181968#comment-13181968 ]
Robert Newson commented on COUCHDB-1367:
----------------------------------------
FYI: I've fixed couchdb-lucene. Instead of using "update_seq" from GET /dbname I instead grab "last_seq" from GET /dbname/_changes?limit=0&descending=true.
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172432#comment-13172432 ]
Paul Joseph Davis commented on COUCHDB-1367:
--------------------------------------------
Quite right. update_seq will be incremented for a number of API's that make changes while not updating documents. There's even an increment_update_seq API that just bumps it directly.
Having a "last_update" in the db info blob that lists the last update for managing changes feeds might be useful in general and seems like it'd be an easy enough addition.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Dave Cottlehuber (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173539#comment-13173539 ]
Dave Cottlehuber commented on COUCHDB-1367:
-------------------------------------------
+1 on "last_update" being consistent across replication, db, changes. I can't see anywhere else this is exposed via API. Now where to document that?
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1367) update_seq does not always reflect
the seq of the latest document update
Posted by "Robert Newson (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson updated COUCHDB-1367:
-----------------------------------
Fix Version/s: (was: 1.1.2)
(was: 1.3)
(was: 1.2)
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Jason Smith (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175877#comment-13175877 ]
Jason Smith commented on COUCHDB-1367:
--------------------------------------
Wait a second. Robert, you are not fixing a bug in C-L, you are working around a deficiency in CouchDB.
The only way to know the latest sequence id is to make a complete _changes query. Next, follow that up with a continuous feed if you want to keep the state fresh.
That is a paper cut.
What if I want to see the most recent five changes? What if there are a hundred million documents? What if 99% of the time, update_seq equals last_seq and so developers assume it means something it doesn't?
Everybody wants to know the id of the latest change. Nobody wants to know the "update sequence," whatever that is. If CouchDB can cache last_seq in the header and provide it in the DB response, that would be fantastic. Kindly reopen this ticket, then. Thanks!
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172245#comment-13172245 ]
Bob Dionne commented on COUCHDB-1367:
-------------------------------------
Henrik,
Thanks for the report. We discussed this a bit on irc this morning. So last_seq in the changes feed and update_seq in the db info are not intended to be the same, or at least there's some confusion about the semantics. couchdb-lucene uses continuous changes feeds so it doesn't have access to the last_seq value of a normal changes feed. When update_seq changes due to a call to set_revs_limit it gets out of whack.
In any event the solution may be to simply add last_seq to the db_info record. It shouldn't be hard to fix and as you say this is an edge case. I'm curious are you setting the revs_limit a lot? If so what's the use case?
Bob
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Henrik Hofmeister (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177733#comment-13177733 ]
Henrik Hofmeister commented on COUCHDB-1367:
--------------------------------------------
Although its hard not to agree with Robert on c-l - i still can't really see a use-case for the update_seq value - specifically why one would ever want to know that revs_limit etc. has been updated - without being able to know what has been updated. Just "Something has been done - 5 times" ... ?
I'd say for the sake of expected output - c-l aside - this is still a bug / "hidden feature" - and should at the very least be documented ?
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175219#comment-13175219 ]
Bob Dionne commented on COUCHDB-1367:
-------------------------------------
I agree, it would appear redundant to the user and confusing as it would most likely always be the same. I like the heartbeat solution, especially because it means we don't need to fix anything in couchdb
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173086#comment-13173086 ]
Bob Dionne commented on COUCHDB-1367:
-------------------------------------
I'm a little confused then. It seems if c-l has access to the last_seq then it should be able to determine if there are no changes of interest. I thought the commentary on IRC was that this was not the case because it uses the continuous feed.
Adding last_seq to the db info request will certainly solve it, though look a little redundant to the user as it will be almost always be equal to update_seq
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175368#comment-13175368 ]
Robert Newson commented on COUCHDB-1367:
----------------------------------------
The motivation for c-l's current approach is for indexes to be fresh at all times as well as updating the lucene indexes optimally (in modest sized batches). If it were purely driven by user queries, it would be variably stale, and variably well optimized, like couchdb's indexes. I wanted better for my child.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (COUCHDB-1367) update_seq does not always reflect
the seq of the latest document update
Posted by "Robert Newson (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson resolved COUCHDB-1367.
------------------------------------
Resolution: Fixed
Fix Version/s: 1.1.2
1.3
1.2
Added a clause for empty list for builtin_stats which fixes the function_clause.
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
> Fix For: 1.2, 1.3, 1.1.2
>
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Henrik Hofmeister (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173289#comment-13173289 ]
Henrik Hofmeister commented on COUCHDB-1367:
--------------------------------------------
What i'm puzzled about - is what would i ever need the update_seq for ? It allows me to - see that there has been made a change - however in the changes view it shows me that there are no changes? Only in the cases where it differs for last_seq of course - but what could i ever possibly use that number for? That is - a number - signalling that i have either updated revs_limit or a random other number of internal api calls ? Its absolutly useless - especially while i have no way of getting to know whats changed.
update_seq would - in any possible case - be expected by the user to reflect your core feature - the changes feed?
Not making it into a huge problem - but the only real fix for a production env. product like couchdb is to not add to the confusion - but fix the confusion (like not adding another number to the db info page) . That would give you 2 numbers - one that is useless (update_seq) and one that is the one you'd expect (last_seq). ?
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Bob Dionne (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bob Dionne reassigned COUCHDB-1367:
-----------------------------------
Assignee: Bob Dionne
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172401#comment-13172401 ]
Bob Dionne commented on COUCHDB-1367:
-------------------------------------
sure, I think the confusion is this (from the WIKI):
last_seq is the sequence number of the last update returned. (Currently it will always be the same as the seq of the last item in results.)
It doesn't say that this is identical to update_seq. They are two different things, or at least became that way when other features such as set_revs_limit were added. It might be better for now in your app to use last_seq if you can
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Henrik Hofmeister (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172296#comment-13172296 ]
Henrik Hofmeister commented on COUCHDB-1367:
--------------------------------------------
No im not - i just made a script ensuring it gets set on all dbs - to save a request i did the "just set it to 1" approach - assuming it wouldn't matter if i set it to 1 several times. This is how i discovered there was such a problem. We had been getting it randomly before - without ever realizing what the problem was exactly.
Its true that this specific case is for couchdb-lucene - however the general use case of being able to predict how far you're away from being up to date is not couchdb-lucene specific - i've for one created another in-house application that does exactly this - while performing a chained map-reduce like operation (What im saying is - if you want to reap the benefits of the changes feed and be aware of your progress - you'll need the right number)
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Paul Joseph Davis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175813#comment-13175813 ]
Paul Joseph Davis commented on COUCHDB-1367:
--------------------------------------------
@Randall
>If there's no technical reason why we need to bump the seq for changes that don't modify a document we could just stop doing that. What operations do this currently? How many of these resources have we at one time or another discussed making into a full document?
I think _purge, _revs_limit, and _security. Maybe other things. Not sure about the strictly confirming to theoretical model for _revs_limit and _security, but _purge has obvious semantics that need to be updated in views. Though as I think about this, I'm fairly certain we have a bug in the indexer that no one has ever reported. Granted, that seems to be fairly true of things that touch _purge code.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) update_seq does not always
reflect the seq of the latest document update
Posted by "Henrik Hofmeister (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177817#comment-13177817 ]
Henrik Hofmeister commented on COUCHDB-1367:
--------------------------------------------
@Robert - C-L aside - its still strange that its just some 'random' number of actions taken which is not really usable to the end user. That C-L could use a different implementation is besides the point imo - Its still is usable to get the current latest update seq - for whatever reason - if even to just show a progress bar in a to-be-developed feature in some gui application or whatever.
We use it internally to track progress of an aggragation application.
@Randall: in terms of adding a field or whatever - my only input is - less is more - update_seq makes perfect sense naming wise, it should just be the expected value. If you'll need to track other changes than doc changes later on in terms of replication or whatever - my uneducated guess is you're gonna need to do alot more than just have a number increase in any case. But thats just me :) http://c2.com/xp/YouArentGonnaNeedIt.html
Thanks
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175136#comment-13175136 ]
Randall Leeds commented on COUCHDB-1367:
----------------------------------------
I'm confused about why a client listening to continuous _changes should care whether or not update_seq changes without emitting a document modification. I like to think that in the future other sorts of changes might be allowed to surface in that feed. Is there any place we guarantee that seq in the changes feed should be monotonic? I don't think so. It seems to me like this is not a problem.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Jason Smith (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175286#comment-13175286 ]
Jason Smith commented on COUCHDB-1367:
--------------------------------------
Special-case changes response remind me of the discussion local docs (I think WRT _security). It seems worth considering making couch have *fewer* special cases as it gains features.
Could the revs limit be set in a _local/* document, which would have standard MVCC semantics (but they don't replicate)? Clients can examine and configure databases with their normal document manipulation functions, communicating with Couch through documents.
The list of things that arguably belong in _local/ grows. The security object, and apparently now the revs limit value can still be stored in the file header, but that is only a cache. (Couch might even expose the legacy API and internally convert it to document updates.)
Is it possible?
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) update_seq does not always
reflect the seq of the latest document update
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177798#comment-13177798 ]
Robert Newson commented on COUCHDB-1367:
----------------------------------------
I still don't see why we need do anything. My early mistaken understanding of this value should not be used as motivation here. At the time, I was not a "couchdb committer" so the earlier implication that it escaped even my awesome knowledge is to impute omniscience where none is warranted.
I haven't yet fixed couchdb-lucene as the update model is rather clumsy. Simply calling _changes?since=N instead of comparing update_seq with a local value will radically simplify that piece of couchdb-lucene. It should improve the internals of couchdb-lucene so significantly that I would rather *not* fix update_seq to work the way I expected years ago, in case it misleads someone into making the same mistake I made.
That said, I wouldn't veto the change, but C-L will not depend on either the current or any future meaning of update_seq in the next release.
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175143#comment-13175143 ]
Robert Newson commented on COUCHDB-1367:
----------------------------------------
I would hope it's obvious that update_seq must be monotonously incrementing (i.e, it cannot go down).
You're right in the strict sense here, readers of the changes feed will get a row for each document change, and nothing else. The subtle point I think you've missed in that some applications want to know if they've read all changes up to the current update sequence of the database. non stale=ok view queries already do this, and couchdb-lucene does too. It turns out that *only* the view engine can do it correctly in all cases because it knows the last sequence value that affected a document (that is, it doesn't 'see' the change to _security, and thus doesn't block for that change).
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1367) update_seq does not always reflect
the seq of the latest document update
Posted by "Randall Leeds (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds updated COUCHDB-1367:
-----------------------------------
Description: Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency. (was: If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended. )
Summary: update_seq does not always reflect the seq of the latest document update (was: When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes)
Updated the description and title to reflect the problem in general.
Proposals so far:
1. Add a new header field
a. to track the highest value in the by_seq index
b. to track header updates that do not affect by_seq, causing update_seq to behave in a manner more consistent with expectation
2. Migrate the non-replicable metadata into the document API and hang it within the by_seq index
As far as I can tell I'm the only proponent of (2). Proposal (2) is broader in scope, more difficult to implement, and fails to account for the possibility that other, current or future, database header updates may not fit into the document model. Therefore, I'll formally retract my suggestion that it be pursued as a solution to the present ticket.
Resuming discussion back here (sorry if it was unnecessary or confusing that I migrated it to dev@), how does the community feel about (1a) vs (1b)? I'm in favor of 1b, myself.
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175228#comment-13175228 ]
Randall Leeds commented on COUCHDB-1367:
----------------------------------------
I suggested the heartbeat because we could make it look like last_seq, but we don't even need to use heartbeat. We could emit a change that doesn't have a corresponding id and revisions. Although, since the URL is /db/_revs_limit, we could (to use that example) emit something like:
{"seq":X,"id":"_revs_limit","changes":[]}
I have no idea how badly that confuse existing clients, including CouchDB. Putting it on the db info is the least obtrusive from an API standpoint. From a code internals, I think everything would require some change (with the exception of doing nothing about this). I'm going to step away for now, but if you need any more color swatches I can send over some more samples.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (COUCHDB-1367) When settings revs_limit on db - the
db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds closed COUCHDB-1367.
----------------------------------
Resolution: Not A Problem
Assignee: (was: Bob Dionne)
Closing as "Not A Problem" as per Robert Newson's last comment. If this is incorrect, please re-open with a compelling argument to change this behavior.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175892#comment-13175892 ]
Randall Leeds commented on COUCHDB-1367:
----------------------------------------
> Wait a second. Robert, you are not fixing a bug in C-L, you are working around a deficiency in CouchDB.
Can't both be true?
> The only way to know the latest sequence id is to make a complete _changes query. Next, follow that up with a continuous feed if you want to keep the state fresh.
Nope. You can not ever know. You always know the latest sequence number at some arbitrarily recent point in time. If the (possibly continuous) changes feed says you're at X and you haven't heard anything more yet, then the database is at update sequence >= X. Neveretheless, I think I follow the sentiment. If last_seq were there one could know from an info request whether or not more changes should be available.
> What if I want to see the most recent five changes? What if there are a hundred million documents? What if 99% of the time, update_seq equals last_seq and so developers assume it means something it doesn't?
In order:
* /_changes?descending=true&limit=5
* Not sure how this is relevant
* This does indeed seem to cause some confusion. It clearly surprised Robert and Henrik and it's the first I've heard of this discrepancy.
I submit that this bug is closed accurately with a suggestion to move proposals to improve the situation over to the dev list. Off the top of my head a partial list of suggestions goes something like:
* Add additional information to the changes feed, perhaps with a query parameter (almost the reverse of include docs)
* Stop incrementing the update sequence on certain kinds of non-document changes
* Add more information to the db information response
Please take it from there and we can work through a proposal. Thanks, everyone.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Randall Leeds reopened COUCHDB-1367:
------------------------------------
Re-opening in light of recent dev@ discussion. It appears action will be taken here. Thanks, Henrik.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175262#comment-13175262 ]
Randall Leeds commented on COUCHDB-1367:
----------------------------------------
For _revs_limit we could always surface a change to the document with id "_revs_limit", which is _somewhat_ accurate (though it's not a full doc with revisions). Similar thoughts apply for _security.
If there's no technical reason why we need to bump the seq for changes that don't modify a document we could just stop doing that. What operations do this currently? How many of these resources have we at one time or another discussed making into a full document?
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175168#comment-13175168 ]
Robert Newson commented on COUCHDB-1367:
----------------------------------------
Correct, 'monotonically incrementing' means that the number always goes up but does *not* imply that it always goes up by exactly 1. Because it sorta kind *sounds* like that, I clarified with 'never goes down' and tried a synonym for monotonic. The changes feed will certainly have gaps, but no row N will have a lower update_seq than any previously seen row.
monotonic |ˌmänəˈtänik|
adjective
1 Mathematics (of a function or quantity) varying in such a way that it either never decreases or never increases.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (COUCHDB-1367) update_seq does not always reflect
the seq of the latest document update
Posted by "Robert Newson (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson updated COUCHDB-1367:
-----------------------------------
Comment: was deleted
(was: Added a clause for empty list for builtin_stats which fixes the function_clause.)
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Randall Leeds (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175160#comment-13175160 ]
Randall Leeds commented on COUCHDB-1367:
----------------------------------------
Oh, I messed up. Monotonic does not mean what I think it means. I meant it need not increase by intervals of 1 all the time.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Bob Dionne (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175367#comment-13175367 ]
Bob Dionne commented on COUCHDB-1367:
-------------------------------------
wonderful, I wondering about this as this is precisely how I keep up to date in my native indexer, using _changes?since and storing the last_seq in a checkpoint.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Jason Smith (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175236#comment-13175236 ]
Jason Smith commented on COUCHDB-1367:
--------------------------------------
I disagree with Randal's gut. The metadata from a /db response is not confusing anybody. Most people poke around for values they care about and ignore the rest. That "committed_update_seq" doesn't seem to bother anybody hopefully demonstrates this point.
On the other hand, a new type of change object has more impact. For example, if I add ?include_docs=true, what would the .doc field be, if anything?
So, in other words, IMO "last_seq" in the DB response is perfect. People have seen it in their _changes queries and they will make the right assumptions about its meanings.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (COUCHDB-1367) update_seq does not always reflect
the seq of the latest document update
Posted by "Robert Newson (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Newson reopened COUCHDB-1367:
------------------------------------
resolved in error
> update_seq does not always reflect the seq of the latest document update
> ------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Priority: Minor
> Labels: revs_limit
>
> Certain operations, (currently _revs_limit and _security changes) cause the database header's update_seq to increase when the by_seq index (and therefore _changes) has not changed, which is confusing in light of the naming consistency.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1367) When settings revs_limit on db -
the db increases its update_seq counter when viewing stats - but not when
getting changes
Posted by "Robert Newson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172791#comment-13172791 ]
Robert Newson commented on COUCHDB-1367:
----------------------------------------
Oh, ye of little faith. couchdb-lucene certainly can handle the last_seq output (and has since approximately forever). It's just so rarely encountered that it doesn't help with respect to the issue at hand.
> When settings revs_limit on db - the db increases its update_seq counter when viewing stats - but not when getting changes
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: COUCHDB-1367
> URL: https://issues.apache.org/jira/browse/COUCHDB-1367
> Project: CouchDB
> Issue Type: Bug
> Components: HTTP Interface
> Affects Versions: 1.1.1
> Environment: Any
> Reporter: Henrik Hofmeister
> Assignee: Bob Dionne
> Priority: Minor
> Labels: revs_limit
>
> If you put a number to _revs_limit on a db (to update it) - the http://host/dbname/ info document gets an increase in update_seq number - however the changes feed does not contain this change (while its not a change). This causes the update_seq in the dbinfo doc and the last seq in the changes feed to differ - which breaks any application depending on the update_seq number as the expected sequence size of the db (in my case - couchdb-lucene that will only respond to stale requests because it thinks its not up to date)
> I know this is an edge case - but still its something fairly fundamental - that clearly is not working as intended.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira