You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "nickva (via GitHub)" <gi...@apache.org> on 2026/03/09 18:29:33 UTC

[I] [Bug]: Dreyfus purge error [couchdb]

nickva opened a new issue, #5916:
URL: https://github.com/apache/couchdb/issues/5916

   ### Version
   
   unknown
   
   ### Describe the problem you're encountering
   
   Purging a dreyfus (clouseau) text index when  the min purge infos limit is > 0 and then querying is broken and raises  this kind of an error:
   
   ```
   Querying search index
   {
     "error": "{invalid_start_purge_seq,0,3}",
     "reason": "[{couch_bt_engine,fold_purge_infos,5,\n                  [{file,\"src/couch_bt_engine.erl\"},{line,619}]},\n {dreyfus_index_updater,purge_index,3,\n                        [{file,\"src/dreyfus_index_updater.erl\"},{line,116}]},\n {dreyfus_index_updater,update,2,\n                        [{file,\"src/dreyfus_index_updater.erl\"},{line,52}]}]"
   }
   ```
   
   ### Expected Behaviour
   
   Dreyfus/clouseau indexes should handle purges even after they've been compacted and start at a value > 0
   
   ### Steps to Reproduce
   
   Purge greater than purge infos limit number of docs, then compact, then create a clouseau/dreyfus search index
   
   ### Your Environment
   
   _No response_
   
   ### Additional Context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: Dreyfus purge error [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on issue #5916:
URL: https://github.com/apache/couchdb/issues/5916#issuecomment-4025984684

   A third issue, slightly tangential to the above is dreyfus/clouseau index don't have a the reset functionality on purge seq inconsistency detection as we for main view indexes. Basically there is no equivalent to:
   
   https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/couch_index/src/couch_index_updater.erl#L218-L222


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: Dreyfus purge error [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on issue #5916:
URL: https://github.com/apache/couchdb/issues/5916#issuecomment-4159435128

   The second issue is fixed in https://github.com/cloudant-labs/clouseau/pull/146
   
   And we decided we won't add the reset functionality in for now.
   
   Closing this issue as fixed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: Dreyfus purge error [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on issue #5916:
URL: https://github.com/apache/couchdb/issues/5916#issuecomment-4025900026

   One of the causes is using this 0 as the starting/ending PurgeSeq in https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_index_updater.erl#L116-L118
   
   If there not purges present the fold will end up with a 0 and both set the new index purge sequence to 0 (which may not be true) and also set the purge checkpoint local doc to 0 as well. Next time the fold will start from 0 and that's will trigger the invalid_start_purge_seq since it looks like index had skipped a bunch of purges.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: Dreyfus purge error [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on issue #5916:
URL: https://github.com/apache/couchdb/issues/5916#issuecomment-4027070433

   Pr to fix the first issue https://github.com/apache/couchdb/pull/5918


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: Dreyfus purge error [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva closed issue #5916: [Bug]: Dreyfus purge error
URL: https://github.com/apache/couchdb/issues/5916


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug]: Dreyfus purge error [couchdb]

Posted by "nickva (via GitHub)" <gi...@apache.org>.
nickva commented on issue #5916:
URL: https://github.com/apache/couchdb/issues/5916#issuecomment-4025967432

   Another issue is how purge_seq is handled by clouseau/dreyfus indexes.
   
   When we initialize the purge checkpoint in https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_index.erl#L126 we also set the purge seq in the index:
   
   https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_util.erl#L352-L354
   
   That is done with the idea that later on in https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_index_updater.erl#L90-L91 we can read that purge_seq from the index and pass that as the initial start purge sequence. However setting a purge_seq on an index then immediately reading it back, we won't get the value we set. Until it commits we'll get the old committed value. That is incorrect behavior as far how purging logic is concerned because of this sequence of steps when the purge sequence is non-0 for a db.
   
   For example is db purge seq is 42, the minimum value may be 12
   
    1. When index is first created we set clouseau's purge seq = 42
    2. In purge_index https://github.com/apache/couchdb/blob/25e1269170e7d056d9c5242946224900eb5f759f/src/dreyfus/src/dreyfus_index_updater.erl#L91 we read it right back from the index but instead of 42 we'll get 0
    3. Folding purges starting from 0 will fail, because the minimum purge seq is 12, and to the purge checkpoint logic it looks like the index just skipped a bunch of purges and is now out of sync.
   
   Maybe instead we should use the local doc checkpoints to keep track of the index purge seq instead of using the index. We already have to track them in the checkpoints anyway. Or, we have to force the index to properly commit and a set_purge_seq checkpoint is initialized (I am not sure if we a synchronous commit on purge_seq set though...)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org