You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by "lpcy (via GitHub)" <gi...@apache.org> on 2024/06/16 10:39:08 UTC
[I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
lpcy opened a new issue, #5090:
URL: https://github.com/apache/couchdb/issues/5090
[NOTE]: # ( ^^ Provide a general summary of the issue in the title above. ^^ )
First, execute the DELETE command to delete the document. Then immediately execute _find, and the deleted document appears in the result. But when we execute the _find later, it disappears.
## Expected Behaviour
After the document is deleted, it immediately disappears from the return result of _find.
## Your Environment
* CouchDB version used: 3.3.3
* Browser name and version: chrome 126
* Operating system and version: windows 10
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2172755446
Ah, that makes sense, thank you for clarifying.
I you don't need to keep deleted documents, as you never replicate, you can use the [purge](https://docs.couchdb.org/en/stable/api/database/misc.html) endpoint. The main downside to keeping them is the disk space they will continue to occupy. This is quite small (assuming you used the DELETE method which also empties the document body) but it is not zero.
Alternative strategies;
1) if your data is temporal/time-based, you could make a database for distinct time periods (say, monthly), and when your oldest database contains only deleted documents you simply delete the entire database.
2) periodically replicate the database to a new database but with a filter that rejects deleted documents, then switch usage to the new database.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "lpcy (via GitHub)" <gi...@apache.org>.
lpcy commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2172855644
> Ah, that makes sense, thank you for clarifying.
>
> If you don't need to keep deleted documents, as you never replicate, you can use the [purge](https://docs.couchdb.org/en/stable/api/database/misc.html) endpoint. The main downside to keeping them is the disk space they will continue to occupy. This is quite small (assuming you used the DELETE method which also empties the document body) but it is not zero.
>
> Alternative strategies;
>
> 1. if your data is temporal/time-based, you could make a database for distinct time periods (say, monthly), and when your oldest database contains only deleted documents you simply delete the entire database.
> 2. periodically replicate the database to a new database but with a filter that rejects deleted documents, then switch usage to the new database.
Thank you again for your answer, I've got it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2172500693
"Tombstone" is a loose term, more precisely it is a document with the deleted flag set to true, and may contain other data. They are preserved forever, just as non-deleted documents are, to ensure that replication works correctly. You can replicate with a filter to drop them (or any other subset of documents) as long as you're aware of that consequence.
Delete is not asynchronous (any more than doc create or update is), I'm referring to the way we only wait for the first 2 of the total 3 responses in a 3 or more node cluster, which seems not to apply in your case.
If this is a single node setup then your opening comment is a bit more interesting. when the DELETE response is returned the document has been marked as deleted, and so any subsequent request should reflect that, including indexes (`_view`, `_find`, etc). Are you querying with stale=ok or update=false parameters? Assuming not, how long is the delay between the deleted document appearing in results after deletion and it finally being gone?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "rnewson (via GitHub)" <gi...@apache.org>.
rnewson commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2171770016
This is possible if N>1 (i.e, you have a cluster, not a standalone single node) for a period of time. Once the DELETE has happened at all N nodes subsequent queries (assuming you didn't specify update=false or stale=ok) will not return that document. There can be a period where a DELETE is completed (i.e, you get a 200 OK response) but one or more nodes have not yet processed it, a _find at that time might get a response from one of those nodes (all queries are inherently R=1, they read just one of the copies).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "lpcy (via GitHub)" <gi...@apache.org>.
lpcy closed issue #5090: Why do deleted documents appear in the return result of executing the _find command immediately?
URL: https://github.com/apache/couchdb/issues/5090
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "lpcy (via GitHub)" <gi...@apache.org>.
lpcy commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2172113263
> This is possible if N>1 (i.e, you have a cluster, not a standalone single node) for a period of time. Once the DELETE has happened at all N nodes subsequent queries (assuming you didn't specify update=false or stale=ok) will not return that document. There can be a period where a DELETE is completed (i.e, you get a 200 OK response) but one or more nodes have not yet processed it, a _find at that time might get a response from one of those nodes (all queries are inherently R=1, they read just one of the copies).
Thank you, I'm sorry but I realized that I made an error: delete in the code is asynchronous.
At the same time, there is a new question: Is the tombstone information always retained? Is there any way to clean it up? It seems that Google's method is to synchronize to a new database while excluding deleted documents, which seems cumbersome. Currently, I am using a single node.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "lpcy (via GitHub)" <gi...@apache.org>.
lpcy commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2172115008
> Do you have a minimal working example, like a script to reproduce this?
I rewrote the script and found it to be working properly. Sorry, it's my problem: delete is asynchronous.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "lpcy (via GitHub)" <gi...@apache.org>.
lpcy commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2172716154
> "Tombstone" is a loose term, more precisely it is a document with the deleted flag set to true, and may contain other data. They are preserved forever, just as non-deleted documents are, to ensure that replication works correctly. You can replicate with a filter to drop them (or any other subset of documents) as long as you're aware of that consequence.
>
> Delete is not asynchronous (any more than doc create or update is), I'm referring to the way we only wait for the first 2 of the total 3 responses in a 3 or more node cluster, which seems not to apply in your case.
>
> If this is a single node setup then your opening comment is a bit more interesting. when the DELETE response is returned the document has been marked as deleted, and so any subsequent request should reflect that, including indexes (`_view`, `_find`, etc). Are you querying with stale=ok or update=false parameters? Assuming not, how long is the delay between the deleted document appearing in results after deletion and it finally being gone?
Thank you for your patient answer. Actually, what I meant was that the reason for my original question was that I used DELETE in the asynchronous environment of JavaScript, which caused _find to retrieve old data at the same time. This issue can be ignored.
The "Tombstone" you mentioned is for replication, but if I don't have replication requirements, is there a simple command to clear them? Will not cleaning them have an impact on database performance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Why do deleted documents appear in the return result of executing the _find command immediately? [couchdb]
Posted by "big-r81 (via GitHub)" <gi...@apache.org>.
big-r81 commented on issue #5090:
URL: https://github.com/apache/couchdb/issues/5090#issuecomment-2171745244
Do you have a minimal working example, like a script to reproduce this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@couchdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org