You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "James Howe (JIRA)" <ji...@apache.org> on 2011/08/17 11:38:27 UTC
[jira] [Created] (COUCHDB-1252) A way to have views return _deleted
documents
A way to have views return _deleted documents
---------------------------------------------
Key: COUCHDB-1252
URL: https://issues.apache.org/jira/browse/COUCHDB-1252
Project: CouchDB
Issue Type: New Feature
Components: JavaScript View Server
Affects Versions: 1.0.3, 1.1
Reporter: James Howe
Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (COUCHDB-1252) A way to have views return _deleted documents
Posted by Paul Davis <pa...@gmail.com>.
Kinda sorta. Except for how they're different.
On Aug 19, 2011, at 3:01 AM, Jason Smith wrote:
> Cross-post to user@, why CouchDB is like Git.
>
> On Fri, Aug 19, 2011 at 10:26 AM, Paul Joseph Davis (JIRA)
> <ji...@apache.org> wrote:
>>
>> [ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087489#comment-13087489 ]
>>
>> Paul Joseph Davis commented on COUCHDB-1252:
>> --------------------------------------------
>>
>> @Jason,
>>
>> A couple weeks ago I would have agreed completely with you. So much in fact that when someone reported a bug that _bulk_docs was storing the body of docs submitted with _deleted: true set, it was immediately assumed to be a bug and was promptly fixed.
>>
>> Until Damien said "That was on purpose so people can say why something was deleted."
>>
>> In reality, the only "special" sauce for deleted docs is that they're not considered conflicts. Other than that, you can still create a doc based on the revision of a deleted doc. In fact you can rescue an edit history from any known revision. But I digress.
>
> Oh, so you are saying revisions are like Git?
>
> Cool. That makes it easy to understand.
>
> --
> Iris Couch
Re: [jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by Jason Smith <jh...@iriscouch.com>.
Cross-post to user@, why CouchDB is like Git.
On Fri, Aug 19, 2011 at 10:26 AM, Paul Joseph Davis (JIRA)
<ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087489#comment-13087489 ]
>
> Paul Joseph Davis commented on COUCHDB-1252:
> --------------------------------------------
>
> @Jason,
>
> A couple weeks ago I would have agreed completely with you. So much in fact that when someone reported a bug that _bulk_docs was storing the body of docs submitted with _deleted: true set, it was immediately assumed to be a bug and was promptly fixed.
>
> Until Damien said "That was on purpose so people can say why something was deleted."
>
> In reality, the only "special" sauce for deleted docs is that they're not considered conflicts. Other than that, you can still create a doc based on the revision of a deleted doc. In fact you can rescue an edit history from any known revision. But I digress.
Oh, so you are saying revisions are like Git?
Cool. That makes it easy to understand.
--
Iris Couch
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086395#comment-13086395 ]
Paul Joseph Davis commented on COUCHDB-1252:
--------------------------------------------
We could fairly easily add a include_deleted option to the view options parameter the same we do for include_design.
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "Paul Joseph Davis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087489#comment-13087489 ]
Paul Joseph Davis commented on COUCHDB-1252:
--------------------------------------------
@Jason,
A couple weeks ago I would have agreed completely with you. So much in fact that when someone reported a bug that _bulk_docs was storing the body of docs submitted with _deleted: true set, it was immediately assumed to be a bug and was promptly fixed.
Until Damien said "That was on purpose so people can say why something was deleted."
In reality, the only "special" sauce for deleted docs is that they're not considered conflicts. Other than that, you can still create a doc based on the revision of a deleted doc. In fact you can rescue an edit history from any known revision. But I digress.
Either way, "auditing information about deleted docs" is apparently precisely what deleted doc bodies were for, so I don't see a reason to not have an option to see them.
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "James Howe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097067#comment-13097067 ]
James Howe commented on COUCHDB-1252:
-------------------------------------
ETA on this patch Jason?
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "James Howe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107698#comment-13107698 ]
James Howe commented on COUCHDB-1252:
-------------------------------------
Time's up :p
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087597#comment-13087597 ]
Robert Newson commented on COUCHDB-1252:
----------------------------------------
Same level as include_design, which is per ddoc.
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Created] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by Benoit Chesneau <bc...@gmail.com>.
On Friday, August 19, 2011, Paul Davis <pa...@gmail.com> wrote:
> I don't think its an issue that the 404 is there. It *is* deleted after
all. Though there's an argument to be made why we don't show the deleted
body in the response.
>
> I would say that it should be a view option, but this will get a bit wonky
for you. You'll need to see if *any* of the views have it included and then
read the body from disk in the enum_changes_since loop. After that you'll
have to filter it before passing it to the doc mapper. Also, I'm currently
in the middle of rewriting the entire view engine so I wouldn't go too far
if you end up needing to change lots of code.
>
> Also, you can't see them in a view because we've never had the request to
write this patch. It wasn't *too* long ago that it was impossible to see
_design docs in a view (and now that's even off by default).
>
> On Aug 18, 2011, at 11:07 PM, Jason Smith (JIRA) wrote:
>
>>
>> [
https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087509#comment-13087509]
>>
>> Jason Smith commented on COUCHDB-1252:
>> --------------------------------------
>>
>> To Damien, I say: well, why can't I see deleted docs in a view? Why do I
get a 404 when I query for them? Because they are deleted. Damien's code
contradicts Damien's words, and the code makes better sense. To expose
_deleted documents everywhere is a leaky abstraction.
>>
>> However, in the spirit of "design by committee is bad" and to learn a
different part of CouchDB, I would like to write this patch. Paul, should it
be a design option or a view option?
>>
>> For example:
>>
>> { "_id": "_design/example"
>> , "options": { "these_are":"options global to the ddoc",
"local_seq":true, "include_deleted":true}
>> , "views":
>> { "deleted_docs_A":
>> { "options": {"these_are":"local to this view", "include_design":true,
"include_deleted":true}
>> , "map":"function(D) { if(D._deleted) emit(D._id, 1) }"
>> }
>> , "deleted_docs_B":
>> { "options": {"not using include_deleted": true}
>> , "map":"function(D) { if(D._deleted) emit(D._id, 1) /* Does it emit?
*/ }
>> }
>> }
>> }
>>
>> With design options, deleted_docs_B will emit, but using view options, it
won't. I prefer it to be per-view. In fact I did not even realize design
options existed until I checked the source.
>>
>> Thoughts? Thanks.
>>
>>> A way to have views return _deleted documents
>>> ---------------------------------------------
>>>
>>> Key: COUCHDB-1252
>>> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
>>> Project: CouchDB
>>> Issue Type: New Feature
>>> Components: JavaScript View Server
>>> Affects Versions: 1.1, 1.0.3
>>> Reporter: James Howe
>>>
>>> Given that documents can be 'soft' deleted / deleted with auditing data
by updating the document to include the _deleted property, it would be
incredibly useful if there were a way to access these documents in a map
function. Otherwise it is very difficult to find the auditing data - even
more so if the original ids are unknown.
>>> I was thinking along the lines of a view query parameter
'include_deleted', but don't really mind how this is implemented, as long as
it is there.
>>
>> --
>> This message is automatically generated by JIRA.
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>
>
maybe we should return a 410 instead of a 404 on _deleted but i'm n,ot sure
it really worth it. 404 czn be a temporary response according the spec.
Re: [jira] [Commented] (COUCHDB-1252) A way to have views return _deleted documents
Posted by Paul Davis <pa...@gmail.com>.
I don't think its an issue that the 404 is there. It *is* deleted after all. Though there's an argument to be made why we don't show the deleted body in the response.
I would say that it should be a view option, but this will get a bit wonky for you. You'll need to see if *any* of the views have it included and then read the body from disk in the enum_changes_since loop. After that you'll have to filter it before passing it to the doc mapper. Also, I'm currently in the middle of rewriting the entire view engine so I wouldn't go too far if you end up needing to change lots of code.
Also, you can't see them in a view because we've never had the request to write this patch. It wasn't *too* long ago that it was impossible to see _design docs in a view (and now that's even off by default).
On Aug 18, 2011, at 11:07 PM, Jason Smith (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087509#comment-13087509 ]
>
> Jason Smith commented on COUCHDB-1252:
> --------------------------------------
>
> To Damien, I say: well, why can't I see deleted docs in a view? Why do I get a 404 when I query for them? Because they are deleted. Damien's code contradicts Damien's words, and the code makes better sense. To expose _deleted documents everywhere is a leaky abstraction.
>
> However, in the spirit of "design by committee is bad" and to learn a different part of CouchDB, I would like to write this patch. Paul, should it be a design option or a view option?
>
> For example:
>
> { "_id": "_design/example"
> , "options": { "these_are":"options global to the ddoc", "local_seq":true, "include_deleted":true}
> , "views":
> { "deleted_docs_A":
> { "options": {"these_are":"local to this view", "include_design":true, "include_deleted":true}
> , "map":"function(D) { if(D._deleted) emit(D._id, 1) }"
> }
> , "deleted_docs_B":
> { "options": {"not using include_deleted": true}
> , "map":"function(D) { if(D._deleted) emit(D._id, 1) /* Does it emit? */ }
> }
> }
> }
>
> With design options, deleted_docs_B will emit, but using view options, it won't. I prefer it to be per-view. In fact I did not even realize design options existed until I checked the source.
>
> Thoughts? Thanks.
>
>> A way to have views return _deleted documents
>> ---------------------------------------------
>>
>> Key: COUCHDB-1252
>> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
>> Project: CouchDB
>> Issue Type: New Feature
>> Components: JavaScript View Server
>> Affects Versions: 1.1, 1.0.3
>> Reporter: James Howe
>>
>> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
>> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "Jason Smith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087509#comment-13087509 ]
Jason Smith commented on COUCHDB-1252:
--------------------------------------
To Damien, I say: well, why can't I see deleted docs in a view? Why do I get a 404 when I query for them? Because they are deleted. Damien's code contradicts Damien's words, and the code makes better sense. To expose _deleted documents everywhere is a leaky abstraction.
However, in the spirit of "design by committee is bad" and to learn a different part of CouchDB, I would like to write this patch. Paul, should it be a design option or a view option?
For example:
{ "_id": "_design/example"
, "options": { "these_are":"options global to the ddoc", "local_seq":true, "include_deleted":true}
, "views":
{ "deleted_docs_A":
{ "options": {"these_are":"local to this view", "include_design":true, "include_deleted":true}
, "map":"function(D) { if(D._deleted) emit(D._id, 1) }"
}
, "deleted_docs_B":
{ "options": {"not using include_deleted": true}
, "map":"function(D) { if(D._deleted) emit(D._id, 1) /* Does it emit? */ }
}
}
}
With design options, deleted_docs_B will emit, but using view options, it won't. I prefer it to be per-view. In fact I did not even realize design options existed until I checked the source.
Thoughts? Thanks.
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "Jason Smith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087485#comment-13087485 ]
Jason Smith commented on COUCHDB-1252:
--------------------------------------
I propose this issue be closed as NOFIX (or whatever it's called). Allowing include_deleted would confuse the concept of "deleted" in the way that calling them "revisions" implies that CouchDB revisions are great for a wiki.
James, your requirements are brilliant. That sounds like a great app. But IMO if you need data to remain indefinitely (for audit), then you should not be deleting documents and then exploiting the fact that CouchDB keeps deleted documents for revision tracking. In your application, "deleted" means "invisible to users but the auditors still want to know about it" so that is an application-level detail. That is exactly what views are for.
For an application, _deleted means it's dead. Advanced tools like the replicator must follow the doc into the afterlife but that's not an application's concern.
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (COUCHDB-1252) A way to have views return
_deleted documents
Posted by "Jason Smith (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097657#comment-13097657 ]
Jason Smith commented on COUCHDB-1252:
--------------------------------------
Hi, James. There is an Iris Couch sprint starting Thursday September 8, running for one week. I expect to complete it during the sprint.
> A way to have views return _deleted documents
> ---------------------------------------------
>
> Key: COUCHDB-1252
> URL: https://issues.apache.org/jira/browse/COUCHDB-1252
> Project: CouchDB
> Issue Type: New Feature
> Components: JavaScript View Server
> Affects Versions: 1.1, 1.0.3
> Reporter: James Howe
>
> Given that documents can be 'soft' deleted / deleted with auditing data by updating the document to include the _deleted property, it would be incredibly useful if there were a way to access these documents in a map function. Otherwise it is very difficult to find the auditing data - even more so if the original ids are unknown.
> I was thinking along the lines of a view query parameter 'include_deleted', but don't really mind how this is implemented, as long as it is there.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira