You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Paul Bonser <mi...@gmail.com> on 2008/07/23 07:36:24 UTC
[PATCH] Views: Multiple keys
After exploring the source code a bit and abandoning a few other ways of achieving this, I've managed to get multi-key requests working. The only way to make it more efficient would require some digging into the couch_btree folding code, I think.
You use it by POSTing a JSON array of keys to a view url. Any passed-in values for start_key and end_key are ignored.
I haven't written any unittests for this yet, but I wanted to see what everyone else thinks of this and ask if there's any obvious better way to accomplish the same goal.
This patch is against the latest in SVN as of now. Any comments or suggestions for this are very much welcome.
Signed-off-by: Paul Bonser <mi...@gmail.com>
--
diff --git a/src/couchdb/couch_httpd.erl b/src/couchdb/couch_httpd.erl
index af8d9b4..c813a4c 100644
--- a/src/couchdb/couch_httpd.erl
+++ b/src/couchdb/couch_httpd.erl
@@ -363,6 +363,37 @@ handle_db_request(Req, 'GET', {DbName, _Db, ["_view", DocId, ViewName]}) ->
end
end;
+% Multi-key request, with a JSON array of keys as the POST body
+handle_db_request(Req, 'POST', {DbName, _Db, ["_view", DocId, ViewName]}) ->
+ #view_query_args{
+ count = Count,
+ skip = SkipCount,
+ direction = Dir,
+ start_docid = StartDocId
+ } = QueryArgs = parse_view_query(Req),
+
+ case Req:get_primary_header_value("content-type") of
+ undefined -> ok;
+ "application/json" -> ok;
+ Else -> throw({incorrect_mime_type, Else})
+ end,
+ JsonKeys = tuple_to_list(cjson:decode(Req:recv_body())),
+
+ {ok, View} = couch_view:get_map_view({DbName, "_design/" ++ DocId, ViewName}),
+ {ok, RowCount} = couch_view:get_row_count(View),
+ FoldAccInit = {Count, SkipCount, undefined, []},
+ FoldResult = lists:foldl(fun(Key, {ok, FoldAcc}) ->
+ Start = {Key, StartDocId},
+ FoldlFun = make_view_fold_fun(Req, QueryArgs#view_query_args {
+ start_key = Key,
+ end_key = Key
+ },
+ RowCount, fun couch_view:reduce_to_count/1),
+ couch_view:fold(View, Start, Dir, FoldlFun, FoldAcc)
+ end, {ok, FoldAccInit}, JsonKeys),
+ finish_view_fold(Req, RowCount, FoldResult);
+
+
handle_db_request(_Req, _Method, {_DbName, _Db, ["_view", _DocId, _ViewName]}) ->
throw({method_not_allowed, "GET,HEAD"});
Re: [PATCH] Views: Multiple keys
Posted by Chris Anderson <jc...@grabb.it>.
On Wed, Aug 6, 2008 at 9:16 PM, Paul Bonser <mi...@gmail.com> wrote:
>
> I never looked into the temp view code, so I'll be of little help with
> that at the moment. Good luck on that fresh look.
I've updated the patch. The temp_view bug turned out to be
lack-of-sleep related, so it's working now, as well as multi-key
requests against reduce views (so long as group=true).
The new patch is available at
https://issues.apache.org/jira/browse/COUCHDB-101
Cheers!
--
Chris Anderson
http://jchris.mfdz.com
Re: [PATCH] Views: Multiple keys
Posted by Paul Bonser <mi...@gmail.com>.
On 8/6/08, Chris Anderson <jc...@grabb.it> wrote:
> Thanks for the patch Paul. I updated it against current trunk and
> added some unit tests to it. I took the opportunity to do some
> refactoring and create a output_map_view function. It's working well
> for design docs but for some reason I can't get it to behave correctly
> with _temp_views.
I'd been meaning to write some unit tests, but I've been really busy
lately with a full-time job and some contract work on the side. I'm
glad to hear that it works alright (except for the temp views).
> If anyone can apply it and try to figure out the temp-view stuff,
> that'd be super helpful. Otherwise maybe sleep and a fresh look will
> let me finish it.
I never looked into the temp view code, so I'll be of little help with
that at the moment. Good luck on that fresh look.
Hopefully I'll be able to contribute more once I'm not trying to do so
many things.
--
Paul Bonser
http://blog.paulbonser.com
Re: [PATCH] Views: Multiple keys
Posted by Chris Anderson <jc...@grabb.it>.
Thanks for the patch Paul. I updated it against current trunk and
added some unit tests to it. I took the opportunity to do some
refactoring and create a output_map_view function. It's working well
for design docs but for some reason I can't get it to behave correctly
with _temp_views.
The behavior is documented in the attached unit tests, but the gist of
it is that POSTs to /mydb/_view/design/view with a body like:
{"keys":[array of keys]}
return just the view rows that match the keys. There is still work to
be done to test the interactions with other request params. I started
down the path of getting reduce functions to work with keys as well,
but I think that will require getting deep into couch_btree.erl
The patch is here:
http://friendpaste.com/7Va7jgWK
If anyone can apply it and try to figure out the temp-view stuff,
that'd be super helpful. Otherwise maybe sleep and a fresh look will
let me finish it.
Chris
On Tue, Jul 22, 2008 at 10:36 PM, Paul Bonser <mi...@gmail.com> wrote:
> After exploring the source code a bit and abandoning a few other ways of achieving this, I've managed to get multi-key requests working. The only way to make it more efficient would require some digging into the couch_btree folding code, I think.
>
> You use it by POSTing a JSON array of keys to a view url. Any passed-in values for start_key and end_key are ignored.
>
> I haven't written any unittests for this yet, but I wanted to see what everyone else thinks of this and ask if there's any obvious better way to accomplish the same goal.
>
> This patch is against the latest in SVN as of now. Any comments or suggestions for this are very much welcome.
>
>
> Signed-off-by: Paul Bonser <mi...@gmail.com>
> --
>
> diff --git a/src/couchdb/couch_httpd.erl b/src/couchdb/couch_httpd.erl
> index af8d9b4..c813a4c 100644
> --- a/src/couchdb/couch_httpd.erl
> +++ b/src/couchdb/couch_httpd.erl
> @@ -363,6 +363,37 @@ handle_db_request(Req, 'GET', {DbName, _Db, ["_view", DocId, ViewName]}) ->
> end
> end;
>
> +% Multi-key request, with a JSON array of keys as the POST body
> +handle_db_request(Req, 'POST', {DbName, _Db, ["_view", DocId, ViewName]}) ->
> + #view_query_args{
> + count = Count,
> + skip = SkipCount,
> + direction = Dir,
> + start_docid = StartDocId
> + } = QueryArgs = parse_view_query(Req),
> +
> + case Req:get_primary_header_value("content-type") of
> + undefined -> ok;
> + "application/json" -> ok;
> + Else -> throw({incorrect_mime_type, Else})
> + end,
> + JsonKeys = tuple_to_list(cjson:decode(Req:recv_body())),
> +
> + {ok, View} = couch_view:get_map_view({DbName, "_design/" ++ DocId, ViewName}),
> + {ok, RowCount} = couch_view:get_row_count(View),
> + FoldAccInit = {Count, SkipCount, undefined, []},
> + FoldResult = lists:foldl(fun(Key, {ok, FoldAcc}) ->
> + Start = {Key, StartDocId},
> + FoldlFun = make_view_fold_fun(Req, QueryArgs#view_query_args {
> + start_key = Key,
> + end_key = Key
> + },
> + RowCount, fun couch_view:reduce_to_count/1),
> + couch_view:fold(View, Start, Dir, FoldlFun, FoldAcc)
> + end, {ok, FoldAccInit}, JsonKeys),
> + finish_view_fold(Req, RowCount, FoldResult);
> +
> +
> handle_db_request(_Req, _Method, {_DbName, _Db, ["_view", _DocId, _ViewName]}) ->
> throw({method_not_allowed, "GET,HEAD"});
>
>
--
Chris Anderson
http://jchris.mfdz.com
Re: [PATCH] Views: Multiple keys
Posted by Paul Davis <pa...@gmail.com>.
There was an idea floated on another thread to make _bulk_docs support
a post body of something like:
{
"put": [ { "_id": "doc1", "foo": "bar" } ],
"post": [ { "baz": "foo" } ],
"delete": [ {"_id": "doc2" } ],
"get": [ { "_id": "doc3" }, { "_id": "doc4"} ]
}
I looked briefly into this, but the code that runs bulk docs is a lot
deeper than I thought it was. There's quite a bit of stuff related to
replication and consistency.
Paul
On Wed, Jul 23, 2008 at 3:42 PM, Chris Anderson <jc...@grabb.it> wrote:
> On Wed, Jul 23, 2008 at 1:59 PM, Paul Bonser <mi...@gmail.com> wrote:
>> Does anyone have any thoughts on allowing multi-docid requests?
>
> I'm guessing they should live at a different endpoint url - as they
> aren't tied to a particular view. Maybe _load_bulk_docs or
> something... too bad about GET being infeasible due to URL length, it
> would make most sense to have a GET from _bulk_docs, but POST there is
> already in use.
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>
Re: [PATCH] Views: Multiple keys
Posted by Chris Anderson <jc...@grabb.it>.
On Wed, Jul 23, 2008 at 1:59 PM, Paul Bonser <mi...@gmail.com> wrote:
> Does anyone have any thoughts on allowing multi-docid requests?
I'm guessing they should live at a different endpoint url - as they
aren't tied to a particular view. Maybe _load_bulk_docs or
something... too bad about GET being infeasible due to URL length, it
would make most sense to have a GET from _bulk_docs, but POST there is
already in use.
--
Chris Anderson
http://jchris.mfdz.com
Re: [PATCH] Views: Multiple keys
Posted by Paul Bonser <mi...@gmail.com>.
On 7/23/08, Chris Anderson <jc...@grabb.it> wrote:
> Your patch looks good to me - haven't applied it yet but it seems
> right. Seeing the unit tests would make it quite clear how to interact
> with it. For instance, it may be that POSTing to the view url needs
> more flexibility than just an array of keys. In that case, maybe
> having the post body look like:
>
> {"keys":["key1","key2","key3"]} rather than just ["key1","key2","key3"]
>
> would be more flexible going forward.
Yeah, that's how I was originally doing it, but I decided to just get
it working so I could play around with it first. I was thinking of
allowing a list of docids as well, but I wasn't sure what to do in the
case when somebody passed in both. I guess it should probably be just
one or the other.
> Having some unit tests around the interaction with the MultiKeys and
> Count, SkipCount, and StartDocId params would also be helpful I think.
I expected that nobody would want to apply the patch without first
seeing some tests and seeing how the interaction with Count,
SkipCount, etc. would work. I've not had time to do any sort of
extensive testing, but I tried out Count, and it seems to work as
expected. I'll write up some unit tests when I get the chance in the
next day or so and then submit a patch with both those and these
changes.
Does anyone have any thoughts on allowing multi-docid requests?
--
Paul Bonser
http://blog.paulbonser.com
Re: [PATCH] Views: Multiple keys
Posted by Chris Anderson <jc...@grabb.it>.
On Wed, Jul 23, 2008 at 1:36 AM, Paul Bonser <mi...@gmail.com> wrote:
> You use it by POSTing a JSON array of keys to a view url. Any passed-in values for start_key and end_key are ignored.
>
> I haven't written any unittests for this yet, but I wanted to see what everyone else thinks of this and ask if there's any obvious better way to accomplish the same goal.
Your patch looks good to me - haven't applied it yet but it seems
right. Seeing the unit tests would make it quite clear how to interact
with it. For instance, it may be that POSTing to the view url needs
more flexibility than just an array of keys. In that case, maybe
having the post body look like:
{"keys":["key1","key2","key3"]} rather than just ["key1","key2","key3"]
would be more flexible going forward.
Having some unit tests around the interaction with the MultiKeys and
Count, SkipCount, and StartDocId params would also be helpful I think.
Good work and thanks for the patch!
--
Chris Anderson
http://jchris.mfdz.com
Re: [PATCH] Views: Multiple keys
Posted by Antony Blakey <an...@gmail.com>.
On 05/11/2008, at 9:28 AM, Chris Anderson wrote:
> On Tue, Nov 4, 2008 at 2:41 PM, Kore Nordmann <mail@kore-
> nordmann.de> wrote:
>>
>> I would actually prefer using GET with a request body for fetching
>> view
>> results with multiple keys, since otherwise the semantics [1] of POST
>> requests are violated.
>
> I agree, GET would be nice. Perhaps we should support GET (via url
> params) as well as POST.
>
>> Starting with using the HTTP methods in a
>> different meaning then they have been developed for, just because of
>> potential implementation details, may open a can of worms, breaking
>> the
>> whole HTTP standard.
>
> I don't think the situation is as dire as this. There's an argument
> that because multi-key view requests are in effect asking the server
> to do a lot of processing, POST is appropriate. GET is suitable only
> if we're willing to accept limitations the number of keys (remember, a
> single key can be arbitrarily long) based on practical URL length
> limitations.
>
> I guess I'm not a big believer in the cacheability of requests that
> include bodies, regardless of verb. It seems that the best argument
> for GET is cacheablity. I would fully support a patch to extract a
> JSON array of keys from the URL for view GET requests.
>
> I think we should keep POST available, because there are people who
> will need to make requests for more keys than URL parameters can
> handle.
One solution would be to separate the specification of the key set
from the document retrieval operation.
e.g.
client POST the keys -> return a UUID for a 'key set'
server would cache these for some amount of time, and might return the
same id to different POSTs of the same key set.
client GET documents with the key set as a parameter.
Thus you get cacheable multiple-document requests without breaking
HTTP semantics or running into the URL length restriction.
Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787
Plurality is not to be assumed without necessity
-- William of Ockham (ca. 1285-1349)
Re: [PATCH] Views: Multiple keys
Posted by Kore Nordmann <ma...@kore-nordmann.de>.
Chris Anderson wrote:
> On Tue, Nov 4, 2008 at 2:41 PM, Kore Nordmann <ma...@kore-nordmann.de> wrote:
>> I would actually prefer using GET with a request body for fetching view
>> results with multiple keys, since otherwise the semantics [1] of POST
>> requests are violated.
>
> I agree, GET would be nice. Perhaps we should support GET (via url
> params) as well as POST.
>
>> Starting with using the HTTP methods in a
>> different meaning then they have been developed for, just because of
>> potential implementation details, may open a can of worms, breaking the
>> whole HTTP standard.
>
> I don't think the situation is as dire as this. There's an argument
> that because multi-key view requests are in effect asking the server
> to do a lot of processing, POST is appropriate. GET is suitable only
> if we're willing to accept limitations the number of keys (remember, a
> single key can be arbitrarily long) based on practical URL length
> limitations.
Allowing both, keys as URL parameters and inside the body, as well as
GET and POST as the used HTTP method sounds like a very well option. So
the user can chose the method he wants and fall back to the other way,
if there are issues with that.
I would then suggest something like ?key=["foo", "bar"], and also allow
specifying all the other view parameters in the JSON struct included in
the request body for POST requests. This would be a quite consistent
solution, I guess.
Kind regards,
Kore
--
Kore Nordmann (GPG 0xDDC70BBB)
http://kore-nordmann.de/portfolio.html
Re: [PATCH] Views: Multiple keys
Posted by Chris Anderson <jc...@apache.org>.
On Tue, Nov 4, 2008 at 2:41 PM, Kore Nordmann <ma...@kore-nordmann.de> wrote:
>
> I would actually prefer using GET with a request body for fetching view
> results with multiple keys, since otherwise the semantics [1] of POST
> requests are violated.
I agree, GET would be nice. Perhaps we should support GET (via url
params) as well as POST.
> Starting with using the HTTP methods in a
> different meaning then they have been developed for, just because of
> potential implementation details, may open a can of worms, breaking the
> whole HTTP standard.
I don't think the situation is as dire as this. There's an argument
that because multi-key view requests are in effect asking the server
to do a lot of processing, POST is appropriate. GET is suitable only
if we're willing to accept limitations the number of keys (remember, a
single key can be arbitrarily long) based on practical URL length
limitations.
I guess I'm not a big believer in the cacheability of requests that
include bodies, regardless of verb. It seems that the best argument
for GET is cacheablity. I would fully support a patch to extract a
JSON array of keys from the URL for view GET requests.
I think we should keep POST available, because there are people who
will need to make requests for more keys than URL parameters can
handle.
Chris
--
Chris Anderson
http://jchris.mfdz.com
Re: [PATCH] Views: Multiple keys
Posted by Kore Nordmann <ma...@kore-nordmann.de>.
Hi,
Paul Bonser wrote:
> On 7/23/08, Kevin Jackson <fo...@gmail.com> wrote:
>>>> Although no one uses it, GET (and even HEAD) requests are not
>> >> forbidden from having bodies. (I think.)
>> >
>> > That does break the whole REST idea, though, doesn't it?
>>
>>
>> Yeah you'd lose 'buzzword compliance' at that point :)
>
> Yeah. I suppose a better argument is that I *know* that POST requests
> are not forbidden from having bodies. I don't see why doing it with a
> GET would make it any better. There are libraries out there that
> probably don't support a body with a GET request.
>
> No one uses it because it doesn't make sense to use it.
Sorry for the late answer, but I did not get in touch with the multi-key
support earlier.
I would actually prefer using GET with a request body for fetching view
results with multiple keys, since otherwise the semantics [1] of POST
requests are violated. Starting with using the HTTP methods in a
different meaning then they have been developed for, just because of
potential implementation details, may open a can of worms, breaking the
whole HTTP standard.
Sending GET request including data is actually not an issue, at least
with curl (curl -X GET --data-binary 'Some content' http://couch/), or
the PHP HTTP stream wrapper. (Just quickly tested those two).
But it still might be an issue with proxies, like, for example, lighttpd
always responds with a "400 - Bad Request" to such requests.
I still think that semantics of such requests really matter - especially
when they are part of the public API. If some tool does not support GET
with a request body, it is actually a bug in that tool and should be
fixed there [2]:
> A server SHOULD
read and forward a message-body on any request; if the request method
does not include defined semantics for an entity-body, then the
message-body SHOULD be ignored when handling the request.
And, if implemented with the wrong semantics the RFC clearly states,
that the implementing server breaks with the RFC [3]:
> The methods GET
and HEAD MUST be supported by all general-purpose servers. All other
methods are OPTIONAL; however, if the above methods are implemented,
they MUST be implemented with the same semantics as those specified
in section 9.
If GET allows the inclusion of an entity-body is actually unspecified,
as you can see in section #9.3. Otherwise it might still be the better
way to go with URL parameters, as proposed in the first patch, even this
would invoke limits on the number of used keys.
Sorry for nitpicking on semantics, I really prefer the feature and, of
course, the work on CouchDB in general - but it simply does not feel
right violating the RFC just to bypass potential limits.
[1] http://tools.ietf.org/html/rfc2616#section-9.5
[2] http://tools.ietf.org/html/rfc2616#section-4.3
[3] http://tools.ietf.org/html/rfc2616#section-5.1.1
--
Kore Nordmann (GPG 0xDDC70BBB)
http://kore-nordmann.de/portfolio.html
Re: [PATCH] Views: Multiple keys
Posted by Paul Bonser <mi...@gmail.com>.
On 7/23/08, Kevin Jackson <fo...@gmail.com> wrote:
> >> Although no one uses it, GET (and even HEAD) requests are not
> >> forbidden from having bodies. (I think.)
> >
> > That does break the whole REST idea, though, doesn't it?
>
>
> Yeah you'd lose 'buzzword compliance' at that point :)
Yeah. I suppose a better argument is that I *know* that POST requests
are not forbidden from having bodies. I don't see why doing it with a
GET would make it any better. There are libraries out there that
probably don't support a body with a GET request.
No one uses it because it doesn't make sense to use it.
--
Paul Bonser
http://blog.paulbonser.com
Re: [PATCH] Views: Multiple keys
Posted by Kevin Jackson <fo...@gmail.com>.
>> Although no one uses it, GET (and even HEAD) requests are not
>> forbidden from having bodies. (I think.)
>
> That does break the whole REST idea, though, doesn't it?
Yeah you'd lose 'buzzword compliance' at that point :)
Kev
Re: [PATCH] Views: Multiple keys
Posted by Paul Bonser <mi...@gmail.com>.
On 7/23/08, ryah dahl <ry...@tinyclouds.org> wrote:
> Although no one uses it, GET (and even HEAD) requests are not
> forbidden from having bodies. (I think.)
That does break the whole REST idea, though, doesn't it?
--
Paul Bonser
http://blog.paulbonser.com
Re: [PATCH] Views: Multiple keys
Posted by ryah dahl <ry...@tinyclouds.org>.
Although no one uses it, GET (and even HEAD) requests are not
forbidden from having bodies. (I think.)
On Wed, Jul 23, 2008 at 3:29 PM, Paul Bonser <mi...@gmail.com> wrote:
> On 7/23/08, Matt Goodall <ma...@gmail.com> wrote:
>> Hi,
>>
>> I think this functionality would be hugely beneficial in CouchDB but
>> POST'ing to a view seems so very, very wrong to me.
>
> I originally suggested doing it as a key_list GET parameter, but
> decided against it after a comment from Damien "The only thing is I
> don't think a GET with all the keys are URL args is the way to go as
> it can get really long and cause problems with proxies and HTTP libs,
> a POST with the lookup keys as the body is probably better."
>
>> How about using a GET on the view but support multiple 'key' args, one
>> for each of the view rows to return, e.g.
>>
>> GET /somedb/_views/somedoc/someview?key=1&key=2&key=3
>
> The problem with doing it that way is that some people will want to
> give a list of hundreds or thousands of keys, and and that point the
> query would be too long and start breaking in all sorts of different
> situations.
>
>> The 'key' arg(s) already seem to take precedence over startkey,
>> endkey, etc and the view always returns a 'rows' list so it /should/
>> be a matter of iterating the JSON values sent as 'key' instead of
>> parsing the JSON structure sent as POST content.
>
> The implementation using GET would be just about as simple as the one
> I sent, though some changes to parse_view_query and the
> view_query_args structure would be necessary.
>
>> I guess there's the possibility of hitting max URL length problems on
>> some browsers/servers but that can be worked around by requesting the
>> view in batches of keys.
>
> Once again, if we did that, we'd be right back to the problem of doing
> multiple requests, which is exactly what this new functionality is
> trying to avoid.
>
> --
> Paul Bonser
> http://blog.paulbonser.com
>
Re: [PATCH] Views: Multiple keys
Posted by Paul Bonser <mi...@gmail.com>.
On 7/23/08, Matt Goodall <ma...@gmail.com> wrote:
> Hi,
>
> I think this functionality would be hugely beneficial in CouchDB but
> POST'ing to a view seems so very, very wrong to me.
I originally suggested doing it as a key_list GET parameter, but
decided against it after a comment from Damien "The only thing is I
don't think a GET with all the keys are URL args is the way to go as
it can get really long and cause problems with proxies and HTTP libs,
a POST with the lookup keys as the body is probably better."
> How about using a GET on the view but support multiple 'key' args, one
> for each of the view rows to return, e.g.
>
> GET /somedb/_views/somedoc/someview?key=1&key=2&key=3
The problem with doing it that way is that some people will want to
give a list of hundreds or thousands of keys, and and that point the
query would be too long and start breaking in all sorts of different
situations.
> The 'key' arg(s) already seem to take precedence over startkey,
> endkey, etc and the view always returns a 'rows' list so it /should/
> be a matter of iterating the JSON values sent as 'key' instead of
> parsing the JSON structure sent as POST content.
The implementation using GET would be just about as simple as the one
I sent, though some changes to parse_view_query and the
view_query_args structure would be necessary.
> I guess there's the possibility of hitting max URL length problems on
> some browsers/servers but that can be worked around by requesting the
> view in batches of keys.
Once again, if we did that, we'd be right back to the problem of doing
multiple requests, which is exactly what this new functionality is
trying to avoid.
--
Paul Bonser
http://blog.paulbonser.com
Re: [PATCH] Views: Multiple keys
Posted by Matt Goodall <ma...@gmail.com>.
Hi,
I think this functionality would be hugely beneficial in CouchDB but
POST'ing to a view seems so very, very wrong to me.
How about using a GET on the view but support multiple 'key' args, one
for each of the view rows to return, e.g.
GET /somedb/_views/somedoc/someview?key=1&key=2&key=3
The 'key' arg(s) already seem to take precedence over startkey,
endkey, etc and the view always returns a 'rows' list so it /should/
be a matter of iterating the JSON values sent as 'key' instead of
parsing the JSON structure sent as POST content.
I guess there's the possibility of hitting max URL length problems on
some browsers/servers but that can be worked around by requesting the
view in batches of keys.
Thoughts?
- Matt
2008/7/23 Paul Bonser <mi...@gmail.com>:
> After exploring the source code a bit and abandoning a few other ways of achieving this, I've managed to get multi-key requests working. The only way to make it more efficient would require some digging into the couch_btree folding code, I think.
>
> You use it by POSTing a JSON array of keys to a view url. Any passed-in values for start_key and end_key are ignored.
>
> I haven't written any unittests for this yet, but I wanted to see what everyone else thinks of this and ask if there's any obvious better way to accomplish the same goal.
>
> This patch is against the latest in SVN as of now. Any comments or suggestions for this are very much welcome.
>
>
> Signed-off-by: Paul Bonser <mi...@gmail.com>
> --
>
> diff --git a/src/couchdb/couch_httpd.erl b/src/couchdb/couch_httpd.erl
> index af8d9b4..c813a4c 100644
> --- a/src/couchdb/couch_httpd.erl
> +++ b/src/couchdb/couch_httpd.erl
> @@ -363,6 +363,37 @@ handle_db_request(Req, 'GET', {DbName, _Db, ["_view", DocId, ViewName]}) ->
> end
> end;
>
> +% Multi-key request, with a JSON array of keys as the POST body
> +handle_db_request(Req, 'POST', {DbName, _Db, ["_view", DocId, ViewName]}) ->
> + #view_query_args{
> + count = Count,
> + skip = SkipCount,
> + direction = Dir,
> + start_docid = StartDocId
> + } = QueryArgs = parse_view_query(Req),
> +
> + case Req:get_primary_header_value("content-type") of
> + undefined -> ok;
> + "application/json" -> ok;
> + Else -> throw({incorrect_mime_type, Else})
> + end,
> + JsonKeys = tuple_to_list(cjson:decode(Req:recv_body())),
> +
> + {ok, View} = couch_view:get_map_view({DbName, "_design/" ++ DocId, ViewName}),
> + {ok, RowCount} = couch_view:get_row_count(View),
> + FoldAccInit = {Count, SkipCount, undefined, []},
> + FoldResult = lists:foldl(fun(Key, {ok, FoldAcc}) ->
> + Start = {Key, StartDocId},
> + FoldlFun = make_view_fold_fun(Req, QueryArgs#view_query_args {
> + start_key = Key,
> + end_key = Key
> + },
> + RowCount, fun couch_view:reduce_to_count/1),
> + couch_view:fold(View, Start, Dir, FoldlFun, FoldAcc)
> + end, {ok, FoldAccInit}, JsonKeys),
> + finish_view_fold(Req, RowCount, FoldResult);
> +
> +
> handle_db_request(_Req, _Method, {_DbName, _Db, ["_view", _DocId, _ViewName]}) ->
> throw({method_not_allowed, "GET,HEAD"});
>
>