Re: [PATCH] Views: Multiple keys

[Chris Anderson <jc...@grabb.it>]

Thanks for the patch Paul. I updated it against current trunk and
added some unit tests to it. I took the opportunity to do some
refactoring and create a output_map_view function. It's working well
for design docs but for some reason I can't get it to behave correctly
with _temp_views.

The behavior is documented in the attached unit tests, but the gist of
it is that POSTs to /mydb/_view/design/view with a body like:
{"keys":[array of keys]}
return just the view rows that match the keys. There is still work to
be done to test the interactions with other request params. I started
down the path of getting reduce functions to work with keys as well,
but I think that will require getting deep into couch_btree.erl

The patch is here:

http://friendpaste.com/7Va7jgWK

If anyone can apply it and try to figure out the temp-view stuff,
that'd be super helpful. Otherwise maybe sleep and a fresh look will
let me finish it.

Chris

On Tue, Jul 22, 2008 at 10:36 PM, Paul Bonser <mi...@gmail.com> wrote:
> After exploring the source code a bit and abandoning a few other ways of achieving this, I've managed to get multi-key requests working. The only way to make it more efficient would require some digging into the couch_btree folding code, I think.
>
> You use it by POSTing a JSON array of keys to a view url. Any passed-in values for start_key and end_key are ignored.
>
> I haven't written any unittests for this yet, but I wanted to see what everyone else thinks of this and ask if there's any obvious better way to accomplish the same goal.
>
> This patch is against the latest in SVN as of now. Any comments or suggestions for this are very much welcome.
>
>
> Signed-off-by: Paul Bonser <mi...@gmail.com>
> --
>
> diff --git a/src/couchdb/couch_httpd.erl b/src/couchdb/couch_httpd.erl
> index af8d9b4..c813a4c 100644
> --- a/src/couchdb/couch_httpd.erl
> +++ b/src/couchdb/couch_httpd.erl
> @@ -363,6 +363,37 @@ handle_db_request(Req, 'GET', {DbName, _Db, ["_view", DocId, ViewName]}) ->
>         end
>     end;
>
> +% Multi-key request, with a JSON array of keys as the POST body
> +handle_db_request(Req, 'POST', {DbName, _Db, ["_view", DocId, ViewName]}) ->
> +    #view_query_args{
> +        count = Count,
> +        skip = SkipCount,
> +        direction = Dir,
> +        start_docid = StartDocId
> +    } = QueryArgs = parse_view_query(Req),
> +
> +    case Req:get_primary_header_value("content-type") of
> +        undefined -> ok;
> +        "application/json" -> ok;
> +        Else -> throw({incorrect_mime_type, Else})
> +    end,
> +       JsonKeys = tuple_to_list(cjson:decode(Req:recv_body())),
> +
> +    {ok, View} = couch_view:get_map_view({DbName, "_design/" ++ DocId, ViewName}),
> +    {ok, RowCount} = couch_view:get_row_count(View),
> +    FoldAccInit = {Count, SkipCount, undefined, []},
> +    FoldResult = lists:foldl(fun(Key, {ok, FoldAcc}) ->
> +        Start = {Key, StartDocId},
> +        FoldlFun = make_view_fold_fun(Req, QueryArgs#view_query_args {
> +                    start_key = Key,
> +                    end_key = Key
> +                },
> +                RowCount, fun couch_view:reduce_to_count/1),
> +        couch_view:fold(View, Start, Dir, FoldlFun, FoldAcc)
> +    end, {ok, FoldAccInit}, JsonKeys),
> +    finish_view_fold(Req, RowCount, FoldResult);
> +
> +
>  handle_db_request(_Req, _Method, {_DbName, _Db, ["_view", _DocId, _ViewName]}) ->
>     throw({method_not_allowed, "GET,HEAD"});
>
>



-- 
Chris Anderson
http://jchris.mfdz.com

Re: [PATCH] Views: Multiple keys

[Chris Anderson <jc...@grabb.it>]

On Wed, Aug 6, 2008 at 9:16 PM, Paul Bonser <mi...@gmail.com> wrote:
>
> I never looked into the temp view code, so I'll be of little help with
> that at the moment. Good luck on that fresh look.

I've updated the patch. The temp_view bug turned out to be
lack-of-sleep related, so it's working now, as well as multi-key
requests against reduce views (so long as group=true).

The new patch is available at

https://issues.apache.org/jira/browse/COUCHDB-101

Cheers!

-- 
Chris Anderson
http://jchris.mfdz.com

Re: [PATCH] Views: Multiple keys

[Paul Bonser <mi...@gmail.com>]

On 8/6/08, Chris Anderson <jc...@grabb.it> wrote:
> Thanks for the patch Paul. I updated it against current trunk and
>  added some unit tests to it. I took the opportunity to do some
>  refactoring and create a output_map_view function. It's working well
>  for design docs but for some reason I can't get it to behave correctly
>  with _temp_views.

I'd been meaning to write some unit tests, but I've been really busy
lately with a full-time job and some contract work on the side. I'm
glad to hear that it works alright (except for the temp views).

>  If anyone can apply it and try to figure out the temp-view stuff,
>  that'd be super helpful. Otherwise maybe sleep and a fresh look will
>  let me finish it.

I never looked into the temp view code, so I'll be of little help with
that at the moment. Good luck on that fresh look.

Hopefully I'll be able to contribute more once I'm not trying to do so
many things.

-- 
Paul Bonser
http://blog.paulbonser.com