You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Nitin Borwankar <ni...@borwankar.com> on 2009/08/10 08:48:11 UTC
reduce fn for grouping items with same key
Hello all,
I am trying to write a reduce function which takes the map fn output
(k1, v11),
(k1, v12),
(k1, v13),
(k2, v21),
(k2, v22),
(k3,v31)
and produces as a result of the reduce
{
k1: [v11,v12,v13],
k2: [v21,v22],
k3: [v31]
}
where v11 etc could themselves be lists and do not need to be unpacked in
the final result but can stay as nested list of lists.
But I am lost as far as the (keys, values, rereduce) signature.
So far I have managed by just reusing the return sum(values) on numeric
values, but this is different and I need some conceptual help with
reduce/rereduce for non-numeric values and grouping into lists instead of
taking the sum.
Thanks much,
Nitin
37% of all statistics are made up on the spot
-------------------------------------------------------------------------------------
Nitin Borwankar
nborwankar@gmail.com
Re: reduce fn for grouping items with same key
Posted by Nitin Borwankar <ni...@borwankar.com>.
Hi Jan,
When I return null form the reduce function - here's what I get from the
view
curl $CDBA/atwfoll/_design/twex/_view/folldet?group=true
"rows":[
{"key":"k1","value":null},
{"key":"k2","value":null},
{"key":"k3","value":null},
]}
when I don't have a reduce function at all I get
curl $CDBA/atwfoll/_design/twex/_view/folldet?group=true
{"error":"query_parse_error","reason":"Query parameter \"group\" and/or
\"group_level\" are invalid for map views."}
So the grouping doesn't appear to be working as needed.
my reduce function is simply
function(keys, values, rereduce) {
return null;
};
Nitin
37% of all statistics are made up on the spot
-------------------------------------------------------------------------------------
Nitin Borwankar
nborwankar@gmail.com
On Mon, Aug 10, 2009 at 12:39 AM, Jan Lehnardt <ja...@apache.org> wrote:
> Hi Nitin,
>
>
> On 10 Aug 2009, at 08:48, Nitin Borwankar wrote:
>
> Hello all,
>>
>> I am trying to write a reduce function which takes the map fn output
>>
>>
>> (k1, v11),
>> (k1, v12),
>> (k1, v13),
>> (k2, v21),
>> (k2, v22),
>> (k3,v31)
>>
>> and produces as a result of the reduce
>>
>> {
>> k1: [v11,v12,v13],
>> k2: [v21,v22],
>> k3: [v31]
>> }
>>
>>
> You shouldn't do this in CouchDB. Reducing needs to truly reduce you data
> set. You are trying to turn tall lists into wide lists. that won't work. If
> you want to get the desired result, make your reduce function return null;
> and query the view with group=true. CouchDB does the grouping for you.
>
> where v11 etc could themselves be lists and do not need to be unpacked in
>> the final result but can stay as nested list of lists.
>>
>> But I am lost as far as the (keys, values, rereduce) signature.
>>
>
> keys and values contain lists of keys and associated values. The reduce
> step calls you function in two different ways. The first way is when the map
> output gets fed into the reduce function. In this case rereduce is set to
> false.
>
> All other (one to N) invocations of reduce operate on the output the
> previous run produced. This is the default case and rereduce is set to true.
> Your code might want to (or have to) behave differently on the first
> normalization step and the second reduction step and the rereduce flag tells
> you what step you are in.
>
> Cheers
> Jan
> --
>
>
>
> So far I have managed by just reusing the return sum(values) on numeric
>> values, but this is different and I need some conceptual help with
>> reduce/rereduce for non-numeric values and grouping into lists instead of
>> taking the sum.
>>
>> Thanks much,
>>
>> Nitin
>>
>> 37% of all statistics are made up on the spot
>>
>> -------------------------------------------------------------------------------------
>> Nitin Borwankar
>> nborwankar@gmail.com
>>
>
>
Re: reduce fn for grouping items with same key
Posted by Jan Lehnardt <ja...@apache.org>.
Hi Nitin,
On 10 Aug 2009, at 08:48, Nitin Borwankar wrote:
> Hello all,
>
> I am trying to write a reduce function which takes the map fn output
>
>
> (k1, v11),
> (k1, v12),
> (k1, v13),
> (k2, v21),
> (k2, v22),
> (k3,v31)
>
> and produces as a result of the reduce
>
> {
> k1: [v11,v12,v13],
> k2: [v21,v22],
> k3: [v31]
> }
>
You shouldn't do this in CouchDB. Reducing needs to truly reduce you
data set. You are trying to turn tall lists into wide lists. that
won't work. If you want to get the desired result, make your reduce
function return null; and query the view with group=true. CouchDB does
the grouping for you.
> where v11 etc could themselves be lists and do not need to be
> unpacked in
> the final result but can stay as nested list of lists.
>
> But I am lost as far as the (keys, values, rereduce) signature.
keys and values contain lists of keys and associated values. The
reduce step calls you function in two different ways. The first way is
when the map output gets fed into the reduce function. In this case
rereduce is set to false.
All other (one to N) invocations of reduce operate on the output the
previous run produced. This is the default case and rereduce is set to
true. Your code might want to (or have to) behave differently on the
first normalization step and the second reduction step and the
rereduce flag tells you what step you are in.
Cheers
Jan
--
> So far I have managed by just reusing the return sum(values) on
> numeric
> values, but this is different and I need some conceptual help with
> reduce/rereduce for non-numeric values and grouping into lists
> instead of
> taking the sum.
>
> Thanks much,
>
> Nitin
>
> 37% of all statistics are made up on the spot
> -------------------------------------------------------------------------------------
> Nitin Borwankar
> nborwankar@gmail.com