You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Jaap van der Plas <ja...@brightin.nl> on 2008/09/16 13:32:40 UTC

Combine ( / rereduce?) implementation in CouchDB

Greets,

I've been searching around and found various hints towards the third
parameter of reduce-functions but I have not been able to find a
description of when exactly it is invoked, and if it is possible to
avoid your reduce function to be used in this way. It is also referred
to as re-reduce sometimes?
Would anyone be able to shed some light on this for, or point me to
resources where I can learn more about this?

TIA,
J. van der Plas

Re: Combine ( / rereduce?) implementation in CouchDB

Posted by Jan Lehnardt <ja...@apache.org>.
On Sep 16, 2008, at 13:32, Jaap van der Plas wrote:

> Greets,
>
> I've been searching around and found various hints towards the third
> parameter of reduce-functions but I have not been able to find a
> description of when exactly it is invoked, and if it is possible to
> avoid your reduce function to be used in this way. It is also referred
> to as re-reduce sometimes?
> Would anyone be able to shed some light on this for, or point me to
> resources where I can learn more about this?


rereduce is a phase that the view engine might decide to run to further
reduce its input. if it runs rereduce, the third parameter to your  
reduce
function is set to true. if your reduce function returns data that is  
structurally
different from its input, you need to specially take care of the  
rereduce
call. in case of a simple `return sum(values)` reduce and rereduce work
the same way. you cannot rely on rereduce being run or not, you just  
have
to be prepared to deal with it if you are doing anything fancy.

sorry for the rather lame explanation. :-)

Cheers
Jan
--

Re: Combine ( / rereduce?) implementation in CouchDB

Posted by Mitch Skinner <li...@arctur.us>.
Jaap van der Plas wrote:
> I've been searching around and found various hints towards the third
> parameter of reduce-functions but I have not been able to find a
> description of when exactly it is invoked, and if it is possible to
> avoid your reduce function to be used in this way.

My understanding, which comes from these blog posts:
http://damienkatz.net/2008/02/incremental_map.html
http://damienkatz.net/2008/02/incremental_map_1.html

is that the third parameter is true if the function is being run on its 
own output.  So if the output of your reduce function "looks like" the 
output of your map function, then you don't care about the third 
parameter.  But if you want your reduce function to produce something 
different from what's coming out of your map function, then the third 
parameter tells your reduce function what to expect (false for map 
output, true for reduce output).

Chris Anderson has some examples; one that cares about that parameter:
http://github.com/jchris/couchrest/tree/master/examples/word_count/views/markov
and one that doesn't:
http://github.com/jchris/couchrest/tree/master/examples/word_count/views/word_count


I don't think it makes sense to try and "prevent" your function from 
being used that way; re-running reduce on its own output (as a "combine" 
function) makes it possible to incrementally update the view data, which 
is a pretty nice win IMO.


Speaking up because I happen to have just read up on this one, so please 
forgive any mistakes,
Mitch