You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Matteo Caprari <ma...@gmail.com> on 2009/12/14 15:49:30 UTC
confused by reduce overflow error
Hi.
When I execute the view described below, I get the
"reduce_overflow_error", and can't quite understand why, as it seems
to me that the data is reducing.
The error status:
$ curl http://localhost:5984/intelligence/_design/intelligence/_view/similar_artists
{"error":"reduce_overflow_error","reason":"Reduce output must shrink
more rapidly: Current output:
'[{\"Whitesnake\": null,\"Kamelot\": null ,\"Metallica\":
null,\"System of a Down\": null,\"Rage Against the Ma'... (first 100
of 1266 bytes)"}
the reduce function does some logging and I've pasted the relevant
output on pastebin: http://pastebin.com/f1e1d509c
Each document in the db describes a user's favorite tracks:
{
"_id": "top_tracks.AbreLosOjoss",
"_rev": "1-24de6a09d52fb26214619490d4d1325f",
"user":"AbreLosOjoss",
"toptracks": [
{"track": "Calling All Angels", "artist": "Lenny Kravitz",
"playcount": 44 },
...
]
}
// map.js, emits a list of unique artists and their cumulative
normalized playcount:
// emits {"Whitesnake": 100,"Kamelot": 186.13569321533924, ...}
function(doc) {
if(doc.user && doc.toptracks) {
var artists = {};
// find max value
var max = 0;
doc.toptracks.forEach(function(track) {
if (track.playcount > max)
max = track.playcount;
});
var scale = 100/max;
doc.toptracks.forEach(function(track) {
if (!artists[track.artist]) artists[track.artist] = 0;
artists[track.artist] += track.playcount * scale;
});
emit(doc.user, artists);
}
}
// reduce.js, returns an object where keys are artist names
// returns {"Whitesnake":null, "Kamelot":null, ... }
function(keys, values, rereduce) {
var artists = {};
values.forEach(function (playlist) {
for(var artist in playlist)
artists[artist] = null;
});
log({len: values.length, values: values});
log({reduced:artists});
return artists;
}
Thanks!
--
:Matteo Caprari
matteo.caprari@gmail.com
caprazzi.net
Re: confused by reduce overflow error
Posted by Zachary Zolton <za...@gmail.com>.
I think you'll need to emit each object in the "toptracks" attribute
separately in a for-loop, instead of emitting the all of the artist at
once. So, your emitted key will be [doc.user, toptrack.artist], which
can then be reduced by using startkey=["user"] and endkey=["user",{}]
querying parameters.
Cheers,
Zach
On Mon, Dec 14, 2009 at 8:49 AM, Matteo Caprari
<ma...@gmail.com> wrote:
> Hi.
>
> When I execute the view described below, I get the
> "reduce_overflow_error", and can't quite understand why, as it seems
> to me that the data is reducing.
>
> The error status:
> $ curl http://localhost:5984/intelligence/_design/intelligence/_view/similar_artists
> {"error":"reduce_overflow_error","reason":"Reduce output must shrink
> more rapidly: Current output:
> '[{\"Whitesnake\": null,\"Kamelot\": null ,\"Metallica\":
> null,\"System of a Down\": null,\"Rage Against the Ma'... (first 100
> of 1266 bytes)"}
>
> the reduce function does some logging and I've pasted the relevant
> output on pastebin: http://pastebin.com/f1e1d509c
>
> Each document in the db describes a user's favorite tracks:
>
> {
> "_id": "top_tracks.AbreLosOjoss",
> "_rev": "1-24de6a09d52fb26214619490d4d1325f",
> "user":"AbreLosOjoss",
> "toptracks": [
> {"track": "Calling All Angels", "artist": "Lenny Kravitz",
> "playcount": 44 },
> ...
> ]
> }
>
> // map.js, emits a list of unique artists and their cumulative
> normalized playcount:
> // emits {"Whitesnake": 100,"Kamelot": 186.13569321533924, ...}
> function(doc) {
> if(doc.user && doc.toptracks) {
> var artists = {};
> // find max value
> var max = 0;
> doc.toptracks.forEach(function(track) {
> if (track.playcount > max)
> max = track.playcount;
> });
> var scale = 100/max;
> doc.toptracks.forEach(function(track) {
> if (!artists[track.artist]) artists[track.artist] = 0;
> artists[track.artist] += track.playcount * scale;
> });
> emit(doc.user, artists);
> }
> }
>
> // reduce.js, returns an object where keys are artist names
> // returns {"Whitesnake":null, "Kamelot":null, ... }
> function(keys, values, rereduce) {
> var artists = {};
>
> values.forEach(function (playlist) {
> for(var artist in playlist)
> artists[artist] = null;
> });
>
> log({len: values.length, values: values});
> log({reduced:artists});
>
> return artists;
> }
>
> Thanks!
> --
> :Matteo Caprari
> matteo.caprari@gmail.com
> caprazzi.net
>
Re: confused by reduce overflow error
Posted by Paweł Stawicki <pa...@gmail.com>.
I'm not sure what you want to achieve, but idea of reduce is to create
some scalar value (usually a number) based on many nodes (key/value
pairs, which are result of map function).
If you could tell us more what you need, maybe we'll be able to help more.
Regards
--
Paweł Stawicki
http://pawelstawicki.blogspot.com
http://szczecin.jug.pl
http://www.java4people.com