You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Yue Chuan Lim <sh...@gmail.com> on 2010/08/06 14:38:47 UTC
Crash when using builtin reduce _stats
Fairly standard map/reduce function, map looks something like
function(doc) {
if(doc.data[12] > 0) {
key = doc._id.split("|")[0];
emit(key, doc.data[12]);
}
}
Data returned by map is along the lines of: ("test1", 123), ("test1", 245).
Reduce is just: _stats
Crash log: http://pastebin.com/aX7jeeLm
Platform: Windows 7 (32-bit)
Using the installer by Mark Hammond. Incidentally, is the installer on
couch.io the preferred one for windows deployment?
Thanks
Re: Crash when using builtin reduce _stats
Posted by J Chris Anderson <jc...@apache.org>.
On Aug 6, 2010, at 5:57 PM, Yue Chuan Lim wrote:
> Also, I can verify that a temp view with any number of trailing linefeeds
> (and whitespace) on the reduce function works fine (looks like for a
> temp-view the text gets tossed into a javascript eval?), but for an actual
> view, the reduce function needs to be just "_stats".
>
> Should I still open a JIRA ticket?
>
Modifying the test case to test that "_stats-and-then-some-junk" works would be the most helpful thing you could do.
http://github.com/apache/couchdb/blob/trunk/share/www/script/test/reduce_builtin.js
It may be that Futon is trimming the whitespace for you. I bet a tempview with more whitespace (when sent to couch) would also break.
Chris
> Thanks for your help in tracking this down, much appreciated!
> Yue Chuan
>
> On Sat, Aug 7, 2010 at 8:50 AM, J Chris Anderson <jc...@apache.org> wrote:
>
>>
>> On Aug 6, 2010, at 5:41 PM, Yue Chuan Lim wrote:
>>
>>> It appears that a trailing linefeed is the cause of my problems.
>>>
>>> On the working function, the reduce function is simply "_stats" on the
>>> failing function it is "_stats\u000a"
>>>
>>
>> ahhh, ye old linefeed bug.
>>
>> nuff said. if someone can provide a test case for this, the actual bug fix
>> should be trivial.
>>
>> I think the behavior should be a prefix test, so that even "_statsorama" or
>> "_stats\nare\awesome" should work just as "_stats" does.
>>
>> Chris
>>
>>> --- message below retained for prosperity, in the event I am wrong
>>>
>>> After playing around with the function (as above, ensuring that there are
>> no
>>> non-numeric values, as well as removing and eventually readding the > 0
>>> test), it suddenly starts working.
>>>
>>> // revision 18 - works, renamed to stats_pass
>>> {
>>> "_id": "_design/stats",
>>> "_rev": "18-776464ea3e4e548f44c55aa8b889f8a1",
>>> "langauge": "javascript",
>>> "views": {
>>> "active_power": {
>>> "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
>>> key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
>>> }\u000a}\u000a\u000a",
>>> "reduce": "_stats"
>>> }
>>> },
>>> "language": "javascript"
>>> }
>>>
>>> // revision 14 (15, 16, 17 are variants of this), when I removed the
>> reduce
>>> function to run the tests as per above
>>> {
>>> "_id": "_design/stats",
>>> "_rev": "14-416ed97296c70d0c53215c210ebcf464",
>>> "langauge": "javascript",
>>> "views": {
>>> "active_power": {
>>> "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
>>> key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
>>> }\u000a}\u000a\u000a"
>>> }
>>> },
>>> "language": "javascript"
>>> }
>>>
>>> // revision 13, the failing revision
>>> {
>>> "_id": "_design/stats",
>>> "_rev": "13-9549c38bb346d590749adc81f3f6c7e2",
>>> "langauge": "javascript",
>>> "views": {
>>> "active_power": {
>>> "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
>>> key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
>>> }\u000a}\u000a\u000a",
>>> "reduce": "_stats\u000a"
>>> }
>>> },
>>> "language": "javascript"
>>> }
>>>
>>> These appear to me as essentially identical views (13 and 18). but 13
>> fails
>>> and 18 passes. Copy/pasting them to a new view (i.e. stats_pass (rev 18)
>> and
>>> stats_fail (rev 13)) on a couch.io instance, and they continue to pass
>> and
>>> fail respectively.
>>>
>>> http://shasderias.couchone_remove_me_
>>> .com/gsc_lt2/_design/stats_pass/_view/active_power
>>> http://shasderias.couchone_remove_me_
>>> .com/gsc_lt2/_design/stats_fails/_view/active_power
>>>
>>> On Sat, Aug 7, 2010 at 8:24 AM, J Chris Anderson <jc...@gmail.com>
>> wrote:
>>>
>>>>
>>>> On Aug 6, 2010, at 5:21 PM, Yue Chuan Lim wrote:
>>>>
>>>>> I highly doubt it, modifying the function to:
>>>>>
>>>>> function(doc) {
>>>>> if(doc.data[12] > 0) {
>>>>> key = doc._id.split("|")[0];
>>>>> emit(doc.data[12], doc.data[12]);
>>>>> }
>>>>> }
>>>>>
>>>>> and sorting by key, it appears all the numbers are indeed numbers.
>>>>>
>>>>> function(doc) {
>>>>> if(doc.data[12] > 0) {
>>>>> key = doc._id.split("|")[0];
>>>>> if(doc.data[12] != parseInt(doc.data[12])) {
>>>>> emit(doc.data[12], doc.data[12]);
>>>>> }
>>>>> }
>>>>> }
>>>>>
>>>>> Returns a empty set.
>>>>>
>>>>> On a hunch, I removed the doc.data[12] > 0 line, and the query seems to
>>>> run
>>>>> fine
>>>>>
>>>>> (with group=true)
>>>>>
>>>>> {"rows":[
>>>>>
>>>>
>> {"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
>>>>>
>>>>
>> {"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
>>>>>
>>>>
>> {"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
>>>>> ]}
>>>>>
>>>>> So it appears that _stats doesn't like value sets without 0s :P (or
>> maybe
>>>>> reduce + group=true has something against unequal count values)
>>>>>
>>>>> Thanks for looking into this :)
>>>>
>>>> Since there isn't an easy answer in sight, and I'm up to my neck in
>> other
>>>> patches right now, maybe the best thing to do is start a Jira ticket to
>>>> record your observations while they are still fresh.
>>>>
>>>> http://issues.apache.org/jira/browse/COUCHDB
>>>>
>>>> Thanks,
>>>> Chris
>>>>
>>>>> Yue Chuan
>>>>>
>>>>>
>>>>> On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jc...@apache.org>
>>>> wrote:
>>>>>
>>>>>>
>>>>>> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
>>>>>>
>>>>>>> Further tested on the couch.io windows installer as well as a
>>>>>>> couch.ioinstance with the same results, works on a temp view, crashes
>>>>>>> on a actual
>>>>>>> view.
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
>>>>>>>
>>>>>>> is the couchone instance.
>>>>>>
>>>>>> Is there any chance you have a few docs where the value is not a
>> number?
>>>> If
>>>>>> you remove the reduce function completely and query the map is every
>>>> single
>>>>>> row's value a number?
>>>>>>
>>>>>> Thanks for helping us get to the bottom of this.
>>>>>>
>>>>>> Chris
>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> And curiously enough, running it as a temp view works just fine.
>>>>>>>>
>>>>>>>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
>>>>>>>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Fairly standard map/reduce function, map looks something like
>>>>>>>>>
>>>>>>>>> function(doc) {
>>>>>>>>> if(doc.data[12] > 0) {
>>>>>>>>> key = doc._id.split("|")[0];
>>>>>>>>> emit(key, doc.data[12]);
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Data returned by map is along the lines of: ("test1", 123),
>> ("test1",
>>>>>>>>> 245).
>>>>>>>>>
>>>>>>>>> Reduce is just: _stats
>>>>>>>>>
>>>>>>>>> Crash log: http://pastebin.com/aX7jeeLm
>>>>>>>>> Platform: Windows 7 (32-bit)
>>>>>>>>>
>>>>>>>>> Using the installer by Mark Hammond. Incidentally, is the installer
>>>> on
>>>>>>>>> couch.io the preferred one for windows deployment?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: Crash when using builtin reduce _stats
Posted by Yue Chuan Lim <sh...@gmail.com>.
Also, I can verify that a temp view with any number of trailing linefeeds
(and whitespace) on the reduce function works fine (looks like for a
temp-view the text gets tossed into a javascript eval?), but for an actual
view, the reduce function needs to be just "_stats".
Should I still open a JIRA ticket?
Thanks for your help in tracking this down, much appreciated!
Yue Chuan
On Sat, Aug 7, 2010 at 8:50 AM, J Chris Anderson <jc...@apache.org> wrote:
>
> On Aug 6, 2010, at 5:41 PM, Yue Chuan Lim wrote:
>
> > It appears that a trailing linefeed is the cause of my problems.
> >
> > On the working function, the reduce function is simply "_stats" on the
> > failing function it is "_stats\u000a"
> >
>
> ahhh, ye old linefeed bug.
>
> nuff said. if someone can provide a test case for this, the actual bug fix
> should be trivial.
>
> I think the behavior should be a prefix test, so that even "_statsorama" or
> "_stats\nare\awesome" should work just as "_stats" does.
>
> Chris
>
> > --- message below retained for prosperity, in the event I am wrong
> >
> > After playing around with the function (as above, ensuring that there are
> no
> > non-numeric values, as well as removing and eventually readding the > 0
> > test), it suddenly starts working.
> >
> > // revision 18 - works, renamed to stats_pass
> > {
> > "_id": "_design/stats",
> > "_rev": "18-776464ea3e4e548f44c55aa8b889f8a1",
> > "langauge": "javascript",
> > "views": {
> > "active_power": {
> > "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
> > key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
> > }\u000a}\u000a\u000a",
> > "reduce": "_stats"
> > }
> > },
> > "language": "javascript"
> > }
> >
> > // revision 14 (15, 16, 17 are variants of this), when I removed the
> reduce
> > function to run the tests as per above
> > {
> > "_id": "_design/stats",
> > "_rev": "14-416ed97296c70d0c53215c210ebcf464",
> > "langauge": "javascript",
> > "views": {
> > "active_power": {
> > "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
> > key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
> > }\u000a}\u000a\u000a"
> > }
> > },
> > "language": "javascript"
> > }
> >
> > // revision 13, the failing revision
> > {
> > "_id": "_design/stats",
> > "_rev": "13-9549c38bb346d590749adc81f3f6c7e2",
> > "langauge": "javascript",
> > "views": {
> > "active_power": {
> > "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
> > key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
> > }\u000a}\u000a\u000a",
> > "reduce": "_stats\u000a"
> > }
> > },
> > "language": "javascript"
> > }
> >
> > These appear to me as essentially identical views (13 and 18). but 13
> fails
> > and 18 passes. Copy/pasting them to a new view (i.e. stats_pass (rev 18)
> and
> > stats_fail (rev 13)) on a couch.io instance, and they continue to pass
> and
> > fail respectively.
> >
> > http://shasderias.couchone_remove_me_
> > .com/gsc_lt2/_design/stats_pass/_view/active_power
> > http://shasderias.couchone_remove_me_
> > .com/gsc_lt2/_design/stats_fails/_view/active_power
> >
> > On Sat, Aug 7, 2010 at 8:24 AM, J Chris Anderson <jc...@gmail.com>
> wrote:
> >
> >>
> >> On Aug 6, 2010, at 5:21 PM, Yue Chuan Lim wrote:
> >>
> >>> I highly doubt it, modifying the function to:
> >>>
> >>> function(doc) {
> >>> if(doc.data[12] > 0) {
> >>> key = doc._id.split("|")[0];
> >>> emit(doc.data[12], doc.data[12]);
> >>> }
> >>> }
> >>>
> >>> and sorting by key, it appears all the numbers are indeed numbers.
> >>>
> >>> function(doc) {
> >>> if(doc.data[12] > 0) {
> >>> key = doc._id.split("|")[0];
> >>> if(doc.data[12] != parseInt(doc.data[12])) {
> >>> emit(doc.data[12], doc.data[12]);
> >>> }
> >>> }
> >>> }
> >>>
> >>> Returns a empty set.
> >>>
> >>> On a hunch, I removed the doc.data[12] > 0 line, and the query seems to
> >> run
> >>> fine
> >>>
> >>> (with group=true)
> >>>
> >>> {"rows":[
> >>>
> >>
> {"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
> >>>
> >>
> {"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
> >>>
> >>
> {"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
> >>> ]}
> >>>
> >>> So it appears that _stats doesn't like value sets without 0s :P (or
> maybe
> >>> reduce + group=true has something against unequal count values)
> >>>
> >>> Thanks for looking into this :)
> >>
> >> Since there isn't an easy answer in sight, and I'm up to my neck in
> other
> >> patches right now, maybe the best thing to do is start a Jira ticket to
> >> record your observations while they are still fresh.
> >>
> >> http://issues.apache.org/jira/browse/COUCHDB
> >>
> >> Thanks,
> >> Chris
> >>
> >>> Yue Chuan
> >>>
> >>>
> >>> On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jc...@apache.org>
> >> wrote:
> >>>
> >>>>
> >>>> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
> >>>>
> >>>>> Further tested on the couch.io windows installer as well as a
> >>>>> couch.ioinstance with the same results, works on a temp view, crashes
> >>>>> on a actual
> >>>>> view.
> >>>>>
> >>>>>
> >>>>
> >>
> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
> >>>>>
> >>>>> is the couchone instance.
> >>>>
> >>>> Is there any chance you have a few docs where the value is not a
> number?
> >> If
> >>>> you remove the reduce function completely and query the map is every
> >> single
> >>>> row's value a number?
> >>>>
> >>>> Thanks for helping us get to the bottom of this.
> >>>>
> >>>> Chris
> >>>>
> >>>>>
> >>>>> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> And curiously enough, running it as a temp view works just fine.
> >>>>>>
> >>>>>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
> >>>>>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
> >>>>> wrote:
> >>>>>>
> >>>>>>> Fairly standard map/reduce function, map looks something like
> >>>>>>>
> >>>>>>> function(doc) {
> >>>>>>> if(doc.data[12] > 0) {
> >>>>>>> key = doc._id.split("|")[0];
> >>>>>>> emit(key, doc.data[12]);
> >>>>>>> }
> >>>>>>> }
> >>>>>>>
> >>>>>>> Data returned by map is along the lines of: ("test1", 123),
> ("test1",
> >>>>>>> 245).
> >>>>>>>
> >>>>>>> Reduce is just: _stats
> >>>>>>>
> >>>>>>> Crash log: http://pastebin.com/aX7jeeLm
> >>>>>>> Platform: Windows 7 (32-bit)
> >>>>>>>
> >>>>>>> Using the installer by Mark Hammond. Incidentally, is the installer
> >> on
> >>>>>>> couch.io the preferred one for windows deployment?
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>
Re: Crash when using builtin reduce _stats
Posted by J Chris Anderson <jc...@apache.org>.
On Aug 6, 2010, at 5:41 PM, Yue Chuan Lim wrote:
> It appears that a trailing linefeed is the cause of my problems.
>
> On the working function, the reduce function is simply "_stats" on the
> failing function it is "_stats\u000a"
>
ahhh, ye old linefeed bug.
nuff said. if someone can provide a test case for this, the actual bug fix should be trivial.
I think the behavior should be a prefix test, so that even "_statsorama" or "_stats\nare\awesome" should work just as "_stats" does.
Chris
> --- message below retained for prosperity, in the event I am wrong
>
> After playing around with the function (as above, ensuring that there are no
> non-numeric values, as well as removing and eventually readding the > 0
> test), it suddenly starts working.
>
> // revision 18 - works, renamed to stats_pass
> {
> "_id": "_design/stats",
> "_rev": "18-776464ea3e4e548f44c55aa8b889f8a1",
> "langauge": "javascript",
> "views": {
> "active_power": {
> "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
> key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
> }\u000a}\u000a\u000a",
> "reduce": "_stats"
> }
> },
> "language": "javascript"
> }
>
> // revision 14 (15, 16, 17 are variants of this), when I removed the reduce
> function to run the tests as per above
> {
> "_id": "_design/stats",
> "_rev": "14-416ed97296c70d0c53215c210ebcf464",
> "langauge": "javascript",
> "views": {
> "active_power": {
> "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
> key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
> }\u000a}\u000a\u000a"
> }
> },
> "language": "javascript"
> }
>
> // revision 13, the failing revision
> {
> "_id": "_design/stats",
> "_rev": "13-9549c38bb346d590749adc81f3f6c7e2",
> "langauge": "javascript",
> "views": {
> "active_power": {
> "map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
> key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
> }\u000a}\u000a\u000a",
> "reduce": "_stats\u000a"
> }
> },
> "language": "javascript"
> }
>
> These appear to me as essentially identical views (13 and 18). but 13 fails
> and 18 passes. Copy/pasting them to a new view (i.e. stats_pass (rev 18) and
> stats_fail (rev 13)) on a couch.io instance, and they continue to pass and
> fail respectively.
>
> http://shasderias.couchone_remove_me_
> .com/gsc_lt2/_design/stats_pass/_view/active_power
> http://shasderias.couchone_remove_me_
> .com/gsc_lt2/_design/stats_fails/_view/active_power
>
> On Sat, Aug 7, 2010 at 8:24 AM, J Chris Anderson <jc...@gmail.com> wrote:
>
>>
>> On Aug 6, 2010, at 5:21 PM, Yue Chuan Lim wrote:
>>
>>> I highly doubt it, modifying the function to:
>>>
>>> function(doc) {
>>> if(doc.data[12] > 0) {
>>> key = doc._id.split("|")[0];
>>> emit(doc.data[12], doc.data[12]);
>>> }
>>> }
>>>
>>> and sorting by key, it appears all the numbers are indeed numbers.
>>>
>>> function(doc) {
>>> if(doc.data[12] > 0) {
>>> key = doc._id.split("|")[0];
>>> if(doc.data[12] != parseInt(doc.data[12])) {
>>> emit(doc.data[12], doc.data[12]);
>>> }
>>> }
>>> }
>>>
>>> Returns a empty set.
>>>
>>> On a hunch, I removed the doc.data[12] > 0 line, and the query seems to
>> run
>>> fine
>>>
>>> (with group=true)
>>>
>>> {"rows":[
>>>
>> {"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
>>>
>> {"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
>>>
>> {"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
>>> ]}
>>>
>>> So it appears that _stats doesn't like value sets without 0s :P (or maybe
>>> reduce + group=true has something against unequal count values)
>>>
>>> Thanks for looking into this :)
>>
>> Since there isn't an easy answer in sight, and I'm up to my neck in other
>> patches right now, maybe the best thing to do is start a Jira ticket to
>> record your observations while they are still fresh.
>>
>> http://issues.apache.org/jira/browse/COUCHDB
>>
>> Thanks,
>> Chris
>>
>>> Yue Chuan
>>>
>>>
>>> On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jc...@apache.org>
>> wrote:
>>>
>>>>
>>>> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
>>>>
>>>>> Further tested on the couch.io windows installer as well as a
>>>>> couch.ioinstance with the same results, works on a temp view, crashes
>>>>> on a actual
>>>>> view.
>>>>>
>>>>>
>>>>
>> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
>>>>>
>>>>> is the couchone instance.
>>>>
>>>> Is there any chance you have a few docs where the value is not a number?
>> If
>>>> you remove the reduce function completely and query the map is every
>> single
>>>> row's value a number?
>>>>
>>>> Thanks for helping us get to the bottom of this.
>>>>
>>>> Chris
>>>>
>>>>>
>>>>> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> And curiously enough, running it as a temp view works just fine.
>>>>>>
>>>>>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
>>>>>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
>>>>> wrote:
>>>>>>
>>>>>>> Fairly standard map/reduce function, map looks something like
>>>>>>>
>>>>>>> function(doc) {
>>>>>>> if(doc.data[12] > 0) {
>>>>>>> key = doc._id.split("|")[0];
>>>>>>> emit(key, doc.data[12]);
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> Data returned by map is along the lines of: ("test1", 123), ("test1",
>>>>>>> 245).
>>>>>>>
>>>>>>> Reduce is just: _stats
>>>>>>>
>>>>>>> Crash log: http://pastebin.com/aX7jeeLm
>>>>>>> Platform: Windows 7 (32-bit)
>>>>>>>
>>>>>>> Using the installer by Mark Hammond. Incidentally, is the installer
>> on
>>>>>>> couch.io the preferred one for windows deployment?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: Crash when using builtin reduce _stats
Posted by Yue Chuan Lim <sh...@gmail.com>.
It appears that a trailing linefeed is the cause of my problems.
On the working function, the reduce function is simply "_stats" on the
failing function it is "_stats\u000a"
--- message below retained for prosperity, in the event I am wrong
After playing around with the function (as above, ensuring that there are no
non-numeric values, as well as removing and eventually readding the > 0
test), it suddenly starts working.
// revision 18 - works, renamed to stats_pass
{
"_id": "_design/stats",
"_rev": "18-776464ea3e4e548f44c55aa8b889f8a1",
"langauge": "javascript",
"views": {
"active_power": {
"map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
}\u000a}\u000a\u000a",
"reduce": "_stats"
}
},
"language": "javascript"
}
// revision 14 (15, 16, 17 are variants of this), when I removed the reduce
function to run the tests as per above
{
"_id": "_design/stats",
"_rev": "14-416ed97296c70d0c53215c210ebcf464",
"langauge": "javascript",
"views": {
"active_power": {
"map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
}\u000a}\u000a\u000a"
}
},
"language": "javascript"
}
// revision 13, the failing revision
{
"_id": "_design/stats",
"_rev": "13-9549c38bb346d590749adc81f3f6c7e2",
"langauge": "javascript",
"views": {
"active_power": {
"map": "function(doc) {\u000a if(doc.data[12] > 0) {\u000a
key = doc._id.split(\"|\")[0];\u000a emit(key, doc.data[12]);\u000a
}\u000a}\u000a\u000a",
"reduce": "_stats\u000a"
}
},
"language": "javascript"
}
These appear to me as essentially identical views (13 and 18). but 13 fails
and 18 passes. Copy/pasting them to a new view (i.e. stats_pass (rev 18) and
stats_fail (rev 13)) on a couch.io instance, and they continue to pass and
fail respectively.
http://shasderias.couchone_remove_me_
.com/gsc_lt2/_design/stats_pass/_view/active_power
http://shasderias.couchone_remove_me_
.com/gsc_lt2/_design/stats_fails/_view/active_power
On Sat, Aug 7, 2010 at 8:24 AM, J Chris Anderson <jc...@gmail.com> wrote:
>
> On Aug 6, 2010, at 5:21 PM, Yue Chuan Lim wrote:
>
> > I highly doubt it, modifying the function to:
> >
> > function(doc) {
> > if(doc.data[12] > 0) {
> > key = doc._id.split("|")[0];
> > emit(doc.data[12], doc.data[12]);
> > }
> > }
> >
> > and sorting by key, it appears all the numbers are indeed numbers.
> >
> > function(doc) {
> > if(doc.data[12] > 0) {
> > key = doc._id.split("|")[0];
> > if(doc.data[12] != parseInt(doc.data[12])) {
> > emit(doc.data[12], doc.data[12]);
> > }
> > }
> > }
> >
> > Returns a empty set.
> >
> > On a hunch, I removed the doc.data[12] > 0 line, and the query seems to
> run
> > fine
> >
> > (with group=true)
> >
> > {"rows":[
> >
> {"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
> >
> {"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
> >
> {"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
> > ]}
> >
> > So it appears that _stats doesn't like value sets without 0s :P (or maybe
> > reduce + group=true has something against unequal count values)
> >
> > Thanks for looking into this :)
>
> Since there isn't an easy answer in sight, and I'm up to my neck in other
> patches right now, maybe the best thing to do is start a Jira ticket to
> record your observations while they are still fresh.
>
> http://issues.apache.org/jira/browse/COUCHDB
>
> Thanks,
> Chris
>
> > Yue Chuan
> >
> >
> > On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jc...@apache.org>
> wrote:
> >
> >>
> >> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
> >>
> >>> Further tested on the couch.io windows installer as well as a
> >>> couch.ioinstance with the same results, works on a temp view, crashes
> >>> on a actual
> >>> view.
> >>>
> >>>
> >>
> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
> >>>
> >>> is the couchone instance.
> >>
> >> Is there any chance you have a few docs where the value is not a number?
> If
> >> you remove the reduce function completely and query the map is every
> single
> >> row's value a number?
> >>
> >> Thanks for helping us get to the bottom of this.
> >>
> >> Chris
> >>
> >>>
> >>> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com>
> >> wrote:
> >>>
> >>>> And curiously enough, running it as a temp view works just fine.
> >>>>
> >>>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
> >>>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
> >>>>
> >>>>
> >>>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
> >>> wrote:
> >>>>
> >>>>> Fairly standard map/reduce function, map looks something like
> >>>>>
> >>>>> function(doc) {
> >>>>> if(doc.data[12] > 0) {
> >>>>> key = doc._id.split("|")[0];
> >>>>> emit(key, doc.data[12]);
> >>>>> }
> >>>>> }
> >>>>>
> >>>>> Data returned by map is along the lines of: ("test1", 123), ("test1",
> >>>>> 245).
> >>>>>
> >>>>> Reduce is just: _stats
> >>>>>
> >>>>> Crash log: http://pastebin.com/aX7jeeLm
> >>>>> Platform: Windows 7 (32-bit)
> >>>>>
> >>>>> Using the installer by Mark Hammond. Incidentally, is the installer
> on
> >>>>> couch.io the preferred one for windows deployment?
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: Crash when using builtin reduce _stats
Posted by J Chris Anderson <jc...@gmail.com>.
On Aug 6, 2010, at 5:21 PM, Yue Chuan Lim wrote:
> I highly doubt it, modifying the function to:
>
> function(doc) {
> if(doc.data[12] > 0) {
> key = doc._id.split("|")[0];
> emit(doc.data[12], doc.data[12]);
> }
> }
>
> and sorting by key, it appears all the numbers are indeed numbers.
>
> function(doc) {
> if(doc.data[12] > 0) {
> key = doc._id.split("|")[0];
> if(doc.data[12] != parseInt(doc.data[12])) {
> emit(doc.data[12], doc.data[12]);
> }
> }
> }
>
> Returns a empty set.
>
> On a hunch, I removed the doc.data[12] > 0 line, and the query seems to run
> fine
>
> (with group=true)
>
> {"rows":[
> {"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
> {"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
> {"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
> ]}
>
> So it appears that _stats doesn't like value sets without 0s :P (or maybe
> reduce + group=true has something against unequal count values)
>
> Thanks for looking into this :)
Since there isn't an easy answer in sight, and I'm up to my neck in other patches right now, maybe the best thing to do is start a Jira ticket to record your observations while they are still fresh.
http://issues.apache.org/jira/browse/COUCHDB
Thanks,
Chris
> Yue Chuan
>
>
> On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jc...@apache.org> wrote:
>
>>
>> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
>>
>>> Further tested on the couch.io windows installer as well as a
>>> couch.ioinstance with the same results, works on a temp view, crashes
>>> on a actual
>>> view.
>>>
>>>
>> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
>>>
>>> is the couchone instance.
>>
>> Is there any chance you have a few docs where the value is not a number? If
>> you remove the reduce function completely and query the map is every single
>> row's value a number?
>>
>> Thanks for helping us get to the bottom of this.
>>
>> Chris
>>
>>>
>>> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com>
>> wrote:
>>>
>>>> And curiously enough, running it as a temp view works just fine.
>>>>
>>>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
>>>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
>>>>
>>>>
>>>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
>>> wrote:
>>>>
>>>>> Fairly standard map/reduce function, map looks something like
>>>>>
>>>>> function(doc) {
>>>>> if(doc.data[12] > 0) {
>>>>> key = doc._id.split("|")[0];
>>>>> emit(key, doc.data[12]);
>>>>> }
>>>>> }
>>>>>
>>>>> Data returned by map is along the lines of: ("test1", 123), ("test1",
>>>>> 245).
>>>>>
>>>>> Reduce is just: _stats
>>>>>
>>>>> Crash log: http://pastebin.com/aX7jeeLm
>>>>> Platform: Windows 7 (32-bit)
>>>>>
>>>>> Using the installer by Mark Hammond. Incidentally, is the installer on
>>>>> couch.io the preferred one for windows deployment?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>
>>
>>
Re: Crash when using builtin reduce _stats
Posted by J Chris Anderson <jc...@apache.org>.
On Aug 6, 2010, at 5:21 PM, Yue Chuan Lim wrote:
> I highly doubt it, modifying the function to:
>
> function(doc) {
> if(doc.data[12] > 0) {
> key = doc._id.split("|")[0];
> emit(doc.data[12], doc.data[12]);
> }
> }
>
> and sorting by key, it appears all the numbers are indeed numbers.
>
> function(doc) {
> if(doc.data[12] > 0) {
> key = doc._id.split("|")[0];
> if(doc.data[12] != parseInt(doc.data[12])) {
> emit(doc.data[12], doc.data[12]);
> }
> }
> }
>
> Returns a empty set.
>
> On a hunch, I removed the doc.data[12] > 0 line, and the query seems to run
> fine
>
> (with group=true)
>
> {"rows":[
> {"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
> {"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
> {"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
> ]}
>
> So it appears that _stats doesn't like value sets without 0s :P (or maybe
> reduce + group=true has something against unequal count values)
>
> Thanks for looking into this :)
Come to think of it, if you can get this test to reproduce the error, we will have gone a Long Way toward fixing it:
http://github.com/apache/couchdb/blob/trunk/share/www/script/test/reduce_builtin.js
Thanks,
Chris
> Yue Chuan
>
>
> On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jc...@apache.org> wrote:
>
>>
>> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
>>
>>> Further tested on the couch.io windows installer as well as a
>>> couch.ioinstance with the same results, works on a temp view, crashes
>>> on a actual
>>> view.
>>>
>>>
>> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
>>>
>>> is the couchone instance.
>>
>> Is there any chance you have a few docs where the value is not a number? If
>> you remove the reduce function completely and query the map is every single
>> row's value a number?
>>
>> Thanks for helping us get to the bottom of this.
>>
>> Chris
>>
>>>
>>> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com>
>> wrote:
>>>
>>>> And curiously enough, running it as a temp view works just fine.
>>>>
>>>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
>>>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
>>>>
>>>>
>>>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
>>> wrote:
>>>>
>>>>> Fairly standard map/reduce function, map looks something like
>>>>>
>>>>> function(doc) {
>>>>> if(doc.data[12] > 0) {
>>>>> key = doc._id.split("|")[0];
>>>>> emit(key, doc.data[12]);
>>>>> }
>>>>> }
>>>>>
>>>>> Data returned by map is along the lines of: ("test1", 123), ("test1",
>>>>> 245).
>>>>>
>>>>> Reduce is just: _stats
>>>>>
>>>>> Crash log: http://pastebin.com/aX7jeeLm
>>>>> Platform: Windows 7 (32-bit)
>>>>>
>>>>> Using the installer by Mark Hammond. Incidentally, is the installer on
>>>>> couch.io the preferred one for windows deployment?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>>
>>>>
>>
>>
Re: Crash when using builtin reduce _stats
Posted by Yue Chuan Lim <sh...@gmail.com>.
I highly doubt it, modifying the function to:
function(doc) {
if(doc.data[12] > 0) {
key = doc._id.split("|")[0];
emit(doc.data[12], doc.data[12]);
}
}
and sorting by key, it appears all the numbers are indeed numbers.
function(doc) {
if(doc.data[12] > 0) {
key = doc._id.split("|")[0];
if(doc.data[12] != parseInt(doc.data[12])) {
emit(doc.data[12], doc.data[12]);
}
}
}
Returns a empty set.
On a hunch, I removed the doc.data[12] > 0 line, and the query seems to run
fine
(with group=true)
{"rows":[
{"key":"gsc_test_01","value":{"sum":335168,"count":1000,"min":0,"max":389,"sumsqr":128263850}},
{"key":"gsc_test_02","value":{"sum":381805,"count":1000,"min":0,"max":765,"sumsqr":176920147}},
{"key":"gsc_test_03","value":{"sum":411391,"count":1000,"min":187,"max":761,"sumsqr":178307983}}
]}
So it appears that _stats doesn't like value sets without 0s :P (or maybe
reduce + group=true has something against unequal count values)
Thanks for looking into this :)
Yue Chuan
On Sat, Aug 7, 2010 at 8:13 AM, J Chris Anderson <jc...@apache.org> wrote:
>
> On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
>
> > Further tested on the couch.io windows installer as well as a
> > couch.ioinstance with the same results, works on a temp view, crashes
> > on a actual
> > view.
> >
> >
> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
> >
> > is the couchone instance.
>
> Is there any chance you have a few docs where the value is not a number? If
> you remove the reduce function completely and query the map is every single
> row's value a number?
>
> Thanks for helping us get to the bottom of this.
>
> Chris
>
> >
> > On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com>
> wrote:
> >
> >> And curiously enough, running it as a temp view works just fine.
> >>
> >> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
> >> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
> >>
> >>
> >> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <shasderias@gmail.com
> >wrote:
> >>
> >>> Fairly standard map/reduce function, map looks something like
> >>>
> >>> function(doc) {
> >>> if(doc.data[12] > 0) {
> >>> key = doc._id.split("|")[0];
> >>> emit(key, doc.data[12]);
> >>> }
> >>> }
> >>>
> >>> Data returned by map is along the lines of: ("test1", 123), ("test1",
> >>> 245).
> >>>
> >>> Reduce is just: _stats
> >>>
> >>> Crash log: http://pastebin.com/aX7jeeLm
> >>> Platform: Windows 7 (32-bit)
> >>>
> >>> Using the installer by Mark Hammond. Incidentally, is the installer on
> >>> couch.io the preferred one for windows deployment?
> >>>
> >>> Thanks
> >>>
> >>>
> >>>
> >>
>
>
Re: Crash when using builtin reduce _stats
Posted by J Chris Anderson <jc...@apache.org>.
On Aug 6, 2010, at 5:09 PM, Yue Chuan Lim wrote:
> Further tested on the couch.io windows installer as well as a
> couch.ioinstance with the same results, works on a temp view, crashes
> on a actual
> view.
>
> shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
>
> is the couchone instance.
Is there any chance you have a few docs where the value is not a number? If you remove the reduce function completely and query the map is every single row's value a number?
Thanks for helping us get to the bottom of this.
Chris
>
> On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com> wrote:
>
>> And curiously enough, running it as a temp view works just fine.
>>
>> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
>> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
>>
>>
>> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <sh...@gmail.com>wrote:
>>
>>> Fairly standard map/reduce function, map looks something like
>>>
>>> function(doc) {
>>> if(doc.data[12] > 0) {
>>> key = doc._id.split("|")[0];
>>> emit(key, doc.data[12]);
>>> }
>>> }
>>>
>>> Data returned by map is along the lines of: ("test1", 123), ("test1",
>>> 245).
>>>
>>> Reduce is just: _stats
>>>
>>> Crash log: http://pastebin.com/aX7jeeLm
>>> Platform: Windows 7 (32-bit)
>>>
>>> Using the installer by Mark Hammond. Incidentally, is the installer on
>>> couch.io the preferred one for windows deployment?
>>>
>>> Thanks
>>>
>>>
>>>
>>
Re: Crash when using builtin reduce _stats
Posted by Yue Chuan Lim <sh...@gmail.com>.
Further tested on the couch.io windows installer as well as a
couch.ioinstance with the same results, works on a temp view, crashes
on a actual
view.
shasderias.couchone_remove_me_.com/_utils/database.html?gsc_lt2/_design/stats/_view/active_power
is the couchone instance.
On Fri, Aug 6, 2010 at 9:01 PM, Yue Chuan Lim <sh...@gmail.com> wrote:
> And curiously enough, running it as a temp view works just fine.
>
> [info] [<0.1859.0>] 127.0.0.1 - - 'POST'
> /gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
>
>
> On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <sh...@gmail.com>wrote:
>
>> Fairly standard map/reduce function, map looks something like
>>
>> function(doc) {
>> if(doc.data[12] > 0) {
>> key = doc._id.split("|")[0];
>> emit(key, doc.data[12]);
>> }
>> }
>>
>> Data returned by map is along the lines of: ("test1", 123), ("test1",
>> 245).
>>
>> Reduce is just: _stats
>>
>> Crash log: http://pastebin.com/aX7jeeLm
>> Platform: Windows 7 (32-bit)
>>
>> Using the installer by Mark Hammond. Incidentally, is the installer on
>> couch.io the preferred one for windows deployment?
>>
>> Thanks
>>
>>
>>
>
Re: Crash when using builtin reduce _stats
Posted by Yue Chuan Lim <sh...@gmail.com>.
And curiously enough, running it as a temp view works just fine.
[info] [<0.1859.0>] 127.0.0.1 - - 'POST'
/gsc_lt2/_temp_view?limit=11&descending=true&group=true 200
On Fri, Aug 6, 2010 at 8:38 PM, Yue Chuan Lim <sh...@gmail.com> wrote:
> Fairly standard map/reduce function, map looks something like
>
> function(doc) {
> if(doc.data[12] > 0) {
> key = doc._id.split("|")[0];
> emit(key, doc.data[12]);
> }
> }
>
> Data returned by map is along the lines of: ("test1", 123), ("test1", 245).
>
> Reduce is just: _stats
>
> Crash log: http://pastebin.com/aX7jeeLm
> Platform: Windows 7 (32-bit)
>
> Using the installer by Mark Hammond. Incidentally, is the installer on
> couch.io the preferred one for windows deployment?
>
> Thanks
>
>
>