You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Kiril Stankov <ki...@open-net.biz> on 2016/09/28 20:37:56 UTC
Re: Issue with Reduce - NOT SOLVED.
Well,
actually, it got from bad to worst.
Compact view doesn't fix the problem anymore.
Now I always get wrong output.
Modifying the reduce to return just vals.length returns always 2.
I found that the issue appears after each insertion of new doc to the DB.
Any help will be highly appreciated, this is a production environment... :(
------------------------------------------------------------------------
*With best regards,*
Kiril Stankov,
On 28.9.2016 \u0433. 22:15, Jan Lehnardt wrote:
> Thanks!
>
> Do you have any more info on the type of operating system this runs on, the file system situation, etc?
>
> Best
> Jan
> --
>
>> On 28 Sep 2016, at 17:47, Kiril Stankov <ki...@open-net.biz> wrote:
>>
>> Sure. See below.
>>
>> The view filters documents from all docs, the reduce gives statistics
>> depending on values.
>> Very simple.
>>
>> An interesting thing is that when it breaks vals.length always equals 2.
>>
>> BTW - just happened again, second time today. Compact of the view always
>> fixes this.
>>
>> View:
>>
>> function(doc) {
>> if(doc.status!='deleted' && (!doc.endUser || doc.forwarded) &&
>> !doc.pID) {
>> emit(null, [(doc.status=='new'),(doc.status=='forwarded')]);
>> }
>> }
>>
>> Reduce:
>>
>> function(keys, vals, rereduce) {
>> // reduce function
>>
>> var result = {'new':0, 'total':vals.length,forwarded:0};
>> for(var i = 0; i < vals.length; i++) {
>> result.new += vals[i][0]?1:0;
>> result.forwarded+= vals[i][1]?1:0;
>> }
>> return result;
>> }
>>
>> Thanks!
>>
>>
>> On 28.9.2016 \u0433. 18:16, Jan Lehnardt wrote:
>>> Heya,
>>>
>>> Can you share the view code?
>>>
>>> Best
>>> Jan
>>> --
>>>
>>>> On 28 Sep 2016, at 15:38, Kiril Stankov <ki...@open-net.biz> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I just saw this again.
>>>> Reduce didn't work until compacting the View.
>>>>
>>>> This starts to be a serious problem.
>>>> Any help or suggestion how to debug this will be appreciated.
>>>>
>>>> Thanks!
>>>>
>>>> ------------------------------------------------------------------------
>>>> *With best regards,*
>>>> Kiril Stankov,
>>>>
>>>> On 24.9.2016 \u0433. 20:43, Kiril Stankov wrote:
>>>>> Hi all,
>>>>>
>>>>> I use CouchDb 1.6 and have a view with reduce (same on all databases),
>>>>> which returns some statistics.
>>>>>
>>>>> Suddenly, today, it started returning strange things, and I think that
>>>>> the problem was that vals.length was returning 0. There are only few
>>>>> dozens docs in this DB.
>>>>> I have a replication DB where everything worked fine, checked that the
>>>>> code is exactly the same.
>>>>>
>>>>> After compacting the views on the first DB, it also started showing the
>>>>> correct values.
>>>>>
>>>>> Any idea what could cause this? View corruption? There are plenty of
>>>>> resources (disk, memory) on the host machine.
>>>>>
>>
Re: Issue with Reduce - SOLVED.
Posted by Robert Samuel Newson <rn...@apache.org>.
aaah, ok. :) Well found!
> On 28 Sep 2016, at 23:17, Kiril Stankov <ki...@open-net.biz> wrote:
>
> Hi all,
>
> Please excuse my panic and numerous emails.
> Issue is solved.
> I didn't include in my code if(rereduce) {...}
> Because the output of the first reduce is an object, in needs to be
> processed differently than values passed initially to the reduce.
> And it's all in the manual:
> http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#Reduce_Functions
> (Promise to RTM in the future, and think a bit more before starting to
> panic).
>
> Hope this may be of use for others.
>
> Sorry again, cheers and thanks!
>
>
> On 29.9.2016 г. 0:35, Kiril Stankov wrote:
>> Ok, sorry to bother with this issue, but it becomes a real mystery..
>>
>> So, this code for the reduce:
>>
>> function(keys, vals, rereduce) {
>> /*
>> some commented out for loop
>> */
>> return vals.length;
>> }
>>
>> returns different values for these view code variations:
>>
>> emit(doc._id, [doc.enabled, doc.trialEnabled]); // returns 2
>> emit(doc, [doc.enabled, doc.trialEnabled]); // returns 7
>> emit(null, [doc.enabled, doc.trialEnabled]); // returns 31 - THE
>> CORRECT count.
>>
>> How can the key affect the reduce?
>>
>> The view itself, without the reduce returns the correct docs always!
>>
>> On 28.9.2016 г. 23:37, Kiril Stankov wrote:
>>> Well,
>>>
>>> actually, it got from bad to worst.
>>> Compact view doesn't fix the problem anymore.
>>> Now I always get wrong output.
>>> Modifying the reduce to return just vals.length returns always 2.
>>>
>>> I found that the issue appears after each insertion of new doc to the DB.
>>>
>>> Any help will be highly appreciated, this is a production environment... :(
>>>
>>> ------------------------------------------------------------------------
>>> *With best regards,*
>>> Kiril Stankov,
>>>
>>> On 28.9.2016 г. 22:15, Jan Lehnardt wrote:
>>>> Thanks!
>>>>
>>>> Do you have any more info on the type of operating system this runs on, the file system situation, etc?
>>>>
>>>> Best
>>>> Jan
>>>> --
>>>>
>>>>> On 28 Sep 2016, at 17:47, Kiril Stankov <ki...@open-net.biz> wrote:
>>>>>
>>>>> Sure. See below.
>>>>>
>>>>> The view filters documents from all docs, the reduce gives statistics
>>>>> depending on values.
>>>>> Very simple.
>>>>>
>>>>> An interesting thing is that when it breaks vals.length always equals 2.
>>>>>
>>>>> BTW - just happened again, second time today. Compact of the view always
>>>>> fixes this.
>>>>>
>>>>> View:
>>>>>
>>>>> function(doc) {
>>>>> if(doc.status!='deleted' && (!doc.endUser || doc.forwarded) &&
>>>>> !doc.pID) {
>>>>> emit(null, [(doc.status=='new'),(doc.status=='forwarded')]);
>>>>> }
>>>>> }
>>>>>
>>>>> Reduce:
>>>>>
>>>>> function(keys, vals, rereduce) {
>>>>> // reduce function
>>>>>
>>>>> var result = {'new':0, 'total':vals.length,forwarded:0};
>>>>> for(var i = 0; i < vals.length; i++) {
>>>>> result.new += vals[i][0]?1:0;
>>>>> result.forwarded+= vals[i][1]?1:0;
>>>>> }
>>>>> return result;
>>>>> }
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On 28.9.2016 г. 18:16, Jan Lehnardt wrote:
>>>>>> Heya,
>>>>>>
>>>>>> Can you share the view code?
>>>>>>
>>>>>> Best
>>>>>> Jan
>>>>>> --
>>>>>>
>>>>>>> On 28 Sep 2016, at 15:38, Kiril Stankov <ki...@open-net.biz> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I just saw this again.
>>>>>>> Reduce didn't work until compacting the View.
>>>>>>>
>>>>>>> This starts to be a serious problem.
>>>>>>> Any help or suggestion how to debug this will be appreciated.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>> *With best regards,*
>>>>>>> Kiril Stankov,
>>>>>>>
>>>>>>> On 24.9.2016 г. 20:43, Kiril Stankov wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I use CouchDb 1.6 and have a view with reduce (same on all databases),
>>>>>>>> which returns some statistics.
>>>>>>>>
>>>>>>>> Suddenly, today, it started returning strange things, and I think that
>>>>>>>> the problem was that vals.length was returning 0. There are only few
>>>>>>>> dozens docs in this DB.
>>>>>>>> I have a replication DB where everything worked fine, checked that the
>>>>>>>> code is exactly the same.
>>>>>>>>
>>>>>>>> After compacting the views on the first DB, it also started showing the
>>>>>>>> correct values.
>>>>>>>>
>>>>>>>> Any idea what could cause this? View corruption? There are plenty of
>>>>>>>> resources (disk, memory) on the host machine.
>>>>>>>>
>>
>
>
Re: Issue with Reduce - SOLVED.
Posted by Kiril Stankov <ki...@open-net.biz>.
Hi all,
Please excuse my panic and numerous emails.
Issue is solved.
I didn't include in my code if(rereduce) {...}
Because the output of the first reduce is an object, in needs to be
processed differently than values passed initially to the reduce.
And it's all in the manual:
http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#Reduce_Functions
(Promise to RTM in the future, and think a bit more before starting to
panic).
Hope this may be of use for others.
Sorry again, cheers and thanks!
On 29.9.2016 г. 0:35, Kiril Stankov wrote:
> Ok, sorry to bother with this issue, but it becomes a real mystery..
>
> So, this code for the reduce:
>
> function(keys, vals, rereduce) {
> /*
> some commented out for loop
> */
> return vals.length;
> }
>
> returns different values for these view code variations:
>
> emit(doc._id, [doc.enabled, doc.trialEnabled]); // returns 2
> emit(doc, [doc.enabled, doc.trialEnabled]); // returns 7
> emit(null, [doc.enabled, doc.trialEnabled]); // returns 31 - THE
> CORRECT count.
>
> How can the key affect the reduce?
>
> The view itself, without the reduce returns the correct docs always!
>
> On 28.9.2016 г. 23:37, Kiril Stankov wrote:
>> Well,
>>
>> actually, it got from bad to worst.
>> Compact view doesn't fix the problem anymore.
>> Now I always get wrong output.
>> Modifying the reduce to return just vals.length returns always 2.
>>
>> I found that the issue appears after each insertion of new doc to the DB.
>>
>> Any help will be highly appreciated, this is a production environment... :(
>>
>> ------------------------------------------------------------------------
>> *With best regards,*
>> Kiril Stankov,
>>
>> On 28.9.2016 г. 22:15, Jan Lehnardt wrote:
>>> Thanks!
>>>
>>> Do you have any more info on the type of operating system this runs on, the file system situation, etc?
>>>
>>> Best
>>> Jan
>>> --
>>>
>>>> On 28 Sep 2016, at 17:47, Kiril Stankov <ki...@open-net.biz> wrote:
>>>>
>>>> Sure. See below.
>>>>
>>>> The view filters documents from all docs, the reduce gives statistics
>>>> depending on values.
>>>> Very simple.
>>>>
>>>> An interesting thing is that when it breaks vals.length always equals 2.
>>>>
>>>> BTW - just happened again, second time today. Compact of the view always
>>>> fixes this.
>>>>
>>>> View:
>>>>
>>>> function(doc) {
>>>> if(doc.status!='deleted' && (!doc.endUser || doc.forwarded) &&
>>>> !doc.pID) {
>>>> emit(null, [(doc.status=='new'),(doc.status=='forwarded')]);
>>>> }
>>>> }
>>>>
>>>> Reduce:
>>>>
>>>> function(keys, vals, rereduce) {
>>>> // reduce function
>>>>
>>>> var result = {'new':0, 'total':vals.length,forwarded:0};
>>>> for(var i = 0; i < vals.length; i++) {
>>>> result.new += vals[i][0]?1:0;
>>>> result.forwarded+= vals[i][1]?1:0;
>>>> }
>>>> return result;
>>>> }
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On 28.9.2016 г. 18:16, Jan Lehnardt wrote:
>>>>> Heya,
>>>>>
>>>>> Can you share the view code?
>>>>>
>>>>> Best
>>>>> Jan
>>>>> --
>>>>>
>>>>>> On 28 Sep 2016, at 15:38, Kiril Stankov <ki...@open-net.biz> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I just saw this again.
>>>>>> Reduce didn't work until compacting the View.
>>>>>>
>>>>>> This starts to be a serious problem.
>>>>>> Any help or suggestion how to debug this will be appreciated.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>> *With best regards,*
>>>>>> Kiril Stankov,
>>>>>>
>>>>>> On 24.9.2016 г. 20:43, Kiril Stankov wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I use CouchDb 1.6 and have a view with reduce (same on all databases),
>>>>>>> which returns some statistics.
>>>>>>>
>>>>>>> Suddenly, today, it started returning strange things, and I think that
>>>>>>> the problem was that vals.length was returning 0. There are only few
>>>>>>> dozens docs in this DB.
>>>>>>> I have a replication DB where everything worked fine, checked that the
>>>>>>> code is exactly the same.
>>>>>>>
>>>>>>> After compacting the views on the first DB, it also started showing the
>>>>>>> correct values.
>>>>>>>
>>>>>>> Any idea what could cause this? View corruption? There are plenty of
>>>>>>> resources (disk, memory) on the host machine.
>>>>>>>
>
Re: Issue with Reduce - NOT SOLVED.
Posted by Kiril Stankov <ki...@open-net.biz>.
Ok, sorry to bother with this issue, but it becomes a real mystery..
So, this code for the reduce:
function(keys, vals, rereduce) {
/*
some commented out for loop
*/
return vals.length;
}
returns different values for these view code variations:
emit(doc._id, [doc.enabled, doc.trialEnabled]); // returns 2
emit(doc, [doc.enabled, doc.trialEnabled]); // returns 7
emit(null, [doc.enabled, doc.trialEnabled]); // returns 31 - THE
CORRECT count.
How can the key affect the reduce?
The view itself, without the reduce returns the correct docs always!
On 28.9.2016 г. 23:37, Kiril Stankov wrote:
> Well,
>
> actually, it got from bad to worst.
> Compact view doesn't fix the problem anymore.
> Now I always get wrong output.
> Modifying the reduce to return just vals.length returns always 2.
>
> I found that the issue appears after each insertion of new doc to the DB.
>
> Any help will be highly appreciated, this is a production environment... :(
>
> ------------------------------------------------------------------------
> *With best regards,*
> Kiril Stankov,
>
> On 28.9.2016 г. 22:15, Jan Lehnardt wrote:
>> Thanks!
>>
>> Do you have any more info on the type of operating system this runs on, the file system situation, etc?
>>
>> Best
>> Jan
>> --
>>
>>> On 28 Sep 2016, at 17:47, Kiril Stankov <ki...@open-net.biz> wrote:
>>>
>>> Sure. See below.
>>>
>>> The view filters documents from all docs, the reduce gives statistics
>>> depending on values.
>>> Very simple.
>>>
>>> An interesting thing is that when it breaks vals.length always equals 2.
>>>
>>> BTW - just happened again, second time today. Compact of the view always
>>> fixes this.
>>>
>>> View:
>>>
>>> function(doc) {
>>> if(doc.status!='deleted' && (!doc.endUser || doc.forwarded) &&
>>> !doc.pID) {
>>> emit(null, [(doc.status=='new'),(doc.status=='forwarded')]);
>>> }
>>> }
>>>
>>> Reduce:
>>>
>>> function(keys, vals, rereduce) {
>>> // reduce function
>>>
>>> var result = {'new':0, 'total':vals.length,forwarded:0};
>>> for(var i = 0; i < vals.length; i++) {
>>> result.new += vals[i][0]?1:0;
>>> result.forwarded+= vals[i][1]?1:0;
>>> }
>>> return result;
>>> }
>>>
>>> Thanks!
>>>
>>>
>>> On 28.9.2016 г. 18:16, Jan Lehnardt wrote:
>>>> Heya,
>>>>
>>>> Can you share the view code?
>>>>
>>>> Best
>>>> Jan
>>>> --
>>>>
>>>>> On 28 Sep 2016, at 15:38, Kiril Stankov <ki...@open-net.biz> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I just saw this again.
>>>>> Reduce didn't work until compacting the View.
>>>>>
>>>>> This starts to be a serious problem.
>>>>> Any help or suggestion how to debug this will be appreciated.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>> *With best regards,*
>>>>> Kiril Stankov,
>>>>>
>>>>> On 24.9.2016 г. 20:43, Kiril Stankov wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I use CouchDb 1.6 and have a view with reduce (same on all databases),
>>>>>> which returns some statistics.
>>>>>>
>>>>>> Suddenly, today, it started returning strange things, and I think that
>>>>>> the problem was that vals.length was returning 0. There are only few
>>>>>> dozens docs in this DB.
>>>>>> I have a replication DB where everything worked fine, checked that the
>>>>>> code is exactly the same.
>>>>>>
>>>>>> After compacting the views on the first DB, it also started showing the
>>>>>> correct values.
>>>>>>
>>>>>> Any idea what could cause this? View corruption? There are plenty of
>>>>>> resources (disk, memory) on the host machine.
>>>>>>
>