You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Paul Hirst <pa...@sophos.com> on 2011/04/13 15:07:35 UTC
Spontaneous view rebuild after view compaction?
I've had my largest view spontaneously decide to rebuild itself from scratch a couple of times now, starting immediately after a view compaction finishes.
I seem to remember someone pointing at a ticket for this problem a while ago but searching around in Jira I couldn't find it. Does it ring any bells with anyone?
I have a suspicion I'm hitting some kind of race condition whereby if the view is queried while the compaction process is attempting to swap the files over it just triggers a fresh build. I've turned off my regular view touching for the moment and I'm going to see if it happens again but as my view takes over a day to build it's getting pretty frustrating so any suggestions would be really appreciated.
Thanks.
________________________________
Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 991 2418 08.
Re: Spontaneous view rebuild after view compaction?
Posted by Paul Hirst <pa...@sophos.com>.
Thanks for all the feedback.
I finally managed to get the view to compact successfully. I switched
off my automatic view touching script and there were no systems
accessing the view. I think that those changes, plus maybe a bit of
luck, got it all the way through.
After reading the ticket I was tempted to switch off delayed_commits so
see if that helped but in the end I didn't need too.
Thanks.
On Thu, 2011-04-14 at 16:33 +0100, Adam Kocoloski wrote:
> Hi Paul, I think you're referring to https://issues.apache.org/jira/browse/COUCHDB-994
>
> Adam
>
> On Apr 14, 2011, at 8:13 AM, Cory Zue wrote:
>
> > Hi Paul,
> >
> > I encountered a similar issue with my largest view, but it was
> > rebuilding whenever couch restarted. I didn't tie it to compaction
> > because in my app couch restarts more frequently then we compact,
> > however it might have been the same problem. In the end, rewriting the
> > view to make the footprint much smaller seemed to resolve it for me,
> > but it was a very unsatisfying workaround.
> >
> > Cory
> >
> >
> > On Thu, Apr 14, 2011 at 4:48 AM, Robert Newson <ro...@gmail.com> wrote:
> >> Hi Paul,
> >>
> >> I believe this odd behavior has been seen in the wild but I don't
> >> think the cause has been isolated.
> >>
> >> B.
> >>
> >> On 13 April 2011 14:07, Paul Hirst <pa...@sophos.com> wrote:
> >>> I've had my largest view spontaneously decide to rebuild itself from scratch a couple of times now, starting immediately after a view compaction finishes.
> >>>
> >>> I seem to remember someone pointing at a ticket for this problem a while ago but searching around in Jira I couldn't find it. Does it ring any bells with anyone?
> >>>
> >>> I have a suspicion I'm hitting some kind of race condition whereby if the view is queried while the compaction process is attempting to swap the files over it just triggers a fresh build. I've turned off my regular view touching for the moment and I'm going to see if it happens again but as my view takes over a day to build it's getting pretty frustrating so any suggestions would be really appreciated.
> >>>
> >>> Thanks.
> >>>
> >>>
> >>> ________________________________
> >>> Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
> >>> Company Reg No 2096520. VAT Reg No GB 991 2418 08.
> >>>
> >>
>
Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
Company Reg No 2096520. VAT Reg No GB 991 2418 08.
Re: Spontaneous view rebuild after view compaction?
Posted by Adam Kocoloski <ko...@apache.org>.
Hi Paul, I think you're referring to https://issues.apache.org/jira/browse/COUCHDB-994
Adam
On Apr 14, 2011, at 8:13 AM, Cory Zue wrote:
> Hi Paul,
>
> I encountered a similar issue with my largest view, but it was
> rebuilding whenever couch restarted. I didn't tie it to compaction
> because in my app couch restarts more frequently then we compact,
> however it might have been the same problem. In the end, rewriting the
> view to make the footprint much smaller seemed to resolve it for me,
> but it was a very unsatisfying workaround.
>
> Cory
>
>
> On Thu, Apr 14, 2011 at 4:48 AM, Robert Newson <ro...@gmail.com> wrote:
>> Hi Paul,
>>
>> I believe this odd behavior has been seen in the wild but I don't
>> think the cause has been isolated.
>>
>> B.
>>
>> On 13 April 2011 14:07, Paul Hirst <pa...@sophos.com> wrote:
>>> I've had my largest view spontaneously decide to rebuild itself from scratch a couple of times now, starting immediately after a view compaction finishes.
>>>
>>> I seem to remember someone pointing at a ticket for this problem a while ago but searching around in Jira I couldn't find it. Does it ring any bells with anyone?
>>>
>>> I have a suspicion I'm hitting some kind of race condition whereby if the view is queried while the compaction process is attempting to swap the files over it just triggers a fresh build. I've turned off my regular view touching for the moment and I'm going to see if it happens again but as my view takes over a day to build it's getting pretty frustrating so any suggestions would be really appreciated.
>>>
>>> Thanks.
>>>
>>>
>>> ________________________________
>>> Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
>>> Company Reg No 2096520. VAT Reg No GB 991 2418 08.
>>>
>>
Re: Spontaneous view rebuild after view compaction?
Posted by Cory Zue <cz...@dimagi.com>.
Hi Paul,
I encountered a similar issue with my largest view, but it was
rebuilding whenever couch restarted. I didn't tie it to compaction
because in my app couch restarts more frequently then we compact,
however it might have been the same problem. In the end, rewriting the
view to make the footprint much smaller seemed to resolve it for me,
but it was a very unsatisfying workaround.
Cory
On Thu, Apr 14, 2011 at 4:48 AM, Robert Newson <ro...@gmail.com> wrote:
> Hi Paul,
>
> I believe this odd behavior has been seen in the wild but I don't
> think the cause has been isolated.
>
> B.
>
> On 13 April 2011 14:07, Paul Hirst <pa...@sophos.com> wrote:
>> I've had my largest view spontaneously decide to rebuild itself from scratch a couple of times now, starting immediately after a view compaction finishes.
>>
>> I seem to remember someone pointing at a ticket for this problem a while ago but searching around in Jira I couldn't find it. Does it ring any bells with anyone?
>>
>> I have a suspicion I'm hitting some kind of race condition whereby if the view is queried while the compaction process is attempting to swap the files over it just triggers a fresh build. I've turned off my regular view touching for the moment and I'm going to see if it happens again but as my view takes over a day to build it's getting pretty frustrating so any suggestions would be really appreciated.
>>
>> Thanks.
>>
>>
>> ________________________________
>> Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
>> Company Reg No 2096520. VAT Reg No GB 991 2418 08.
>>
>
Re: Spontaneous view rebuild after view compaction?
Posted by Robert Newson <ro...@gmail.com>.
Hi Paul,
I believe this odd behavior has been seen in the wild but I don't
think the cause has been isolated.
B.
On 13 April 2011 14:07, Paul Hirst <pa...@sophos.com> wrote:
> I've had my largest view spontaneously decide to rebuild itself from scratch a couple of times now, starting immediately after a view compaction finishes.
>
> I seem to remember someone pointing at a ticket for this problem a while ago but searching around in Jira I couldn't find it. Does it ring any bells with anyone?
>
> I have a suspicion I'm hitting some kind of race condition whereby if the view is queried while the compaction process is attempting to swap the files over it just triggers a fresh build. I've turned off my regular view touching for the moment and I'm going to see if it happens again but as my view takes over a day to build it's getting pretty frustrating so any suggestions would be really appreciated.
>
> Thanks.
>
>
> ________________________________
> Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, United Kingdom.
> Company Reg No 2096520. VAT Reg No GB 991 2418 08.
>