You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Will Perry <Wi...@microsoft.com> on 2013/11/15 22:17:26 UTC
view_duplicate_id chaos
Hi Folks-
In a fairly large DB (~8GB) we're seeing crashes, compaction failures and index rebuilds as a result of duplicate doc ids. A selection of related traces includes:
[error] [<0.2990.111>] Compaction daemon - an error ocurred while compacting the view group `System` from database `servicebus`: {view_duplicate_id,
<<"test:-vM0xyKZdeSMCIukpTchI3ZeKhk">>}
[error] [<0.7103.113>] Duplicate docid `test:-vM0xyKZdeSMCIukpTchI3ZeKhk` detected in view group `_design/System`, database `servicebus` - This view needs to be rebuilt.
Can anyone help me understand what's happening here? How can we avoid getting into this situation, and is there a best-practice to resolve it?
Cheers,
Will
Re: view_duplicate_id chaos
Posted by Stefan Kögl <ko...@gmail.com>.
Hi,
On Sat, Nov 16, 2013 at 12:29 AM, Robert Newson <rn...@apache.org> wrote:
> This is a bug fixed many releases ago, you need to compact your
> database, then delete your .view files and rebuild them. The release
> notes at the time explained the procedure. I believe this is
> COUCHDB-968 / COUCHDB-999.
>
> https://issues.apache.org/jira/browse/COUCHDB-968
>
> https://issues.apache.org/jira/browse/COUCHDB-999
>
>
I'm not sure if this is completely solved yet. I'm seeing the issue from
time to time also with recent versions
https://issues.apache.org/jira/browse/COUCHDB-1769
-- Stefan
RE: view_duplicate_id chaos
Posted by Will Perry <Wi...@microsoft.com>.
Awesome - thanks for the hint.
I'll give this a go over the weekend and let you know how it goes.
Cheers,
Will
-----Original Message-----
From: Robert Newson [mailto:rnewson@apache.org]
Sent: Friday, November 15, 2013 3:30 PM
To: user@couchdb.apache.org
Subject: Re: view_duplicate_id chaos
This is a bug fixed many releases ago, you need to compact your database, then delete your .view files and rebuild them. The release notes at the time explained the procedure. I believe this is
COUCHDB-968 / COUCHDB-999.
https://issues.apache.org/jira/browse/COUCHDB-968
https://issues.apache.org/jira/browse/COUCHDB-999
B.
On 15 November 2013 21:44, Will Perry <Wi...@microsoft.com> wrote:
> I'm wondering how duplicate doc ids could have got there, tho- how is it even possible for 2 docs to have the same id? Shouldn't the second PUT result in a 409?
>
> If the error message is correct (and lets just assume that it is) then
> we have duplicate doc ids. While we can delete and rebuild the views,
> this takes about 8 hours and isn't something we can afford to be doing
> continuously in production :)
>
> I'd like to try and figure out the root cause for this problem and try to apply a fix to prevent it occurring again in the future - any pointers?
>
> Cheers,
>
> Will
>
> -----Original Message-----
> From: Tim Tisdall [mailto:tisdall@gmail.com]
> Sent: Friday, November 15, 2013 1:31 PM
> To: user@couchdb.apache.org
> Subject: Re: view_duplicate_id chaos
>
> Do you have duplicate doc ids in your data??
>
> I think the view is just corrupted and needs to be deleted and recreated from scratch.
>
> -Tim
>
> On Fri, Nov 15, 2013 at 4:17 PM, Will Perry <Wi...@microsoft.com> wrote:
>> Hi Folks-
>>
>> In a fairly large DB (~8GB) we're seeing crashes, compaction failures and index rebuilds as a result of duplicate doc ids. A selection of related traces includes:
>>
>> [error] [<0.2990.111>] Compaction daemon - an error ocurred while
>> compacting the view group `System` from database `servicebus`:
>> {view_duplicate_id,
>>
>> <<"test:-vM0xyKZdeSMCIukpTchI3ZeKhk">>}
>>
>> [error] [<0.7103.113>] Duplicate docid `test:-vM0xyKZdeSMCIukpTchI3ZeKhk` detected in view group `_design/System`, database `servicebus` - This view needs to be rebuilt.
>>
>>
>> Can anyone help me understand what's happening here? How can we avoid getting into this situation, and is there a best-practice to resolve it?
>>
>> Cheers,
>>
>> Will
Re: view_duplicate_id chaos
Posted by Robert Newson <rn...@apache.org>.
This is a bug fixed many releases ago, you need to compact your
database, then delete your .view files and rebuild them. The release
notes at the time explained the procedure. I believe this is
COUCHDB-968 / COUCHDB-999.
https://issues.apache.org/jira/browse/COUCHDB-968
https://issues.apache.org/jira/browse/COUCHDB-999
B.
On 15 November 2013 21:44, Will Perry <Wi...@microsoft.com> wrote:
> I'm wondering how duplicate doc ids could have got there, tho- how is it even possible for 2 docs to have the same id? Shouldn't the second PUT result in a 409?
>
> If the error message is correct (and lets just assume that it is) then we have duplicate doc ids. While we can delete and rebuild the views, this takes about 8 hours and isn't something we can afford to be doing continuously in production :)
>
> I'd like to try and figure out the root cause for this problem and try to apply a fix to prevent it occurring again in the future - any pointers?
>
> Cheers,
>
> Will
>
> -----Original Message-----
> From: Tim Tisdall [mailto:tisdall@gmail.com]
> Sent: Friday, November 15, 2013 1:31 PM
> To: user@couchdb.apache.org
> Subject: Re: view_duplicate_id chaos
>
> Do you have duplicate doc ids in your data??
>
> I think the view is just corrupted and needs to be deleted and recreated from scratch.
>
> -Tim
>
> On Fri, Nov 15, 2013 at 4:17 PM, Will Perry <Wi...@microsoft.com> wrote:
>> Hi Folks-
>>
>> In a fairly large DB (~8GB) we're seeing crashes, compaction failures and index rebuilds as a result of duplicate doc ids. A selection of related traces includes:
>>
>> [error] [<0.2990.111>] Compaction daemon - an error ocurred while compacting the view group `System` from database `servicebus`: {view_duplicate_id,
>>
>> <<"test:-vM0xyKZdeSMCIukpTchI3ZeKhk">>}
>>
>> [error] [<0.7103.113>] Duplicate docid `test:-vM0xyKZdeSMCIukpTchI3ZeKhk` detected in view group `_design/System`, database `servicebus` - This view needs to be rebuilt.
>>
>>
>> Can anyone help me understand what's happening here? How can we avoid getting into this situation, and is there a best-practice to resolve it?
>>
>> Cheers,
>>
>> Will
RE: view_duplicate_id chaos
Posted by Will Perry <Wi...@microsoft.com>.
I'm wondering how duplicate doc ids could have got there, tho- how is it even possible for 2 docs to have the same id? Shouldn't the second PUT result in a 409?
If the error message is correct (and lets just assume that it is) then we have duplicate doc ids. While we can delete and rebuild the views, this takes about 8 hours and isn't something we can afford to be doing continuously in production :)
I'd like to try and figure out the root cause for this problem and try to apply a fix to prevent it occurring again in the future - any pointers?
Cheers,
Will
-----Original Message-----
From: Tim Tisdall [mailto:tisdall@gmail.com]
Sent: Friday, November 15, 2013 1:31 PM
To: user@couchdb.apache.org
Subject: Re: view_duplicate_id chaos
Do you have duplicate doc ids in your data??
I think the view is just corrupted and needs to be deleted and recreated from scratch.
-Tim
On Fri, Nov 15, 2013 at 4:17 PM, Will Perry <Wi...@microsoft.com> wrote:
> Hi Folks-
>
> In a fairly large DB (~8GB) we're seeing crashes, compaction failures and index rebuilds as a result of duplicate doc ids. A selection of related traces includes:
>
> [error] [<0.2990.111>] Compaction daemon - an error ocurred while compacting the view group `System` from database `servicebus`: {view_duplicate_id,
>
> <<"test:-vM0xyKZdeSMCIukpTchI3ZeKhk">>}
>
> [error] [<0.7103.113>] Duplicate docid `test:-vM0xyKZdeSMCIukpTchI3ZeKhk` detected in view group `_design/System`, database `servicebus` - This view needs to be rebuilt.
>
>
> Can anyone help me understand what's happening here? How can we avoid getting into this situation, and is there a best-practice to resolve it?
>
> Cheers,
>
> Will
Re: view_duplicate_id chaos
Posted by Tim Tisdall <ti...@gmail.com>.
Do you have duplicate doc ids in your data??
I think the view is just corrupted and needs to be deleted and
recreated from scratch.
-Tim
On Fri, Nov 15, 2013 at 4:17 PM, Will Perry <Wi...@microsoft.com> wrote:
> Hi Folks-
>
> In a fairly large DB (~8GB) we're seeing crashes, compaction failures and index rebuilds as a result of duplicate doc ids. A selection of related traces includes:
>
> [error] [<0.2990.111>] Compaction daemon - an error ocurred while compacting the view group `System` from database `servicebus`: {view_duplicate_id,
> <<"test:-vM0xyKZdeSMCIukpTchI3ZeKhk">>}
>
> [error] [<0.7103.113>] Duplicate docid `test:-vM0xyKZdeSMCIukpTchI3ZeKhk` detected in view group `_design/System`, database `servicebus` - This view needs to be rebuilt.
>
>
> Can anyone help me understand what's happening here? How can we avoid getting into this situation, and is there a best-practice to resolve it?
>
> Cheers,
>
> Will