[jira] [Commented] (COUCHDB-2886) Concurrent Bulk Updates and Attachments yield Corruption

["Robert Newson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/COUCHDB-2886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014217#comment-15014217 ] 

Robert Newson commented on COUCHDB-2886:
----------------------------------------

Hi,

Are the documents corrupted or just the view?

> Concurrent Bulk Updates and Attachments yield Corruption 
> ---------------------------------------------------------
>
>                 Key: COUCHDB-2886
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-2886
>             Project: CouchDB
>          Issue Type: Bug
>            Reporter: Steven Schaefer
>         Attachments: couch_corrupter.zip
>
>
> I work for Nexidia, and we're evaluating CouchDB for a new system we're developing. I was running some tests and in general trying to push it as hard as I could to help gain experience and set expectations, and I've encountered what appears to be a database corruption issue, when conflicting concurrent batch updates include attachments. I noticed results from one of my views made no sense, and after verifying it was up-to-date, tried running view compaction in an attempt to address that problem. It errored with this:
> {quote}
>     [info] [<0.24269.24>] Compaction started for db: bug idx: _design/dev
>     [info] [<0.24224.24>] 127.0.0.1 - - POST /bug/_compact/dev 202
>     [error] [<0.24269.24>] Duplicate docid `9255f05ad2939b5c487209321c000c79` detected in view group `_design/dev`, database `bug` - This view needs to be rebuilt.
>     [error] [<0.24251.24>] ** Generic server <0.24251.24> terminating 
>     ** Last message in was {'EXIT',<0.24269.24>,
>                                {view_duplicate_id,
>                                    <<"9255f05ad2939b5c487209321c000c79">>}}
>     ** When Server state == {st,<0.24244.24>,couch_mrview_index,<0.24269.24>}
>     ** Reason for termination == 
>     ** unknown_info
> {quote}
> Deleting and then rebuilding the view didn't help, after which view results were still incorrect, and re-running view compaction produced the same error, which suggests to me a more fundamental issue.
> I narrowed it down to a combination of concurrent bulk updates with attachments, or at least I should say I can consistently reproduce when including attachments, but I could never reproduce without them. I have attached a smallish example that usually reproduces the issue on my system.
> Notes for running:
> * Depends on Java 8 and if compiling I set this up to use maven
> * Program will create "bug" database in a Couch instance running at localhost:5984. An existing "bug" db will be deleted.
> * Some variables may need to be altered depending on system specs, look for "NOTE:" in the code. Rebuilding should be as easy as (assuming maven is installed) running from wherever unzipped: mvn install
> * run-test.bat provides the command to run it. Only adjusting the -cp parameter should be necessary for Linux (i.e. ';' --> ':' )
> * Should run fairly quickly, after which "Bug reproduced" should be visible in the console, and the Couch log will have an error like above.
> System Notes:
> * Windows 7
> * 2x Intel Xeon E5-2687W v3 @ 3.10 GHz - 10 cores + hyperthreading (40 logical processors)
> * 32 GB ram (NUMA)
> * CouchDB configuration is default except for db and view storage locations.
> * CouchDB 1.6.1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)