[GitHub] wohali commented on issue #758: Compaction fails due to a file corruption

[gi...@git.apache.org]

wohali commented on issue #758: Compaction fails due to a file corruption
URL: https://github.com/apache/couchdb/issues/758#issuecomment-333049515
 
 
   The error is clear - your database has file corruption. The binary data there is showing the nodes in the btree, but a byte-by-byte analysis is not something anyone on the project right now has time to complete.
   
   The most likely scenarios for this are that you are in a virtualised environment where the disk goes away, you have `delayed_commits` set to `true` and a disk is intermittently failing (lesson: ALWAYS set `delayed_commits` to `false`!), or you are using network-attached storage (such as EBS) and the volume has become corrupted. If you're in a cloud/hosted environment, you may want to check with your hosting provider to determine if anything can be done to guarantee disk performance. Another possibility is failing RAM that caused a bit-flip, or (yes, really) cosmic rays doing the same. The typical safeguard for this in production is to only run on servers with ECC RAM.
   
   CouchDB 1.6.1 has been running in the field for many years on hundreds of thousands of servers, and there are no known issues that definitively cause corruption in databases aside from the well-known scenarios described above.
   
   Your best options are:
   
   1. Restore the database from backup prior to when file corruption occurred. (You are backing up your server, aren't you? ? )
   2. Write a custom replication to read documents from the database and skip documents that fail. You may not be able to use the `/{db}/_changes` feed for this if there is corruption in the sequence; try using `/{db}/_all_docs` if this is true
   3. [Neighbourhoodie](https://neighbourhood.ie/) provides a professional (paid) CouchDB database recovery service when databases have been corrupted if you are completely unable to recover the data yourself. You will need to provide them with a zip/tar of the database directory for this service.
   
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services