You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by "Matt Goodall (JIRA)" <ji...@apache.org> on 2010/06/10 15:42:13 UTC
[jira] Created: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Changes not written if server shutdown during delayed_commits period
--------------------------------------------------------------------
Key: COUCHDB-791
URL: https://issues.apache.org/jira/browse/COUCHDB-791
Project: CouchDB
Issue Type: Bug
Affects Versions: 0.11.1
Environment: Linux (Ubuntu 10.04)
Reporter: Matt Goodall
If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
Simple script to demonstrate the problem is:
db=http://localhost:5984/scratch
curl $db -X DELETE
curl $db -X PUT
curl $db -X POST -d '{}'
/path/to/couchdb/bin/couchdb -d
When couchdb is started again the database is empty.
Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Damien Katz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878667#action_12878667 ]
Damien Katz commented on COUCHDB-791:
-------------------------------------
Sleeping while waiting doesn't give a guarantee. On a heavily loaded server, it could take many seconds to completely flush everything.
If you want to ensure your data is to disk, use full commits. Nothing else gives any guarantees.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Sebastian Cohnen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878589#action_12878589 ]
Sebastian Cohnen commented on COUCHDB-791:
------------------------------------------
@chris: when couchdb -d calls /_ensure_full_commit before it kills the pid, how to avoid on high traffic use cases that there are no more inserts/updates while or after _ensure_full_commit is done and BEFORE the couchdb process is killed? I think if there is such a mechanism, it should work reliably in all cases. Otherwise the current approach of killing the process and hope the best is already sufficient. (for scheduled maintenance there are ways to gracefully shut down couch, like blocking further incoming requests and wait until all updates are finished. but you have to hack it yourself at the moment)
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878639#action_12878639 ]
Chris Anderson commented on COUCHDB-791:
----------------------------------------
On IRC we discussed this and came up with 2 robust options:
1 is to have `couchdb -d` configure the server to delayed_commits=false for a few seconds before shutdown
2 is to call an os level `sync` command after shutdown, to ensure that anything written but not flushed, is flushed.
I don't know enough about 2, but if it is actually robust, it might be simpler than 1.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878646#action_12878646 ]
Robert Newson commented on COUCHDB-791:
---------------------------------------
Fwiw, my position has shifted a little. I think it's a just a little too surprising that a write followed by a /etc/init.d/couchdb stop leads to data loss with the default settings.
That is, since delayed_commits=true is the default, many people might not be conscious of the small window of non-durability. Flipping the default eliminates that concern but then makes out-of-box performance fairly dire (which is why delayed_commit was flipped on by default, iirc).
A defensible middle ground is for /etc/init.d/couchdb/stop (the action performed when shutting a machine down cleanly for h/w maintenance or kernel upgrade, say) to flush pending writes to disk rather than discard them.
I think the safest pattern was, during the 'stop' script call;
1) PUT /_config/couchdb/delayed_commits -d "false"
2) call global sync() (flush all data to all databases)
That way any write between 1 and 2 will also be on disk. No 201 response will be contradicted when the service starts on reboot.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878662#action_12878662 ]
Chris Anderson commented on COUCHDB-791:
----------------------------------------
or
1) flip delayed_commits to false
2) sleep for 2 seconds (1 second is the commit delay)
3) kill the pid
I think this does it. I could be wrong
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Matt Goodall (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877422#action_12877422 ]
Matt Goodall commented on COUCHDB-791:
--------------------------------------
I'm really, really surprised by this response. You're saying that it's ok to lose data if a CouchDB instance is cleanly shutdown?
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (COUCHDB-791) Changes not written if server shutdown
during delayed_commits period
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Anderson closed COUCHDB-791.
----------------------------------
Resolution: Not A Problem
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Damien Katz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878642#action_12878642 ]
Damien Katz commented on COUCHDB-791:
-------------------------------------
Neither of those options is guaranteed to fully write to disk. This just isn't how CouchDB is designed. It's crash-only, never has a shutdown phase, and if it detects an internal error, it immediately will crash.
If you need things to be guaranteed committed, then don't use delayed commits. Otherwise you are always at risk of losing the most recent updates.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877415#action_12877415 ]
Chris Anderson commented on COUCHDB-791:
----------------------------------------
This is the expected behavior. To avoid this, turn delayed_commits off.
In my experience CouchDB has higher throughput under concurrent load without delayed_commits, so they are really only a performance aid in the single-user or naive-benchmark use case.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Benoit Chesneau (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877423#action_12877423 ]
Benoit Chesneau commented on COUCHDB-791:
-----------------------------------------
well not sure this should be an expected behaviour. shutdown != crash. I would expect a graceful shutdown so delayed commit are saved on normal shutdown (couchdb -d is one). Maybe be we could just apply delayed commit on shutdown ?
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877418#action_12877418 ]
Robert Newson commented on COUCHDB-791:
---------------------------------------
Just my opinion but I think this is the correct behavior. Keeping to the "crash only" shutdown behavior is desirable, as soon as couchdb starts caring about clean vs. unclean shutdown, we'll introduce complexity best avoided.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878648#action_12878648 ]
Randall Leeds commented on COUCHDB-791:
---------------------------------------
If we have a global sync command it needs to do a header write and a sync. I believe just flipping delayed_commits to false doesn't trigger a header commit until the next write comes in. Anything that is written with the sync might still be inaccessible from the btree root.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878645#action_12878645 ]
Chris Anderson commented on COUCHDB-791:
----------------------------------------
but wouldn't option 1 do the trick? assuming there isn't a crash just after `couchdb -d` is invoked.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877426#action_12877426 ]
Chris Anderson commented on COUCHDB-791:
----------------------------------------
We even have a test that the last second of updates are lost here:
http://svn.apache.org/repos/asf/couchdb/trunk/share/www/script/test/delayed_commits.js
I'm wary of introducing the concept of a normal shutdown. How often do you shutdown a live server? I'm guessing it's something you have never done. Servers go down because they run out of memory, or get struck by lightning.
Outside of your test scenario, it is hard to see when a server under load would ever be shutdown cleanly. Which is why it doesn't make sense to have a clean shutdown mode.
It is trivial to configure couchdb to force a commit for every write, and as I mentioned before, under concurrent load, it is much faster. Under single user load it is dog slow (10-ish writes per second) which is why we ship with delayed_commits on by default.
I think couchdb -d is just a shell script, so if it called /_ensure_full_commit before it kills the pid, that'd be OK. But core CouchDB shouldn't have shutdown code in it.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877424#action_12877424 ]
Robert Newson commented on COUCHDB-791:
---------------------------------------
Losing data that had reached the disk would, of course, be a scandal, but that's not the case here.
CouchDB does not 'shutdown', you just terminate the process. I think it's an important design choice but clearly its consequences aren't spelled out clearly enough.
I'm familiar with other systems that distinguish 'clean' from 'dirty' shutdowns and the complexity that brings is considerable. Worse, developers and users come to depend on it, meaning that dirty shutdowns are rarely tested, which then leads to genuine data loss.
Since this is a closed ticket, perhaps a new @dev thread is appropriate.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Chris Anderson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878678#action_12878678 ]
Chris Anderson commented on COUCHDB-791:
----------------------------------------
Randall,
That's not a bad idea.
Apache CouchDB has started. Time to relax.
Notice: You should configure delayed_commits=false for better durability. delayed_commits=true is faster for a single user, but doesn't have a performance benefit under concurrent load. More info: http://wiki.apache.org/page/about/delayed_commit
Something like this ^
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Randall Leeds (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878672#action_12878672 ]
Randall Leeds commented on COUCHDB-791:
---------------------------------------
I just had an idea I mentioned in IRC, and I think I like it.
How about just throwing a message into the startup console and into futon a la "Admin Party!" advising people to turn it off when they plan to put things in production? Might be a little confusing. There's the chance that people might accuse CouchDB of being sneaky to improve benchmarks, but really, that's kinda what delayed_commits is doing. A big message about it, however, is far from sneaky.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Robert Newson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878657#action_12878657 ]
Robert Newson commented on COUCHDB-791:
---------------------------------------
The global sync I referred to is the normal POSIX one that flushes all pending data to disk. But, yes, it wouldn't be be able to flush things that haven't been written :)
so it's;
1) flip delayed_commits to false
2) call _ensure_full_commit on all active databases.
and now I'm back to hating it again.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (COUCHDB-791) Changes not written if server
shutdown during delayed_commits period
Posted by "Jan Lehnardt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/COUCHDB-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878586#action_12878586 ]
Jan Lehnardt commented on COUCHDB-791:
--------------------------------------
The problem with teaching `couchdb -d` to do a flush is that it might need to be taught about the server admin, but that seems solvable. I agree that this shouldn't be in core CouchDB, but rather in the shutdown script.
> Changes not written if server shutdown during delayed_commits period
> --------------------------------------------------------------------
>
> Key: COUCHDB-791
> URL: https://issues.apache.org/jira/browse/COUCHDB-791
> Project: CouchDB
> Issue Type: Bug
> Affects Versions: 0.11.1
> Environment: Linux (Ubuntu 10.04)
> Reporter: Matt Goodall
>
> If the couchdb server is shutdown (couchdb -d, Ctrl+C at the console, etc) during the delayed commits period then buffered updates are lost.
> Simple script to demonstrate the problem is:
> db=http://localhost:5984/scratch
> curl $db -X DELETE
> curl $db -X PUT
> curl $db -X POST -d '{}'
> /path/to/couchdb/bin/couchdb -d
> When couchdb is started again the database is empty.
> Affects 0.11.x and trunk branches.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.