You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Mike Leddy <mi...@loop.com.br> on 2013/01/24 21:49:05 UTC
Unexpected behavior from compacted databases
Hi,
I first noticed this strange behavior inserting documents to restart
replications that had failed and decided to investigate what I
observed. It is probably best to give a reproducible example.
First for this test I am using 1.2.0
$ curl localhost:5984
{"couchdb":"Welcome","version":"1.2.0"}
I wrote a trivially simple script to write a sequence of the same
documents in ruby (any ruby should work ensuring the couchrest gem
is installed - 'gem install couchrest')
require 'couchrest'
server = CouchRest.new 'http://localhost:5984'
db = server.database!('test')
doc = db.get('test') rescue nil
doc ||= CouchRest::Document.new(_id: 'test')
1.upto(3) do |index|
doc[:index]=index
db.save_doc(doc)
p doc
end
As this is an email I will do all activities using only this script
and curl.
1. make sure we start form zero
$ curl -X DELETE localhost:5984/test
{"ok":true}
or
{"error":"not_found","reason":"missing"}
2. run script
$ ruby testing.rb
#<CouchRest::Document _id: "test", index: 1, _rev: "1-a0814dba083c9453bffa352768a40264">
#<CouchRest::Document _id: "test", index: 2, _rev: "2-85e66c4f3bb95acc2f133183a7ecc461">
#<CouchRest::Document _id: "test", index: 3, _rev: "3-38668f9c9bcfdadffa9c8aca593a3001">
Perfect, document inserted and updated
3. delete the document
$ curl -X DELETE localhost:5984/test/test?rev=3-38668f9c9bcfdadffa9c8aca593a3001
{"ok":true,"id":"test","rev":"4-507344f27f5e55e3e677ae25f1acf587"}
4. run the script again
$ ruby testing.rb
#<CouchRest::Document _id: "test", index: 1, _rev: "5-41c70119c629df492276862b19a990dc">
#<CouchRest::Document _id: "test", index: 2, _rev: "6-9d9ccd6dbedde3022ef683af3cd3793d">
#<CouchRest::Document _id: "test", index: 3, _rev: "7-8b1c05d8e10b031c52d54549354da6f5">
Perfectly normal behavior, all revision numbers in sequence
5 delete the document again
$ curl -X DELETE localhost:5984/test/test?rev=7-8b1c05d8e10b031c52d54549354da6f5
{"ok":true,"id":"test","rev":"8-6065e64dcdcec1acfc0f1fbf331b948e"}
OK, everything is working as expected. Now compact the database:
curl -H "Content-Type: application/json" -X POST http://localhost:5984/test/_compact
{"ok":true}
6 run the script again
$ ruby testing.rb
#<CouchRest::Document _id: "test", index: 1, _rev: "1-a0814dba083c9453bffa352768a40264">
#<CouchRest::Document _id: "test", index: 2, _rev: "2-85e66c4f3bb95acc2f133183a7ecc461">
#<CouchRest::Document _id: "test", index: 3, _rev: "3-38668f9c9bcfdadffa9c8aca593a3001">
It seems as though everything is fine and we are starting from scratch again but ....
7 check if the document exists
$ curl http://localhost:5984/test/test
{"error":"not_found","reason":"deleted"}
Now that is strange, couchdb has responded that the document was created and updated and
it does not exist in the database !!!
8 run the script again
$ ruby testing.rb
#<CouchRest::Document _id: "test", index: 1, _rev: "9-8979d2a7a845bed3dff9c7c161e95a9e">
#<CouchRest::Document _id: "test", index: 2, _rev: "10-49162ab87eb1cca98391006242e05c1f">
#<CouchRest::Document _id: "test", index: 3, _rev: "11-b3012f6391add21ed626dad28a6e2699">
Notice the revision sequence numbers continue on as if before the compaction.
9 check if the document exists
$ curl http://localhost:5984/test/test
{"_id":"test","_rev":"11-b3012f6391add21ed626dad28a6e2699","index":3}
It would seem that the first run of the script after compaction is inserting documents
that already are in the tombstone and the deleted document continues as the latest
version. The second run of the script appends new versions in the way that would be
expected if the compaction had not been done.
For completeness here is a dump of the meta info for this document at the end of the
test:
$ curl -s http://localhost:5984/test/test?meta=true | json_pp
{
"index" : 3,
"_id" : "test",
"_revs_info" : [
{
"status" : "available",
"rev" : "11-b3012f6391add21ed626dad28a6e2699"
},
{
"status" : "available",
"rev" : "10-49162ab87eb1cca98391006242e05c1f"
},
{
"status" : "available",
"rev" : "9-8979d2a7a845bed3dff9c7c161e95a9e"
},
{
"status" : "deleted",
"rev" : "8-6065e64dcdcec1acfc0f1fbf331b948e"
},
{
"status" : "missing",
"rev" : "7-8b1c05d8e10b031c52d54549354da6f5"
},
{
"status" : "missing",
"rev" : "6-9d9ccd6dbedde3022ef683af3cd3793d"
},
{
"status" : "missing",
"rev" : "5-41c70119c629df492276862b19a990dc"
},
{
"status" : "missing",
"rev" : "4-507344f27f5e55e3e677ae25f1acf587"
},
{
"status" : "available",
"rev" : "3-38668f9c9bcfdadffa9c8aca593a3001"
},
{
"status" : "available",
"rev" : "2-85e66c4f3bb95acc2f133183a7ecc461"
},
{
"status" : "available",
"rev" : "1-a0814dba083c9453bffa352768a40264"
}
],
"_rev" : "11-b3012f6391add21ed626dad28a6e2699"
}
Any comments ? Can anyone explain this behavior ?
Thanks in advance,
Mike
Re: Unexpected behavior from compacted databases
Posted by Robert Newson <ro...@gmail.com>.
Couchdb-1415?
On Thursday, January 24, 2013, Mike Leddy wrote:
> Hi,
>
> I first noticed this strange behavior inserting documents to restart
> replications that had failed and decided to investigate what I
> observed. It is probably best to give a reproducible example.
>
> First for this test I am using 1.2.0
>
> $ curl localhost:5984
> {"couchdb":"Welcome","version":"1.2.0"}
>
> I wrote a trivially simple script to write a sequence of the same
> documents in ruby (any ruby should work ensuring the couchrest gem
> is installed - 'gem install couchrest')
>
> require 'couchrest'
>
> server = CouchRest.new 'http://localhost:5984'
> db = server.database!('test')
>
> doc = db.get('test') rescue nil
> doc ||= CouchRest::Document.new(_id: 'test')
>
> 1.upto(3) do |index|
> doc[:index]=index
> db.save_doc(doc)
> p doc
> end
>
> As this is an email I will do all activities using only this script
> and curl.
>
> 1. make sure we start form zero
>
> $ curl -X DELETE localhost:5984/test
> {"ok":true}
> or
> {"error":"not_found","reason":"missing"}
>
> 2. run script
> $ ruby testing.rb
> #<CouchRest::Document _id: "test", index: 1, _rev:
> "1-a0814dba083c9453bffa352768a40264">
> #<CouchRest::Document _id: "test", index: 2, _rev:
> "2-85e66c4f3bb95acc2f133183a7ecc461">
> #<CouchRest::Document _id: "test", index: 3, _rev:
> "3-38668f9c9bcfdadffa9c8aca593a3001">
>
> Perfect, document inserted and updated
>
> 3. delete the document
>
> $ curl -X DELETE
> localhost:5984/test/test?rev=3-38668f9c9bcfdadffa9c8aca593a3001
> {"ok":true,"id":"test","rev":"4-507344f27f5e55e3e677ae25f1acf587"}
>
> 4. run the script again
>
> $ ruby testing.rb
> #<CouchRest::Document _id: "test", index: 1, _rev:
> "5-41c70119c629df492276862b19a990dc">
> #<CouchRest::Document _id: "test", index: 2, _rev:
> "6-9d9ccd6dbedde3022ef683af3cd3793d">
> #<CouchRest::Document _id: "test", index: 3, _rev:
> "7-8b1c05d8e10b031c52d54549354da6f5">
>
> Perfectly normal behavior, all revision numbers in sequence
>
> 5 delete the document again
>
> $ curl -X DELETE
> localhost:5984/test/test?rev=7-8b1c05d8e10b031c52d54549354da6f5
> {"ok":true,"id":"test","rev":"8-6065e64dcdcec1acfc0f1fbf331b948e"}
>
> OK, everything is working as expected. Now compact the database:
>
> curl -H "Content-Type: application/json" -X POST
> http://localhost:5984/test/_compact
> {"ok":true}
>
> 6 run the script again
>
> $ ruby testing.rb
> #<CouchRest::Document _id: "test", index: 1, _rev:
> "1-a0814dba083c9453bffa352768a40264">
> #<CouchRest::Document _id: "test", index: 2, _rev:
> "2-85e66c4f3bb95acc2f133183a7ecc461">
> #<CouchRest::Document _id: "test", index: 3, _rev:
> "3-38668f9c9bcfdadffa9c8aca593a3001">
>
> It seems as though everything is fine and we are starting from scratch
> again but ....
>
> 7 check if the document exists
>
> $ curl http://localhost:5984/test/test
> {"error":"not_found","reason":"deleted"}
>
> Now that is strange, couchdb has responded that the document was created
> and updated and
> it does not exist in the database !!!
>
> 8 run the script again
>
> $ ruby testing.rb
> #<CouchRest::Document _id: "test", index: 1, _rev:
> "9-8979d2a7a845bed3dff9c7c161e95a9e">
> #<CouchRest::Document _id: "test", index: 2, _rev:
> "10-49162ab87eb1cca98391006242e05c1f">
> #<CouchRest::Document _id: "test", index: 3, _rev:
> "11-b3012f6391add21ed626dad28a6e2699">
>
> Notice the revision sequence numbers continue on as if before the
> compaction.
>
> 9 check if the document exists
>
> $ curl http://localhost:5984/test/test
> {"_id":"test","_rev":"11-b3012f6391add21ed626dad28a6e2699","index":3}
>
> It would seem that the first run of the script after compaction is
> inserting documents
> that already are in the tombstone and the deleted document continues as
> the latest
> version. The second run of the script appends new versions in the way that
> would be
> expected if the compaction had not been done.
>
> For completeness here is a dump of the meta info for this document at the
> end of the
> test:
>
> $ curl -s http://localhost:5984/test/test?meta=true | json_pp
> {
> "index" : 3,
> "_id" : "test",
> "_revs_info" : [
> {
> "status" : "available",
> "rev" : "11-b3012f6391add21ed626dad28a6e2699"
> },
> {
> "status" : "available",
> "rev" : "10-49162ab87eb1cca98391006242e05c1f"
> },
> {
> "status" : "available",
> "rev" : "9-8979d2a7a845bed3dff9c7c161e95a9e"
> },
> {
> "status" : "deleted",
> "rev" : "8-6065e64dcdcec1acfc0f1fbf331b948e"
> },
> {
> "status" : "missing",
> "rev" : "7-8b1c05d8e10b031c52d54549354da6f5"
> },
> {
> "status" : "missing",
> "rev" : "6-9d9ccd6dbedde3022ef683af3cd3793d"
> },
> {
> "status" : "missing",
> "rev" : "5-41c70119c629df492276862b19a990dc"
> },
> {
> "status" : "missing",
> "rev" : "4-507344f27f5e55e3e677ae25f1acf587"
> },
> {
> "status" : "available",
> "rev" : "3-38668f9c9bcfdadffa9c8aca593a3001"
> },
> {
> "status" : "available",
> "rev" : "2-85e66c4f3bb95acc2f133183a7ecc461"
> },
> {
> "status" : "available",
> "rev" : "1-a0814dba083c9453bffa352768a40264"
> }
> ],
> "_rev" : "11-b3012f6391add21ed626dad28a6e2699"
> }
>
> Any comments ? Can anyone explain this behavior ?
>
> Thanks in advance,
>
> Mike
>
>
>
>
>
>
>
>