You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@couchdb.apache.org by GitBox <gi...@apache.org> on 2019/03/04 14:29:29 UTC

[GitHub] [couchdb] eiri commented on issue #1470: Fix calculation of external size for attachments

eiri commented on issue #1470: Fix calculation of external size for attachments
URL: https://github.com/apache/couchdb/pull/1470#issuecomment-469272067
 
 
   @AlexanderKaraberov Hey, sorry I somehow completely missed the notification for your comment and only found in because I'm thinking of reviving this PR.
   
   So, yes db `sizes` is a confusing topic, I'll try to guess here to best of my knowledge.
   
   The issue itself is two folded. First, `file` represents the complete disk space a database's shards are taking on a single node, i.e. the value deduplicated by shard, but summed up for all the shards. So if say I have a single shard empty db my `file` is going to be ~8kB (2 headers with `0` padding to 4kB, as you already found out), but for the same empty db with 8 shards the value is going to be ~66kB. But this shouldn't matter much in your case, as you are updating a single shard.
   
   Second, append-only approach is kind of wasteful by design. When a doc got added to the db we are writing attachment body (if presented), doc body, doc info, IDs b+tree node, update seqs b+tree node and the commit's header, some of those augmented with md5 hashes and zero padding. Updating the same doc effectively makes all of this "inactive" and the whole thing got appended again, there are no some smart deduplication or anything. A side note - adding a new doc is a bit different, as we don't need to update _whole_ b+trees, only the changed path, but that again is not your case.
   
   Said all of that 30Mb does seems excessive indeed, I'm going to run some tests to see if I can reproduce and analyze this. Btw, do you happened to have a size of this db after compression?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services