You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Bill Foshay <bi...@noteandgo.com> on 2014/09/03 21:40:10 UTC

Possible to measure disk space of documents in a view?

Is there a way to measure the amount of disk space used by the documents 
returned by a particular view? As an example, let's say all documents in my 
database are tagged with a userId and I have a view that returns all 
documents by userId. Now I want to measure the amount of space used by 
userId=100? Is this possible?

Thanks,
Bill


Re: Possible to measure disk space of documents in a view?

Posted by Jens Alfke <je...@couchbase.com>.
> On Sep 3, 2014, at 12:51 PM, Guilherme Andrade <g...@gandrade.net> wrote:
> 
> Maybe by finding some reliable / good-enough way to calculate an
> arbitrary document's disk size

It's not hard to walk a JSON object recursively and compute how many bytes it would probably occupy as JSON. Then you'd need to add in the encoded_length of each attachment.

But after that you run into implementation details like: are document bodies stored using some kind of compression (like Snappy)? Are they even stored as JSON at all, vs. serialized Erlang terms?

And what about conflicts — if a doc is in conflict, you really need to add up the size of each conflicting revision, but IIRC you can't access the other revisions from a map function.

—Jens

Re: Possible to measure disk space of documents in a view?

Posted by Guilherme Andrade <g...@gandrade.net>.
Hi Bill,

Maybe by finding some reliable / good-enough way to calculate an
arbitrary document's disk size and emitting such value in the map
function (among whatever else is already being emitted.) And then
calculating that specific sum on reduce. Not sure about attachments though.

In any case, you'd also have to account for the append-only way of
things, which will incur a overhead of up to whatever the compaction
threshold is (my databases and views usually grow to around twice their
real size before they're compacted.)

Cheers,


On 03-09-2014 20:40, Bill Foshay wrote:
> Is there a way to measure the amount of disk space used by the documents 
> returned by a particular view? As an example, let's say all documents in my 
> database are tagged with a userId and I have a view that returns all 
> documents by userId. Now I want to measure the amount of space used by 
> userId=100? Is this possible?
>
> Thanks,
> Bill
>

-- 
Guilherme

https://www.gandrade.net/
PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A  D67A 9330 79B1 35CB 8191