Couchdb 1.x and 2.x partitions

[Michael Power <Mi...@ELOTOUCH.com>]

Hello,

I have a use case where I am building up a lot of read only data for specific time periods.  I might bucket each dataset daily, and then after a certain number of days I want to throw away the data.  I imagine doing a delete of each record that is old will be expensive in execution and also create a lot of versions inside the database slowing down the database as a whole and driving the need to do a compaction.  Does couchdb have a way to partition the data, so that I can do a bulk delete of that data quickly without degrading the storage of the database with unused sections?

At this point in time, I can only think of creating a new database that is dated.  Insert into it, then when it expires delete the whole database.  This forces the clients to be aware of the partitions, they would have to query the correct database given the date range they are interested in.  Is there a way I can hide the partitions from the clients?

Michael Power

Re: Couchdb 1.x and 2.x partitions

[Dave Cottlehuber <dc...@skunkwerks.at>]

On Wed, 3 Feb 2016, at 10:40 PM, Michael Power wrote:
> Hello,
> 
> I have a use case where I am building up a lot of read only data for
> specific time periods.  I might bucket each dataset daily, and then after
> a certain number of days I want to throw away the data.  I imagine doing
> a delete of each record that is old will be expensive in execution and
> also create a lot of versions inside the database slowing down the
> database as a whole and driving the need to do a compaction.  Does
> couchdb have a way to partition the data, so that I can do a bulk delete
> of that data quickly without degrading the storage of the database with
> unused sections?
> 
> At this point in time, I can only think of creating a new database that
> is dated.  Insert into it, then when it expires delete the whole
> database.

Spot on, this is a common pattern.

> This forces the clients to be aware of the partitions, they
> would have to query the correct database given the date range they are
> interested in.  Is there a way I can hide the partitions from the
> clients?
> 
> Michael Power

Not directly. Any user could try to query the old name and should get a
response back. You can change the validation docs or user roles, for
example, to protect the data & make it inaccessible, or do weird things
with a proxy but none of this is couch-native.

A+
Dave