Cleanup of an online database

[Boaz Citrin <bc...@gmail.com>]

Hello,

Having a database with massive writes and updates, its file (and views) get
huge and fragmented.
Even a daily compaction takes few hours, and we cannot allow pausing these
updates as users count on latest information.

So we need some kind of online cleanup...

I wonder it we could somehow copy files to a side database, run compaction,
replicate latest changes and swap databases.

Can anyone recommend a procedure?

Thanks!

Re: Cleanup of an online database

[Benoit Chesneau <bc...@gmail.com>]

On Mon, Sep 30, 2013 at 11:22 AM, Boaz Citrin <bc...@gmail.com> wrote:

> Hello,
>
> Having a database with massive writes and updates, its file (and views) get
> huge and fragmented.
> Even a daily compaction takes few hours, and we cannot allow pausing these
> updates as users count on latest information.


> So we need some kind of online cleanup...
>
> I wonder it we could somehow copy files to a side database, run compaction,
> replicate latest changes and swap databases.
>
> Can anyone recommend a procedure?
>
> Thanks!
>


Replication to a new db will remove any fragmentation but can be
complicated since you switch the dbs for the user.

I would go for small dbs running compaction in background. Maybe you can
shard your content so it can goes in different dbs or use bigcouch to shard
your dbs transparently.

- benoit