View index compaction Re: CouchDB 0.9 and 1.0

[Damien Katz <da...@gmail.com>]

Right now, view indexes just grow and grow with each new index update.  
Since they are just indexes, and not the data itself, compaction is  
simply a matter of deleting the index files.

Also, the current Btree implementation isn't completely self  
balanacing. It misses a balancing condition, partially for efficiency  
(it's an expensive balancing operation), and for expediency. It was  
easier to not implement it and gets the general case perormance boost.

The thing about this is, the btree code can remain as is if the  
indexing compaction just recopies the map values (and back indexes)  
and recomputes the reduction values. That's a very simple design,  
however, if the btree is completely self balancing, then the btree can  
be copied on a node by node basis, instead of a value by value basis,  
and the reduction values need not be recomputed all. This will make  
the compaction significantly faster overall.
On Jul 2, 2008, at 3:08 AM, Jan Lehnardt wrote:

> Hello everybody,
> this thread is meant to collect missing work items (features and
> bugs) for for our 1.0 release and a discussion about how to split
> them up between 0.9 and 1.0.
>
> Take it away: Damien.
>
> Cheers
> Jan
> --

Re: View index compaction Re: CouchDB 0.9 and 1.0

[Jan Lehnardt <ja...@apache.org>]

I opened a bug report for this.

On Jul 2, 2008, at 17:57 , Damien Katz wrote:

> Right now, view indexes just grow and grow with each new index  
> update. Since they are just indexes, and not the data itself,  
> compaction is simply a matter of deleting the index files.
>
> Also, the current Btree implementation isn't completely self  
> balanacing. It misses a balancing condition, partially for  
> efficiency (it's an expensive balancing operation), and for  
> expediency. It was easier to not implement it and gets the general  
> case perormance boost.
>
> The thing about this is, the btree code can remain as is if the  
> indexing compaction just recopies the map values (and back indexes)  
> and recomputes the reduction values. That's a very simple design,  
> however, if the btree is completely self balancing, then the btree  
> can be copied on a node by node basis, instead of a value by value  
> basis, and the reduction values need not be recomputed all. This  
> will make the compaction significantly faster overall.
> On Jul 2, 2008, at 3:08 AM, Jan Lehnardt wrote:
>
>> Hello everybody,
>> this thread is meant to collect missing work items (features and
>> bugs) for for our 1.0 release and a discussion about how to split
>> them up between 0.9 and 1.0.
>>
>> Take it away: Damien.
>>
>> Cheers
>> Jan
>> --
>
>