Re: [user] Couch as a mail store?

[Wout Mertens <wm...@cisco.com>]

On Feb 12, 2009, at 3:13 PM, Kenneth Kalmer wrote:

> Normal filesystem-based storage of mail has other issues:
>
> * Messages often smaller than ethernet jumbo frames, so limited  
> throughput
> (couch can overcome this by bundling messages in a single response)

wow that's really specialized. Is that really an issue?

> * Mostly limited by disk IO and clever tricks around solid state  
> drive usage
> or stripping excessively fast disks

Hmmm makes me wonder - CouchDB doesn't change the position of received  
documents on disk, so retrieving a view could mean many disk accesses.

Is that correct? Is that something all databases suffer from? I  
suppose the only answer for that is caching?

Wout.

Re: [user] Couch as a mail store?

[Kenneth Kalmer <ke...@gmail.com>]

On Thu, Feb 12, 2009 at 4:59 PM, Jan Lehnardt <ja...@apache.org> wrote:

>
> On 12 Feb 2009, at 15:50, Wout Mertens wrote:
>
>  On Feb 12, 2009, at 3:13 PM, Kenneth Kalmer wrote:
>>
>>  Normal filesystem-based storage of mail has other issues:
>>>
>>> * Messages often smaller than ethernet jumbo frames, so limited
>>> throughput
>>> (couch can overcome this by bundling messages in a single response)
>>>
>>
>> wow that's really specialized. Is that really an issue?
>
>
If you're using NFS or GFS, yes, it becomes any issue. We had discussions
with EMC (storage company) a while ago, and one of their supreme techies
(not used lightly, got his doctorate and contributes to the NFS project)
admitted that no matter how great their products are, they're useless for
mail systems where the messages are stored in individual files...


>>  * Mostly limited by disk IO and clever tricks around solid state drive
>>> usage
>>> or stripping excessively fast disks
>>>
>>
>> Hmmm makes me wonder - CouchDB doesn't change the position of received
>> documents on disk, so retrieving a view could mean many disk accesses.
>>
>> Is that correct? Is that something all databases suffer from? I suppose
>> the only answer for that is caching?
>>
>
> The file and b-tree structure is actually made to require a minimal
> number of disk head seeks (which is what makes disks slow) and
> in a typical usage scenario, upper nodes of the btree are cached
> in the filesystem buffer cache too, so there's another speedup.
>

This is great news. I'll pickup on your initial reply in a second.

-- 
Kenneth Kalmer
kenneth.kalmer@gmail.com
http://opensourcery.co.za

Re: [user] Couch as a mail store?

[Jan Lehnardt <ja...@apache.org>]

On 12 Feb 2009, at 15:50, Wout Mertens wrote:

> On Feb 12, 2009, at 3:13 PM, Kenneth Kalmer wrote:
>
>> Normal filesystem-based storage of mail has other issues:
>>
>> * Messages often smaller than ethernet jumbo frames, so limited  
>> throughput
>> (couch can overcome this by bundling messages in a single response)
>
> wow that's really specialized. Is that really an issue?
>
>> * Mostly limited by disk IO and clever tricks around solid state  
>> drive usage
>> or stripping excessively fast disks
>
> Hmmm makes me wonder - CouchDB doesn't change the position of  
> received documents on disk, so retrieving a view could mean many  
> disk accesses.
>
> Is that correct? Is that something all databases suffer from? I  
> suppose the only answer for that is caching?

The file and b-tree structure is actually made to require a minimal
number of disk head seeks (which is what makes disks slow) and
in a typical usage scenario, upper nodes of the btree are cached
in the filesystem buffer cache too, so there's another speedup.

Cheers
Jan
--