You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Martin Hewitt <ma...@thenoi.se> on 2012/04/20 14:34:15 UTC
Configuring CouchDB/BigCouch for high-memory servers
Hi all,
We're building a BigCouch cluster with three nice, new servers with around 32GB of RAM. These will be dedicated database servers, so all they'll do is run, compute and serve our CouchDB backend.
Looking at them in production, however, they aren't using anywhere near the amount of RAM there is available to them. Is this a conscious design decision - to keep everything on-disk instead of in-memory?
Is there any way CouchDB can be configured in a "memory-hungry" mode? We're not bound by CPU, and would quite like to see if we can unlock some performance gains by using in-memory instead of on-disk data.
Thanks,
Martin
Re: Configuring CouchDB/BigCouch for high-memory servers
Posted by Jason Smith <jh...@iriscouch.com>.
Hi, Martin. I am the author of Procouch however it is largely
vaporware at this time.
I am concluding a semi-forced vacation after a semi-death march :) So
I will be working on projects this week, rested and clear-thinking.
(CentOS 5 support is also on my to-do list.)
Anyway, please submit an issue to the Procouch project indicating what
features are more important to you and I will work to get those done
first. Thanks!
On Fri, Apr 20, 2012 at 12:34 PM, Martin Hewitt <ma...@thenoi.se> wrote:
> Thanks Robert and Dave, that procouch project in particular looks very interesting, we already do much of what it does in our application anyway, but moving that into a standalone daemon has a lot of appeal.
>
> Martin
>
> --
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>
>
> On Friday, 20 April 2012 at 14:28, Dave Cottlehuber wrote:
>
>> On 20 April 2012 14:41, Robert Newson <rn...@apache.org> wrote:
>> > Yes, it's deliberate. BigCouch (and CouchDB) doesn't typically
>> > allocate a lot of memory (though lots of replication jobs would change
>> > that). Having lots of RAM is recommended for the OS's disk cache,
>> > however.
>> >
>> > B.
>>
>> You might be able to use it by pre-heating views, or querying with
>> appropriate include_docs. Jason Smith wrote a script to do that for
>> CouchDB which you could try.
>>
>> http://web.archiveorange.com/archive/v/oW6Dz2vTi9LDuZ92hKwy
>> https://github.com/iriscouch/procouch/blob/master/lib/view_heater.js
>> in https://github.com/iriscouch/procouch
>>
>> A+
>> Dave
>>
>>
>
>
--
Iris Couch
Re: Configuring CouchDB/BigCouch for high-memory servers
Posted by Martin Hewitt <ma...@thenoi.se>.
Thanks Robert and Dave, that procouch project in particular looks very interesting, we already do much of what it does in our application anyway, but moving that into a standalone daemon has a lot of appeal.
Martin
--
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Friday, 20 April 2012 at 14:28, Dave Cottlehuber wrote:
> On 20 April 2012 14:41, Robert Newson <rn...@apache.org> wrote:
> > Yes, it's deliberate. BigCouch (and CouchDB) doesn't typically
> > allocate a lot of memory (though lots of replication jobs would change
> > that). Having lots of RAM is recommended for the OS's disk cache,
> > however.
> >
> > B.
>
> You might be able to use it by pre-heating views, or querying with
> appropriate include_docs. Jason Smith wrote a script to do that for
> CouchDB which you could try.
>
> http://web.archiveorange.com/archive/v/oW6Dz2vTi9LDuZ92hKwy
> https://github.com/iriscouch/procouch/blob/master/lib/view_heater.js
> in https://github.com/iriscouch/procouch
>
> A+
> Dave
>
>
Re: Configuring CouchDB/BigCouch for high-memory servers
Posted by Dave Cottlehuber <da...@muse.net.nz>.
On 20 April 2012 14:41, Robert Newson <rn...@apache.org> wrote:
> Yes, it's deliberate. BigCouch (and CouchDB) doesn't typically
> allocate a lot of memory (though lots of replication jobs would change
> that). Having lots of RAM is recommended for the OS's disk cache,
> however.
>
> B.
You might be able to use it by pre-heating views, or querying with
appropriate include_docs. Jason Smith wrote a script to do that for
CouchDB which you could try.
http://web.archiveorange.com/archive/v/oW6Dz2vTi9LDuZ92hKwy
https://github.com/iriscouch/procouch/blob/master/lib/view_heater.js
in https://github.com/iriscouch/procouch
A+
Dave
Re: Configuring CouchDB/BigCouch for high-memory servers
Posted by Robert Newson <rn...@apache.org>.
Yes, it's deliberate. BigCouch (and CouchDB) doesn't typically
allocate a lot of memory (though lots of replication jobs would change
that). Having lots of RAM is recommended for the OS's disk cache,
however.
B.
On 20 April 2012 13:34, Martin Hewitt <ma...@thenoi.se> wrote:
> Hi all,
>
> We're building a BigCouch cluster with three nice, new servers with around 32GB of RAM. These will be dedicated database servers, so all they'll do is run, compute and serve our CouchDB backend.
>
> Looking at them in production, however, they aren't using anywhere near the amount of RAM there is available to them. Is this a conscious design decision - to keep everything on-disk instead of in-memory?
>
> Is there any way CouchDB can be configured in a "memory-hungry" mode? We're not bound by CPU, and would quite like to see if we can unlock some performance gains by using in-memory instead of on-disk data.
>
> Thanks,
>
> Martin