You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Monica Razdan <mo...@almende.org> on 2011/09/19 22:36:56 UTC
Caching in BigCouch
Hello,
I wanted to compare the caching mechanism used in Hadoop - LRU with
the caching in BigCouch. However, I couldn't find details about
underlying caching algorithm used in Bigcouch. Could someone please
direct me to the right resource?
Thanks,
monica
Re: Caching in BigCouch
Posted by Monica Razdan <mo...@almende.org>.
Julien thanks for your reply. Your experience helps me to reflect back on my use case.
Robert, I'm investigating the possibilities of scaling couchdb using BigCouch. However, I haven't been able to find documents or resources about the performance and scaling benchmarks?
Svein,
Thanks for your post! I'm investigating couchbase server as well at this moment.
Has anyone ran some performance and/or scaling test on bigCouch or Couchbase server?
On Tue, Sep 20, 2011 at 9:40 AM, Julien Gagnet <ju...@gmail.com> wrote:
> No. Couchdb and Bigcouch don't provide any caching. It relies on disk caching.
>
> So even if you see the couch process taking little memory, that
> doesn't mean you won't get any improvement in adding more memory to
> the box. The disk cache use a sophisticated LRU algorithm and should
> provide what you're after. We recently were hitted by a performance
> issue and using local disk wasn't an option, we've been able to
> increase the memory up to a point where most of the db fit in the disk
> caching and solve our problem.
>
> Julien
>
> On 19 September 2011 22:15, Monica Razdan <mo...@almende.org> wrote:
>>
>> I'm my quest to find clustering for couchdb using hadoop I stumbled upon this blog
>>
>> http://mpouttuclarke.wordpress.com/2011/01/04/couchdb-and-hadoop/
>>
>> The author mentions here about LRU caching being used by hadoop. I have a huge dataset sorted on timestamp and caching on timestamp like LRU would be important for my use case. Does bigCouch offers such mechanism under it's hood ?
>>
>> Monica
>>
>>
>>
>> On Sep 19, 2011, at 10:43 PM, Robert Newson <ro...@gmail.com> wrote:
>>
>>> What do you mean by 'caching' here?
>>>
>>> On 19 September 2011 21:36, Monica Razdan <mo...@almende.org> wrote:
>>>> Hello,
>>>>
>>>> I wanted to compare the caching mechanism used in Hadoop - LRU with
>>>> the caching in BigCouch. However, I couldn't find details about
>>>> underlying caching algorithm used in Bigcouch. Could someone please
>>>> direct me to the right resource?
>>>>
>>>> Thanks,
>>>> monica
>>>>
>
Re: Caching in BigCouch
Posted by Julien Gagnet <ju...@gmail.com>.
No. Couchdb and Bigcouch don't provide any caching. It relies on disk caching.
So even if you see the couch process taking little memory, that
doesn't mean you won't get any improvement in adding more memory to
the box. The disk cache use a sophisticated LRU algorithm and should
provide what you're after. We recently were hitted by a performance
issue and using local disk wasn't an option, we've been able to
increase the memory up to a point where most of the db fit in the disk
caching and solve our problem.
Julien
On 19 September 2011 22:15, Monica Razdan <mo...@almende.org> wrote:
>
> I'm my quest to find clustering for couchdb using hadoop I stumbled upon this blog
>
> http://mpouttuclarke.wordpress.com/2011/01/04/couchdb-and-hadoop/
>
> The author mentions here about LRU caching being used by hadoop. I have a huge dataset sorted on timestamp and caching on timestamp like LRU would be important for my use case. Does bigCouch offers such mechanism under it's hood ?
>
> Monica
>
>
>
> On Sep 19, 2011, at 10:43 PM, Robert Newson <ro...@gmail.com> wrote:
>
> > What do you mean by 'caching' here?
> >
> > On 19 September 2011 21:36, Monica Razdan <mo...@almende.org> wrote:
> >> Hello,
> >>
> >> I wanted to compare the caching mechanism used in Hadoop - LRU with
> >> the caching in BigCouch. However, I couldn't find details about
> >> underlying caching algorithm used in Bigcouch. Could someone please
> >> direct me to the right resource?
> >>
> >> Thanks,
> >> monica
> >>
Re: Caching in BigCouch
Posted by Monica Razdan <mo...@almende.org>.
I'm my quest to find clustering for couchdb using hadoop I stumbled upon this blog
http://mpouttuclarke.wordpress.com/2011/01/04/couchdb-and-hadoop/
The author mentions here about LRU caching being used by hadoop. I have a huge dataset sorted on timestamp and caching on timestamp like LRU would be important for my use case. Does bigCouch offers such mechanism under it's hood ?
Monica
On Sep 19, 2011, at 10:43 PM, Robert Newson <ro...@gmail.com> wrote:
> What do you mean by 'caching' here?
>
> On 19 September 2011 21:36, Monica Razdan <mo...@almende.org> wrote:
>> Hello,
>>
>> I wanted to compare the caching mechanism used in Hadoop - LRU with
>> the caching in BigCouch. However, I couldn't find details about
>> underlying caching algorithm used in Bigcouch. Could someone please
>> direct me to the right resource?
>>
>> Thanks,
>> monica
>>
Re: Caching in BigCouch
Posted by Robert Newson <ro...@gmail.com>.
What do you mean by 'caching' here?
On 19 September 2011 21:36, Monica Razdan <mo...@almende.org> wrote:
> Hello,
>
> I wanted to compare the caching mechanism used in Hadoop - LRU with
> the caching in BigCouch. However, I couldn't find details about
> underlying caching algorithm used in Bigcouch. Could someone please
> direct me to the right resource?
>
> Thanks,
> monica
>