You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by Curtis Caravone <ca...@gmail.com> on 2009/03/19 12:39:01 UTC
stale views and scalability
The documentation for stale=ok describes the following semantics:
The stale option can be used for higher performance ... Using this
option essentially tells CouchDB that if a reference to the view index
is available in memory, go ahead and use it, even if it may be out of
date. ...if there is no view index pointer in memory, the behavior
with this option is that same as the behavior without the option...
For views of large data sets with frequent updates, it is highly
likely that a random key or range query will fail to find its data in
memory, so the query will block until the view has been updated. In
fact, multiple concurrent requests will all block until the update has
completed, which is an issue for scalability. It seems like there
should be an option to immediately return the view query results based
on the latest available snapshot of the view index, regardless of
whether it is in memory or on disk.
Is this something that's easy to implement?
Curtis
Re: stale views and scalability
Posted by Curtis Caravone <ca...@gmail.com>.
Thanks. I built the latest from the trunk and it works beautifully!
On Thu, Mar 19, 2009 at 4:54 AM, Adam Kocoloski
<ad...@gmail.com> wrote:
> On Mar 19, 2009, at 7:39 AM, Curtis Caravone wrote:
>
>> The documentation for stale=ok describes the following semantics:
>>
>> The stale option can be used for higher performance ... Using this
>> option essentially tells CouchDB that if a reference to the view index
>> is available in memory, go ahead and use it, even if it may be out of
>> date. ...if there is no view index pointer in memory, the behavior
>> with this option is that same as the behavior without the option...
>>
>> For views of large data sets with frequent updates, it is highly
>> likely that a random key or range query will fail to find its data in
>> memory, so the query will block until the view has been updated. In
>> fact, multiple concurrent requests will all block until the update has
>> completed, which is an issue for scalability. It seems like there
>> should be an option to immediately return the view query results based
>> on the latest available snapshot of the view index, regardless of
>> whether it is in memory or on disk.
>>
>> Is this something that's easy to implement?
>>
>> Curtis
>
> Hi Curtis, stale=ok works the way you want it to now. A reference to the
> view index is still available even if the entire index is too large to fit
> in memory. I think the language about finding a reference was meant to
> cover the case where the view had not been built or accessed yet. Best,
>
> Adam
>
Re: stale views and scalability
Posted by Adam Kocoloski <ad...@gmail.com>.
On Mar 19, 2009, at 7:39 AM, Curtis Caravone wrote:
> The documentation for stale=ok describes the following semantics:
>
> The stale option can be used for higher performance ... Using this
> option essentially tells CouchDB that if a reference to the view index
> is available in memory, go ahead and use it, even if it may be out of
> date. ...if there is no view index pointer in memory, the behavior
> with this option is that same as the behavior without the option...
>
> For views of large data sets with frequent updates, it is highly
> likely that a random key or range query will fail to find its data in
> memory, so the query will block until the view has been updated. In
> fact, multiple concurrent requests will all block until the update has
> completed, which is an issue for scalability. It seems like there
> should be an option to immediately return the view query results based
> on the latest available snapshot of the view index, regardless of
> whether it is in memory or on disk.
>
> Is this something that's easy to implement?
>
> Curtis
Hi Curtis, stale=ok works the way you want it to now. A reference to
the view index is still available even if the entire index is too
large to fit in memory. I think the language about finding a
reference was meant to cover the case where the view had not been
built or accessed yet. Best,
Adam