You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Nicolas Fouché <ni...@silentale.com> on 2009/01/13 22:35:02 UTC
Sort by date and find by key
Hi,
Let's say I have documents like this: {date: '2008/06/09 13:52:11
+0000', name:'Roger'}
I'd like to find documents matching a name, and sorted by date. I know
that for sorting documents by date, the 'date' field has to be in the
key emitted by the view.
In this case, what to I have to emit if I need to see only documents
which name is 'Roger' but sorted by date ?
I would do something like:
emit([doc.name, doc.date], null);
Can I query this view without having to specify the date ? I need
these documents to be sorted by date, because I'll paginate on them.
Thanks,
Nicolas
Re: Sort by date and find by key
Posted by Chris Anderson <jc...@gmail.com>.
On Tue, Jan 13, 2009 at 3:03 PM, Nicolas Fouché <ni...@silentale.com> wrote:
> Thanks a lot, it works. The wiki has a lot of examples on how to
> create views, but not how to query them.
>
> Why do you put '{}' as the second element of the array ? Is it because
> hashes are sorted after strings and arrays, as specified in
> http://wiki.apache.org/couchdb/View_collation ? If I'm right, I
> suppose I can replace the hash by the character 'a', and it also
> works, because dates always begin with a number. Right ?
yep, but do note that collation comparisons between strings are slower
than those between objects of different types.
>
> Nicolas
>
> On Tue, Jan 13, 2009 at 10:38 PM, Chris Anderson <jc...@gmail.com> wrote:
>> On Tue, Jan 13, 2009 at 1:35 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>>> Hi,
>>>
>>> Let's say I have documents like this: {date: '2008/06/09 13:52:11
>>> +0000', name:'Roger'}
>>> I'd like to find documents matching a name, and sorted by date. I know
>>> that for sorting documents by date, the 'date' field has to be in the
>>> key emitted by the view.
>>>
>>> In this case, what to I have to emit if I need to see only documents
>>> which name is 'Roger' but sorted by date ?
>>> I would do something like:
>>> emit([doc.name, doc.date], null);
>>>
>>> Can I query this view without having to specify the date ? I need
>>> these documents to be sorted by date, because I'll paginate on them.
>>
>> Totally, all you need is a startkey and endkey that bracket the keys
>> you care about.
>>
>> eg startkey=["Roger"] endkey=["Roger",{}]
>>
>> See http://wiki.apache.org/couchdb/View_collation
>>
>>>
>>> Thanks,
>>> Nicolas
>>>
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchris.mfdz.com
>>
>
--
Chris Anderson
http://jchris.mfdz.com
Re: Sort by date and find by key
Posted by Nicolas Fouché <ni...@silentale.com>.
Thanks a lot, it works. The wiki has a lot of examples on how to
create views, but not how to query them.
Why do you put '{}' as the second element of the array ? Is it because
hashes are sorted after strings and arrays, as specified in
http://wiki.apache.org/couchdb/View_collation ? If I'm right, I
suppose I can replace the hash by the character 'a', and it also
works, because dates always begin with a number. Right ?
Nicolas
On Tue, Jan 13, 2009 at 10:38 PM, Chris Anderson <jc...@gmail.com> wrote:
> On Tue, Jan 13, 2009 at 1:35 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>> Hi,
>>
>> Let's say I have documents like this: {date: '2008/06/09 13:52:11
>> +0000', name:'Roger'}
>> I'd like to find documents matching a name, and sorted by date. I know
>> that for sorting documents by date, the 'date' field has to be in the
>> key emitted by the view.
>>
>> In this case, what to I have to emit if I need to see only documents
>> which name is 'Roger' but sorted by date ?
>> I would do something like:
>> emit([doc.name, doc.date], null);
>>
>> Can I query this view without having to specify the date ? I need
>> these documents to be sorted by date, because I'll paginate on them.
>
> Totally, all you need is a startkey and endkey that bracket the keys
> you care about.
>
> eg startkey=["Roger"] endkey=["Roger",{}]
>
> See http://wiki.apache.org/couchdb/View_collation
>
>>
>> Thanks,
>> Nicolas
>>
>
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>
Re: Sort by date and find by key
Posted by Dave Bordoley <bo...@gmail.com>.
Are there any plans to add this ability to couchdb? This seems like a
common enough use case, for scenarios such as social networking
activity feeds, that having built in support in the DB itself would be
essential.
Dave
On Tue, Jan 13, 2009 at 3:09 PM, Chris Anderson <jc...@gmail.com> wrote:
> On Tue, Jan 13, 2009 at 3:06 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>> What if I need documents from Roger and Marcel ? I suppose I cannot
>> use the 'keys' query argument in this case, right ?
>>
>
> You'll need to run multiple queries for that.
>
>> Nicolas
>>
>> On Tue, Jan 13, 2009 at 10:38 PM, Chris Anderson <jc...@gmail.com> wrote:
>>> On Tue, Jan 13, 2009 at 1:35 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>>>> Hi,
>>>>
>>>> Let's say I have documents like this: {date: '2008/06/09 13:52:11
>>>> +0000', name:'Roger'}
>>>> I'd like to find documents matching a name, and sorted by date. I know
>>>> that for sorting documents by date, the 'date' field has to be in the
>>>> key emitted by the view.
>>>>
>>>> In this case, what to I have to emit if I need to see only documents
>>>> which name is 'Roger' but sorted by date ?
>>>> I would do something like:
>>>> emit([doc.name, doc.date], null);
>>>>
>>>> Can I query this view without having to specify the date ? I need
>>>> these documents to be sorted by date, because I'll paginate on them.
>>>
>>> Totally, all you need is a startkey and endkey that bracket the keys
>>> you care about.
>>>
>>> eg startkey=["Roger"] endkey=["Roger",{}]
>>>
>>> See http://wiki.apache.org/couchdb/View_collation
>>>
>>>>
>>>> Thanks,
>>>> Nicolas
>>>>
>>>
>>>
>>>
>>> --
>>> Chris Anderson
>>> http://jchris.mfdz.com
>>>
>>
>
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>
Re: Sort by date and find by key
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Jan 13, 2009 at 10:31 PM, Paul Davis <pa...@gmail.com>wrote:
> On Tue, Jan 13, 2009 at 8:29 PM, Dean Landolt <de...@deanlandolt.com>
> wrote:
> > On Tue, Jan 13, 2009 at 6:44 PM, Nicolas Fouché <nicolas@silentale.com
> >wrote:
> >
> >> OK, that's what I was thinking. I currently have all my documents in
> >> full-text search indexes. I guess I'll use CouchDB for storage and
> >> some reduce functions to build dashboards, and not for my
> >> "complicated" queries.
> >> I don't want to load my indexes down with the document content. For
> >> more efficiency, they would only contain the reverse indexes.
> >>
> >> As far as I know, the _external process cannot return document ids to
> >> CouchDB, so CouchDB returns the full document to the caller. Would you
> >> recommend doing this on the cliend side ? Query the index, and then
> >> query CouchDB with a bunch of keys ?
> >
> >
> > This is probably a question for Paul D as he's done quite a bit of work
> in
> > this area (and is also working on a pure Erlang FTI as couch plugin). The
> > _external hook returns full docs, sure, but you control the scripts that
> > pump data into your external indices.
> >
>
> Unless I'm very much mistaken on changes made to the _external code,
> there's nothing that requires you to return full docs whatsoever. The
> complete response can be specified by the OS process. _external is
> extremely shallow in terms of what it requires as a response. You can
> mimic views or do something entirely different with them. For all it
> cares, you could draw mandlebrot fractals in JSON and return that as a
> body.
>
> > But don't forget that couch is more than a db -- it's an app server. It
> > wouldn't hurt to poke around github for some of the pdavis work on this.
> You
> > may be surprised with what you could accomplish by using couch as a proxy
> to
> > and interface for other query mechanisms. Also, I'm pretty sure some of
> this
> > FI stuff is set for inclusion by 1.0, which is nice.
> >
>
Sorry about that, I was making an assumption -- I haven't played too much
with it. That was meant to come out "maybe _external requires..." though,
now that I think about it I'd have known that was wrong if I'd just thought
about it.
In any event, all I was trying to say was that there's a lot more to
interacting with external processes than just some DBUpdateNotification
ability to do it.
Re: Sort by date and find by key
Posted by Paul Davis <pa...@gmail.com>.
On Tue, Jan 13, 2009 at 8:29 PM, Dean Landolt <de...@deanlandolt.com> wrote:
> On Tue, Jan 13, 2009 at 6:44 PM, Nicolas Fouché <ni...@silentale.com>wrote:
>
>> OK, that's what I was thinking. I currently have all my documents in
>> full-text search indexes. I guess I'll use CouchDB for storage and
>> some reduce functions to build dashboards, and not for my
>> "complicated" queries.
>> I don't want to load my indexes down with the document content. For
>> more efficiency, they would only contain the reverse indexes.
>>
>> As far as I know, the _external process cannot return document ids to
>> CouchDB, so CouchDB returns the full document to the caller. Would you
>> recommend doing this on the cliend side ? Query the index, and then
>> query CouchDB with a bunch of keys ?
>
>
> This is probably a question for Paul D as he's done quite a bit of work in
> this area (and is also working on a pure Erlang FTI as couch plugin). The
> _external hook returns full docs, sure, but you control the scripts that
> pump data into your external indices.
>
Unless I'm very much mistaken on changes made to the _external code,
there's nothing that requires you to return full docs whatsoever. The
complete response can be specified by the OS process. _external is
extremely shallow in terms of what it requires as a response. You can
mimic views or do something entirely different with them. For all it
cares, you could draw mandlebrot fractals in JSON and return that as a
body.
> But don't forget that couch is more than a db -- it's an app server. It
> wouldn't hurt to poke around github for some of the pdavis work on this. You
> may be surprised with what you could accomplish by using couch as a proxy to
> and interface for other query mechanisms. Also, I'm pretty sure some of this
> FI stuff is set for inclusion by 1.0, which is nice.
>
Re: Sort by date and find by key
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Jan 13, 2009 at 6:44 PM, Nicolas Fouché <ni...@silentale.com>wrote:
> OK, that's what I was thinking. I currently have all my documents in
> full-text search indexes. I guess I'll use CouchDB for storage and
> some reduce functions to build dashboards, and not for my
> "complicated" queries.
> I don't want to load my indexes down with the document content. For
> more efficiency, they would only contain the reverse indexes.
>
> As far as I know, the _external process cannot return document ids to
> CouchDB, so CouchDB returns the full document to the caller. Would you
> recommend doing this on the cliend side ? Query the index, and then
> query CouchDB with a bunch of keys ?
This is probably a question for Paul D as he's done quite a bit of work in
this area (and is also working on a pure Erlang FTI as couch plugin). The
_external hook returns full docs, sure, but you control the scripts that
pump data into your external indices.
But don't forget that couch is more than a db -- it's an app server. It
wouldn't hurt to poke around github for some of the pdavis work on this. You
may be surprised with what you could accomplish by using couch as a proxy to
and interface for other query mechanisms. Also, I'm pretty sure some of this
FI stuff is set for inclusion by 1.0, which is nice.
Re: Sort by date and find by key
Posted by Nicolas Fouché <ni...@silentale.com>.
OK, that's what I was thinking. I currently have all my documents in
full-text search indexes. I guess I'll use CouchDB for storage and
some reduce functions to build dashboards, and not for my
"complicated" queries.
I don't want to load my indexes down with the document content. For
more efficiency, they would only contain the reverse indexes.
As far as I know, the _external process cannot return document ids to
CouchDB, so CouchDB returns the full document to the caller. Would you
recommend doing this on the cliend side ? Query the index, and then
query CouchDB with a bunch of keys ?
Nicolas
On Wed, Jan 14, 2009 at 12:28 AM, Dean Landolt <de...@deanlandolt.com> wrote:
> On Tue, Jan 13, 2009 at 6:22 PM, Nicolas Fouché <ni...@silentale.com>wrote:
>
>> So I have to run multiple queries, then have to mix the results by
>> date. I need to paginate on this final resultset, and knowing that I
>> deal with thousands of documents, the process seems really
>> complicated.
>>
>> e.g show documents from only one view and not the others because the
>> document they returned have a date greater than the results of the
>> first view. And then have to keep track of every keys, to call views
>> with some startkey and endkey.
>
>
> It's really just a mergesort -- nothing major, though given a large enough
> resultset it could get painful. Intertwingling couch with something like
> solr will give you more options for efficient query strategies.
>
> errrrr, is CouchDB compatible with my project needs ?
>
>
> The answer to that question is almost always going to be *probably*,
> depending on how creative you want to get.
>
Re: Sort by date and find by key
Posted by Dean Landolt <de...@deanlandolt.com>.
On Tue, Jan 13, 2009 at 6:22 PM, Nicolas Fouché <ni...@silentale.com>wrote:
> So I have to run multiple queries, then have to mix the results by
> date. I need to paginate on this final resultset, and knowing that I
> deal with thousands of documents, the process seems really
> complicated.
>
> e.g show documents from only one view and not the others because the
> document they returned have a date greater than the results of the
> first view. And then have to keep track of every keys, to call views
> with some startkey and endkey.
It's really just a mergesort -- nothing major, though given a large enough
resultset it could get painful. Intertwingling couch with something like
solr will give you more options for efficient query strategies.
errrrr, is CouchDB compatible with my project needs ?
The answer to that question is almost always going to be *probably*,
depending on how creative you want to get.
Re: Sort by date and find by key
Posted by Nicolas Fouché <ni...@silentale.com>.
So I have to run multiple queries, then have to mix the results by
date. I need to paginate on this final resultset, and knowing that I
deal with thousands of documents, the process seems really
complicated.
e.g show documents from only one view and not the others because the
document they returned have a date greater than the results of the
first view. And then have to keep track of every keys, to call views
with some startkey and endkey.
errrrr, is CouchDB compatible with my project needs ?
Nicolas
On Wed, Jan 14, 2009 at 12:09 AM, Chris Anderson <jc...@gmail.com> wrote:
> On Tue, Jan 13, 2009 at 3:06 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>> What if I need documents from Roger and Marcel ? I suppose I cannot
>> use the 'keys' query argument in this case, right ?
>>
>
> You'll need to run multiple queries for that.
>
>> Nicolas
>>
>> On Tue, Jan 13, 2009 at 10:38 PM, Chris Anderson <jc...@gmail.com> wrote:
>>> On Tue, Jan 13, 2009 at 1:35 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>>>> Hi,
>>>>
>>>> Let's say I have documents like this: {date: '2008/06/09 13:52:11
>>>> +0000', name:'Roger'}
>>>> I'd like to find documents matching a name, and sorted by date. I know
>>>> that for sorting documents by date, the 'date' field has to be in the
>>>> key emitted by the view.
>>>>
>>>> In this case, what to I have to emit if I need to see only documents
>>>> which name is 'Roger' but sorted by date ?
>>>> I would do something like:
>>>> emit([doc.name, doc.date], null);
>>>>
>>>> Can I query this view without having to specify the date ? I need
>>>> these documents to be sorted by date, because I'll paginate on them.
>>>
>>> Totally, all you need is a startkey and endkey that bracket the keys
>>> you care about.
>>>
>>> eg startkey=["Roger"] endkey=["Roger",{}]
>>>
>>> See http://wiki.apache.org/couchdb/View_collation
>>>
>>>>
>>>> Thanks,
>>>> Nicolas
>>>>
>>>
>>>
>>>
>>> --
>>> Chris Anderson
>>> http://jchris.mfdz.com
>>>
>>
>
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>
Re: Sort by date and find by key
Posted by Chris Anderson <jc...@gmail.com>.
On Tue, Jan 13, 2009 at 3:06 PM, Nicolas Fouché <ni...@silentale.com> wrote:
> What if I need documents from Roger and Marcel ? I suppose I cannot
> use the 'keys' query argument in this case, right ?
>
You'll need to run multiple queries for that.
> Nicolas
>
> On Tue, Jan 13, 2009 at 10:38 PM, Chris Anderson <jc...@gmail.com> wrote:
>> On Tue, Jan 13, 2009 at 1:35 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>>> Hi,
>>>
>>> Let's say I have documents like this: {date: '2008/06/09 13:52:11
>>> +0000', name:'Roger'}
>>> I'd like to find documents matching a name, and sorted by date. I know
>>> that for sorting documents by date, the 'date' field has to be in the
>>> key emitted by the view.
>>>
>>> In this case, what to I have to emit if I need to see only documents
>>> which name is 'Roger' but sorted by date ?
>>> I would do something like:
>>> emit([doc.name, doc.date], null);
>>>
>>> Can I query this view without having to specify the date ? I need
>>> these documents to be sorted by date, because I'll paginate on them.
>>
>> Totally, all you need is a startkey and endkey that bracket the keys
>> you care about.
>>
>> eg startkey=["Roger"] endkey=["Roger",{}]
>>
>> See http://wiki.apache.org/couchdb/View_collation
>>
>>>
>>> Thanks,
>>> Nicolas
>>>
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchris.mfdz.com
>>
>
--
Chris Anderson
http://jchris.mfdz.com
Re: Sort by date and find by key
Posted by Nicolas Fouché <ni...@silentale.com>.
What if I need documents from Roger and Marcel ? I suppose I cannot
use the 'keys' query argument in this case, right ?
Nicolas
On Tue, Jan 13, 2009 at 10:38 PM, Chris Anderson <jc...@gmail.com> wrote:
> On Tue, Jan 13, 2009 at 1:35 PM, Nicolas Fouché <ni...@silentale.com> wrote:
>> Hi,
>>
>> Let's say I have documents like this: {date: '2008/06/09 13:52:11
>> +0000', name:'Roger'}
>> I'd like to find documents matching a name, and sorted by date. I know
>> that for sorting documents by date, the 'date' field has to be in the
>> key emitted by the view.
>>
>> In this case, what to I have to emit if I need to see only documents
>> which name is 'Roger' but sorted by date ?
>> I would do something like:
>> emit([doc.name, doc.date], null);
>>
>> Can I query this view without having to specify the date ? I need
>> these documents to be sorted by date, because I'll paginate on them.
>
> Totally, all you need is a startkey and endkey that bracket the keys
> you care about.
>
> eg startkey=["Roger"] endkey=["Roger",{}]
>
> See http://wiki.apache.org/couchdb/View_collation
>
>>
>> Thanks,
>> Nicolas
>>
>
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>
Re: Sort by date and find by key
Posted by Chris Anderson <jc...@gmail.com>.
On Tue, Jan 13, 2009 at 1:35 PM, Nicolas Fouché <ni...@silentale.com> wrote:
> Hi,
>
> Let's say I have documents like this: {date: '2008/06/09 13:52:11
> +0000', name:'Roger'}
> I'd like to find documents matching a name, and sorted by date. I know
> that for sorting documents by date, the 'date' field has to be in the
> key emitted by the view.
>
> In this case, what to I have to emit if I need to see only documents
> which name is 'Roger' but sorted by date ?
> I would do something like:
> emit([doc.name, doc.date], null);
>
> Can I query this view without having to specify the date ? I need
> these documents to be sorted by date, because I'll paginate on them.
Totally, all you need is a startkey and endkey that bracket the keys
you care about.
eg startkey=["Roger"] endkey=["Roger",{}]
See http://wiki.apache.org/couchdb/View_collation
>
> Thanks,
> Nicolas
>
--
Chris Anderson
http://jchris.mfdz.com