You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by hhsuper <hh...@gmail.com> on 2009/06/20 03:05:56 UTC
couchdb soriting with value question?
Hi,
i have a view (impl with map/reduce function), the value need to be
caculated in reduce function(for group reason etc.), and then how could i
sorting by the value column?
--
Yours sincerely
Jack Su
Re: couchdb soriting with value question?
Posted by hhsuper <hh...@gmail.com>.
Thanks
Hi, Adam, i know your mean, but my sort key must isn't available at map
time(it caculated in reduce func)
Hi, Chris, from your answer may i say that couchdb doesn't directly support
this feature now? maybe i don't use couchdb properly, can you give me some
suggestion? my application scenario is:
I have a php+mysql application, and in some part i save some user's
behavior( or you could say user's score stats.) as couchdb document with
Phly adapter, for the analyzing without affecting mysql's query, now i want
get stats result from the couchdb view to display to users on the web page,
so the previous question pop out? i also need paging and sorting
is that a proper usage?
On Sun, Jun 21, 2009 at 3:09 AM, Adam Wolff <aw...@gmail.com> wrote:
> Hmm. I guess this only works if the sort key is available at map time.
> This is more for finding uniques.
>
> A
> On Saturday, June 20, 2009, Adam Wolff <aw...@gmail.com> wrote:
> > A little jumpy with the space bar today. As I was saying,Depending on the
> nature of your reduce function, you may be able to do something like this:
> > emit([key, sortKey], {value: value, sortKey: sortKey})
> >
> > Then you reduce to the top K values and emit a sorted list that looks
> like: [ {value : value, sortKey:first} .... {value: value, sortKey: Kth} ]
> >
> > You have to arbitrary limit the number of values due to this bug/feature:
> >
> http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#head-83d2e6cce911a0d314d0c5234ff5f19646408113
> >
> > If you want to page, you query the index starting with [key, Kth] and
> then reduce the result against the list from the first page (in fact, you
> can use the reduce function you write in the DB for this if you have an app
> environment where you can run javascript.)
> >
> > A
> > On Sat, Jun 20, 2009 at 11:05 AM, Adam Wolff <aw...@gmail.com> wrote:
> > One patter to solve for this appears to be to index by the sort key, so
> you
> >
> > On Sat, Jun 20, 2009 at 8:13 AM, Chris Anderson <jc...@apache.org>
> wrote:
> > On Sat, Jun 20, 2009 at 12:49 AM, hhsuper<hh...@gmail.com> wrote:
> >> Thanks Jan,
> >>
> >> when i need to paginate the data queried from view, you mean the sorting
> and
> >> paginating should all outside the couchdb?
> >
> > Sorting by value is mathematically tough, if you want to read more
> > about the underlying issues check out
> > http://labs.google.com/papers/sawzall.html
> >
> > If you can live with the sorted order being out of date, the simplest
> > thing to do is copy the results of a group=true query to another
> > database (via a scripting language like ruby or python) and then use a
> > view to sort those results by value. It won't be incrementally updated
> > as records are updated in the original database, but it will give you
> > an exact ordering by value.
> >
> > We are thinking of building something like this into CouchDB because
> > it is such a common query, but for now you've got to do it yourself.
> >
> > Chris
> >
> >>
> >> On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
> >>
> >>>
> >>> On 20 Jun 2009, at 03:05, hhsuper wrote:
> >>>
> >>> Hi,
> >>>>
> >>>> i have a view (impl with map/reduce function), the value need to be
> >>>> caculated in reduce function(for group reason etc.), and then how
> could i
> >>>> sorting by the value column?
> >>>>
> >>>
> >>> You need to sort by value outside of CouchDB.
> >>>
> >>> Cheers
> >>> Jan
> >>> --
> >>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Yours sincerely
> >>
> >> Jack Su
> >>
> >
> >
> >
> > --
> > Chris Anderson
> > http://jchrisa.net
> > http://couch.io
> >
> >
> >
> >
> >
>
--
Yours sincerely
Jack Su
Re: couchdb soriting with value question?
Posted by Adam Wolff <aw...@gmail.com>.
Hmm. I guess this only works if the sort key is available at map time.
This is more for finding uniques.
A
On Saturday, June 20, 2009, Adam Wolff <aw...@gmail.com> wrote:
> A little jumpy with the space bar today. As I was saying,Depending on the nature of your reduce function, you may be able to do something like this:
> emit([key, sortKey], {value: value, sortKey: sortKey})
>
> Then you reduce to the top K values and emit a sorted list that looks like: [ {value : value, sortKey:first} .... {value: value, sortKey: Kth} ]
>
> You have to arbitrary limit the number of values due to this bug/feature:
> http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#head-83d2e6cce911a0d314d0c5234ff5f19646408113
>
> If you want to page, you query the index starting with [key, Kth] and then reduce the result against the list from the first page (in fact, you can use the reduce function you write in the DB for this if you have an app environment where you can run javascript.)
>
> A
> On Sat, Jun 20, 2009 at 11:05 AM, Adam Wolff <aw...@gmail.com> wrote:
> One patter to solve for this appears to be to index by the sort key, so you
>
> On Sat, Jun 20, 2009 at 8:13 AM, Chris Anderson <jc...@apache.org> wrote:
> On Sat, Jun 20, 2009 at 12:49 AM, hhsuper<hh...@gmail.com> wrote:
>> Thanks Jan,
>>
>> when i need to paginate the data queried from view, you mean the sorting and
>> paginating should all outside the couchdb?
>
> Sorting by value is mathematically tough, if you want to read more
> about the underlying issues check out
> http://labs.google.com/papers/sawzall.html
>
> If you can live with the sorted order being out of date, the simplest
> thing to do is copy the results of a group=true query to another
> database (via a scripting language like ruby or python) and then use a
> view to sort those results by value. It won't be incrementally updated
> as records are updated in the original database, but it will give you
> an exact ordering by value.
>
> We are thinking of building something like this into CouchDB because
> it is such a common query, but for now you've got to do it yourself.
>
> Chris
>
>>
>> On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
>>
>>>
>>> On 20 Jun 2009, at 03:05, hhsuper wrote:
>>>
>>> Hi,
>>>>
>>>> i have a view (impl with map/reduce function), the value need to be
>>>> caculated in reduce function(for group reason etc.), and then how could i
>>>> sorting by the value column?
>>>>
>>>
>>> You need to sort by value outside of CouchDB.
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>>
>>
>>
>> --
>> Yours sincerely
>>
>> Jack Su
>>
>
>
>
> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>
>
>
>
>
Re: couchdb soriting with value question?
Posted by Adam Wolff <aw...@gmail.com>.
A little jumpy with the space bar today. As I was saying,Depending on the
nature of your reduce function, you may be able to do something like this:
emit([key, sortKey], {value: value, sortKey: sortKey})
Then you reduce to the top K values and emit a sorted list that looks like:
[ {value : value, sortKey:first} .... {value: value, sortKey: Kth} ]
You have to arbitrary limit the number of values due to this bug/feature:
http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#head-83d2e6cce911a0d314d0c5234ff5f19646408113
If you want to page, you query the index starting with [key, Kth] and then
reduce the result against the list from the first page (in fact, you can use
the reduce function you write in the DB for this if you have an app
environment where you can run javascript.)
A
On Sat, Jun 20, 2009 at 11:05 AM, Adam Wolff <aw...@gmail.com> wrote:
> One patter to solve for this appears to be to index by the sort key, so
> you
>
>
> On Sat, Jun 20, 2009 at 8:13 AM, Chris Anderson <jc...@apache.org> wrote:
>
>> On Sat, Jun 20, 2009 at 12:49 AM, hhsuper<hh...@gmail.com> wrote:
>> > Thanks Jan,
>> >
>> > when i need to paginate the data queried from view, you mean the sorting
>> and
>> > paginating should all outside the couchdb?
>>
>> Sorting by value is mathematically tough, if you want to read more
>> about the underlying issues check out
>> http://labs.google.com/papers/sawzall.html
>>
>> If you can live with the sorted order being out of date, the simplest
>> thing to do is copy the results of a group=true query to another
>> database (via a scripting language like ruby or python) and then use a
>> view to sort those results by value. It won't be incrementally updated
>> as records are updated in the original database, but it will give you
>> an exact ordering by value.
>>
>> We are thinking of building something like this into CouchDB because
>> it is such a common query, but for now you've got to do it yourself.
>>
>> Chris
>>
>> >
>> > On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
>> >
>> >>
>> >> On 20 Jun 2009, at 03:05, hhsuper wrote:
>> >>
>> >> Hi,
>> >>>
>> >>> i have a view (impl with map/reduce function), the value need to be
>> >>> caculated in reduce function(for group reason etc.), and then how
>> could i
>> >>> sorting by the value column?
>> >>>
>> >>
>> >> You need to sort by value outside of CouchDB.
>> >>
>> >> Cheers
>> >> Jan
>> >> --
>> >>
>> >>
>> >>
>> >
>> >
>> > --
>> > Yours sincerely
>> >
>> > Jack Su
>> >
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>
>
>
Re: couchdb soriting with value question?
Posted by Adam Wolff <aw...@gmail.com>.
One patter to solve for this appears to be to index by the sort key, so you
On Sat, Jun 20, 2009 at 8:13 AM, Chris Anderson <jc...@apache.org> wrote:
> On Sat, Jun 20, 2009 at 12:49 AM, hhsuper<hh...@gmail.com> wrote:
> > Thanks Jan,
> >
> > when i need to paginate the data queried from view, you mean the sorting
> and
> > paginating should all outside the couchdb?
>
> Sorting by value is mathematically tough, if you want to read more
> about the underlying issues check out
> http://labs.google.com/papers/sawzall.html
>
> If you can live with the sorted order being out of date, the simplest
> thing to do is copy the results of a group=true query to another
> database (via a scripting language like ruby or python) and then use a
> view to sort those results by value. It won't be incrementally updated
> as records are updated in the original database, but it will give you
> an exact ordering by value.
>
> We are thinking of building something like this into CouchDB because
> it is such a common query, but for now you've got to do it yourself.
>
> Chris
>
> >
> > On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
> >
> >>
> >> On 20 Jun 2009, at 03:05, hhsuper wrote:
> >>
> >> Hi,
> >>>
> >>> i have a view (impl with map/reduce function), the value need to be
> >>> caculated in reduce function(for group reason etc.), and then how could
> i
> >>> sorting by the value column?
> >>>
> >>
> >> You need to sort by value outside of CouchDB.
> >>
> >> Cheers
> >> Jan
> >> --
> >>
> >>
> >>
> >
> >
> > --
> > Yours sincerely
> >
> > Jack Su
> >
>
>
>
> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>
Re: couchdb soriting with value question?
Posted by hhsuper <hh...@gmail.com>.
Thanks
@nborwankar, you know i want to impl a physical paging as possiable as i
can, the memory paging on large data cost much server resouce
On Sun, Jun 21, 2009 at 2:17 PM, <nb...@gmail.com> wrote:
> If you're going to pull the results of the group=true query into say
> python, then you might as well do the sorting in python. There are a number
> of commonly known approaches to sorting a dict by value in python (and I'm
> sure in ruby as well).
> Nitin
> Sent via BlackBerry by AT&T
>
> -----Original Message-----
> From: Chris Anderson <jc...@apache.org>
>
> Date: Sat, 20 Jun 2009 08:13:16
> To: <us...@couchdb.apache.org>
> Subject: Re: couchdb soriting with value question?
>
>
> On Sat, Jun 20, 2009 at 12:49 AM, hhsuper<hh...@gmail.com> wrote:
> > Thanks Jan,
> >
> > when i need to paginate the data queried from view, you mean the sorting
> and
> > paginating should all outside the couchdb?
>
> Sorting by value is mathematically tough, if you want to read more
> about the underlying issues check out
> http://labs.google.com/papers/sawzall.html
>
> If you can live with the sorted order being out of date, the simplest
> thing to do is copy the results of a group=true query to another
> database (via a scripting language like ruby or python) and then use a
> view to sort those results by value. It won't be incrementally updated
> as records are updated in the original database, but it will give you
> an exact ordering by value.
>
> We are thinking of building something like this into CouchDB because
> it is such a common query, but for now you've got to do it yourself.
>
> Chris
>
> >
> > On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
> >
> >>
> >> On 20 Jun 2009, at 03:05, hhsuper wrote:
> >>
> >> Hi,
> >>>
> >>> i have a view (impl with map/reduce function), the value need to be
> >>> caculated in reduce function(for group reason etc.), and then how could
> i
> >>> sorting by the value column?
> >>>
> >>
> >> You need to sort by value outside of CouchDB.
> >>
> >> Cheers
> >> Jan
> >> --
> >>
> >>
> >>
> >
> >
> > --
> > Yours sincerely
> >
> > Jack Su
> >
>
>
>
> --
> Chris Anderson
> http://jchrisa.net
> http://couch.io
>
--
Yours sincerely
Jack Su
Re: couchdb soriting with value question?
Posted by nb...@gmail.com.
If you're going to pull the results of the group=true query into say python, then you might as well do the sorting in python. There are a number of commonly known approaches to sorting a dict by value in python (and I'm sure in ruby as well).
Nitin
Sent via BlackBerry by AT&T
-----Original Message-----
From: Chris Anderson <jc...@apache.org>
Date: Sat, 20 Jun 2009 08:13:16
To: <us...@couchdb.apache.org>
Subject: Re: couchdb soriting with value question?
On Sat, Jun 20, 2009 at 12:49 AM, hhsuper<hh...@gmail.com> wrote:
> Thanks Jan,
>
> when i need to paginate the data queried from view, you mean the sorting and
> paginating should all outside the couchdb?
Sorting by value is mathematically tough, if you want to read more
about the underlying issues check out
http://labs.google.com/papers/sawzall.html
If you can live with the sorted order being out of date, the simplest
thing to do is copy the results of a group=true query to another
database (via a scripting language like ruby or python) and then use a
view to sort those results by value. It won't be incrementally updated
as records are updated in the original database, but it will give you
an exact ordering by value.
We are thinking of building something like this into CouchDB because
it is such a common query, but for now you've got to do it yourself.
Chris
>
> On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
>
>>
>> On 20 Jun 2009, at 03:05, hhsuper wrote:
>>
>> Hi,
>>>
>>> i have a view (impl with map/reduce function), the value need to be
>>> caculated in reduce function(for group reason etc.), and then how could i
>>> sorting by the value column?
>>>
>>
>> You need to sort by value outside of CouchDB.
>>
>> Cheers
>> Jan
>> --
>>
>>
>>
>
>
> --
> Yours sincerely
>
> Jack Su
>
--
Chris Anderson
http://jchrisa.net
http://couch.io
Re: couchdb soriting with value question?
Posted by Chris Anderson <jc...@apache.org>.
On Sat, Jun 20, 2009 at 12:49 AM, hhsuper<hh...@gmail.com> wrote:
> Thanks Jan,
>
> when i need to paginate the data queried from view, you mean the sorting and
> paginating should all outside the couchdb?
Sorting by value is mathematically tough, if you want to read more
about the underlying issues check out
http://labs.google.com/papers/sawzall.html
If you can live with the sorted order being out of date, the simplest
thing to do is copy the results of a group=true query to another
database (via a scripting language like ruby or python) and then use a
view to sort those results by value. It won't be incrementally updated
as records are updated in the original database, but it will give you
an exact ordering by value.
We are thinking of building something like this into CouchDB because
it is such a common query, but for now you've got to do it yourself.
Chris
>
> On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
>
>>
>> On 20 Jun 2009, at 03:05, hhsuper wrote:
>>
>> Hi,
>>>
>>> i have a view (impl with map/reduce function), the value need to be
>>> caculated in reduce function(for group reason etc.), and then how could i
>>> sorting by the value column?
>>>
>>
>> You need to sort by value outside of CouchDB.
>>
>> Cheers
>> Jan
>> --
>>
>>
>>
>
>
> --
> Yours sincerely
>
> Jack Su
>
--
Chris Anderson
http://jchrisa.net
http://couch.io
Re: couchdb soriting with value question?
Posted by hhsuper <hh...@gmail.com>.
Thanks Jan,
when i need to paginate the data queried from view, you mean the sorting and
paginating should all outside the couchdb?
On Sat, Jun 20, 2009 at 3:39 PM, Jan Lehnardt <ja...@apache.org> wrote:
>
> On 20 Jun 2009, at 03:05, hhsuper wrote:
>
> Hi,
>>
>> i have a view (impl with map/reduce function), the value need to be
>> caculated in reduce function(for group reason etc.), and then how could i
>> sorting by the value column?
>>
>
> You need to sort by value outside of CouchDB.
>
> Cheers
> Jan
> --
>
>
>
--
Yours sincerely
Jack Su
Re: couchdb soriting with value question?
Posted by Jan Lehnardt <ja...@apache.org>.
On 20 Jun 2009, at 03:05, hhsuper wrote:
> Hi,
>
> i have a view (impl with map/reduce function), the value need to be
> caculated in reduce function(for group reason etc.), and then how
> could i
> sorting by the value column?
You need to sort by value outside of CouchDB.
Cheers
Jan
--