Sorting views with composite keys

[Nick Johnson <ar...@notdot.net>]

How would I construct a view with two keys, a and b, sorted first by a
ascending, then by b descending? Composite keys are easy enough to generate,
but the key [a,b] would only permit sorting both in ascending order (or both
in descending order, if we scan in reverse). Can anyone suggest a way of
modifying the keys to support heterogenous sort orders like this? My best
idea so far is a monstrosity involving hex-encoded binary data. :)

Re: Sorting views with composite keys

[Nick Johnson <ar...@notdot.net>]

On Sun, Oct 5, 2008 at 8:21 PM, Jan Lehnardt <ja...@apache.org> wrote:

> In my defense :)
>
> My suggestion was a 3 minute hack that was done without
> much thinking. If we want to go that route, we'd need to
> add more sorting handlers for other data types. this could
> go into a view-JS standard library (like sum()).
>
> the use-case was just having the sorting by first and second key,
> but only ever query on the first one.


Well, potentially someone might limit based on the second key too, but I
have no issue with preprocessing queries in the same manner.


>
>
> I'm not claiming that this is a stellar idea, only one that might
> be feasible for a specific use case.


Indeed. I'm concerned about the overhead of converting a string into an
array of integers, though. Instinct tells me it may be worse than taking the
bitwise negation of the string and encoding that as a hex string, though.


>
>
> Cheers
> Jan
> --
>
>
> On Oct 5, 2008, at 20:48 , Chris Anderson wrote:
>
>  yeah you'd have to preprocess the keys with a similar function..
>>
>> I think the use case is that you'd query based on the front key, and a
>> count, so as to get, say the last 10 alphabetical tags for a given
>> user.
>>
>> Come to think of it, I'm not sure what sense there is in doing this.
>> Why not just use a regular collation, and if you have more per
>> front-key than you can fit in ram, just page through the front key's
>> second key in reverse...
>>
>>
>> On Sun, Oct 5, 2008 at 11:35 AM, Ayende Rahien <ay...@ayende.com> wrote:
>>
>>> How does querying those works?I mean, the keys wouldn't match, would
>>> they?
>>>
>>> On Sun, Oct 5, 2008 at 8:26 PM, Chris Anderson <jc...@apache.org>
>>> wrote:
>>>
>>>  Jan posted this to the IRC channel.
>>>>
>>>> http://friendpaste.com/rc9CcxUW
>>>>
>>>> I supposed you'd have to write a reverse method for each JS primitive
>>>> type, if you wanted to be completely flexible.
>>>>
>>>>
>>>> On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net>
>>>> wrote:
>>>>
>>>>> How would I construct a view with two keys, a and b, sorted first by a
>>>>> ascending, then by b descending? Composite keys are easy enough to
>>>>>
>>>> generate,
>>>>
>>>>> but the key [a,b] would only permit sorting both in ascending order (or
>>>>>
>>>> both
>>>>
>>>>> in descending order, if we scan in reverse). Can anyone suggest a way
>>>>> of
>>>>> modifying the keys to support heterogenous sort orders like this? My
>>>>> best
>>>>> idea so far is a monstrosity involving hex-encoded binary data. :)
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Chris Anderson
>>>> http://jchris.mfdz.com
>>>>
>>>>
>>>
>>
>>
>> --
>> Chris Anderson
>> http://jchris.mfdz.com
>>
>>
>

Re: Sorting views with composite keys

[Jan Lehnardt <ja...@apache.org>]

In my defense :)

My suggestion was a 3 minute hack that was done without
much thinking. If we want to go that route, we'd need to
add more sorting handlers for other data types. this could
go into a view-JS standard library (like sum()).

the use-case was just having the sorting by first and second key,
but only ever query on the first one.

I'm not claiming that this is a stellar idea, only one that might
be feasible for a specific use case.

Cheers
Jan
--

On Oct 5, 2008, at 20:48 , Chris Anderson wrote:

> yeah you'd have to preprocess the keys with a similar function..
>
> I think the use case is that you'd query based on the front key, and a
> count, so as to get, say the last 10 alphabetical tags for a given
> user.
>
> Come to think of it, I'm not sure what sense there is in doing this.
> Why not just use a regular collation, and if you have more per
> front-key than you can fit in ram, just page through the front key's
> second key in reverse...
>
>
> On Sun, Oct 5, 2008 at 11:35 AM, Ayende Rahien <ay...@ayende.com>  
> wrote:
>> How does querying those works?I mean, the keys wouldn't match,  
>> would they?
>>
>> On Sun, Oct 5, 2008 at 8:26 PM, Chris Anderson <jc...@apache.org>  
>> wrote:
>>
>>> Jan posted this to the IRC channel.
>>>
>>> http://friendpaste.com/rc9CcxUW
>>>
>>> I supposed you'd have to write a reverse method for each JS  
>>> primitive
>>> type, if you wanted to be completely flexible.
>>>
>>>
>>> On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net>  
>>> wrote:
>>>> How would I construct a view with two keys, a and b, sorted first  
>>>> by a
>>>> ascending, then by b descending? Composite keys are easy enough to
>>> generate,
>>>> but the key [a,b] would only permit sorting both in ascending  
>>>> order (or
>>> both
>>>> in descending order, if we scan in reverse). Can anyone suggest a  
>>>> way of
>>>> modifying the keys to support heterogenous sort orders like this?  
>>>> My best
>>>> idea so far is a monstrosity involving hex-encoded binary data. :)
>>>>
>>>
>>>
>>>
>>> --
>>> Chris Anderson
>>> http://jchris.mfdz.com
>>>
>>
>
>
>
> -- 
> Chris Anderson
> http://jchris.mfdz.com
>

Re: Sorting views with composite keys

[Nick Johnson <ar...@notdot.net>]

On Mon, Oct 6, 2008 at 1:10 AM, kowsik <ko...@gmail.com> wrote:

> One possibility is to add a collation function to the design document.
> With this we'll have map, reduce and collate. This allows the user to
> override the default collation mechanism used for a particular design
> view. Seems pretty clean. The view server then becomes responsible
> (using the appropriate language) to perform the sorting using
> whatever's best for the view.
>
> Thoughts?


This seems like an excellent idea. It would also allow users to specify
their own text sorting functions:
https://issues.apache.org/jira/browse/COUCHDB-130 .


>
> K.
>
> On Sun, Oct 5, 2008 at 12:37 PM, Nick Johnson <ar...@notdot.net> wrote:
> > On Sun, Oct 5, 2008 at 7:48 PM, Chris Anderson <jc...@apache.org>
> wrote:
> >
> >> yeah you'd have to preprocess the keys with a similar function..
> >>
> >> I think the use case is that you'd query based on the front key, and a
> >> count, so as to get, say the last 10 alphabetical tags for a given
> >> user.
> >>
> >> Come to think of it, I'm not sure what sense there is in doing this.
> >> Why not just use a regular collation, and if you have more per
> >> front-key than you can fit in ram, just page through the front key's
> >> second key in reverse...
> >
> >
> > Because I don't know at design time how many secondary keys there will be
> > per primary key, or even how many keys there will be.
> >
> >
> >>
> >>
> >> On Sun, Oct 5, 2008 at 11:35 AM, Ayende Rahien <ay...@ayende.com>
> wrote:
> >> > How does querying those works?I mean, the keys wouldn't match, would
> >> they?
> >> >
> >> > On Sun, Oct 5, 2008 at 8:26 PM, Chris Anderson <jc...@apache.org>
> >> wrote:
> >> >
> >> >> Jan posted this to the IRC channel.
> >> >>
> >> >> http://friendpaste.com/rc9CcxUW
> >> >>
> >> >> I supposed you'd have to write a reverse method for each JS primitive
> >> >> type, if you wanted to be completely flexible.
> >> >>
> >> >>
> >> >> On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net>
> >> wrote:
> >> >> > How would I construct a view with two keys, a and b, sorted first
> by a
> >> >> > ascending, then by b descending? Composite keys are easy enough to
> >> >> generate,
> >> >> > but the key [a,b] would only permit sorting both in ascending order
> >> (or
> >> >> both
> >> >> > in descending order, if we scan in reverse). Can anyone suggest a
> way
> >> of
> >> >> > modifying the keys to support heterogenous sort orders like this?
> My
> >> best
> >> >> > idea so far is a monstrosity involving hex-encoded binary data. :)
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Chris Anderson
> >> >> http://jchris.mfdz.com
> >> >>
> >> >
> >>
> >>
> >>
> >> --
> >> Chris Anderson
> >> http://jchris.mfdz.com
> >>
> >
>

Re: Sorting views with composite keys

[kowsik <ko...@gmail.com>]

One possibility is to add a collation function to the design document.
With this we'll have map, reduce and collate. This allows the user to
override the default collation mechanism used for a particular design
view. Seems pretty clean. The view server then becomes responsible
(using the appropriate language) to perform the sorting using
whatever's best for the view.

Thoughts?

K.

On Sun, Oct 5, 2008 at 12:37 PM, Nick Johnson <ar...@notdot.net> wrote:
> On Sun, Oct 5, 2008 at 7:48 PM, Chris Anderson <jc...@apache.org> wrote:
>
>> yeah you'd have to preprocess the keys with a similar function..
>>
>> I think the use case is that you'd query based on the front key, and a
>> count, so as to get, say the last 10 alphabetical tags for a given
>> user.
>>
>> Come to think of it, I'm not sure what sense there is in doing this.
>> Why not just use a regular collation, and if you have more per
>> front-key than you can fit in ram, just page through the front key's
>> second key in reverse...
>
>
> Because I don't know at design time how many secondary keys there will be
> per primary key, or even how many keys there will be.
>
>
>>
>>
>> On Sun, Oct 5, 2008 at 11:35 AM, Ayende Rahien <ay...@ayende.com> wrote:
>> > How does querying those works?I mean, the keys wouldn't match, would
>> they?
>> >
>> > On Sun, Oct 5, 2008 at 8:26 PM, Chris Anderson <jc...@apache.org>
>> wrote:
>> >
>> >> Jan posted this to the IRC channel.
>> >>
>> >> http://friendpaste.com/rc9CcxUW
>> >>
>> >> I supposed you'd have to write a reverse method for each JS primitive
>> >> type, if you wanted to be completely flexible.
>> >>
>> >>
>> >> On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net>
>> wrote:
>> >> > How would I construct a view with two keys, a and b, sorted first by a
>> >> > ascending, then by b descending? Composite keys are easy enough to
>> >> generate,
>> >> > but the key [a,b] would only permit sorting both in ascending order
>> (or
>> >> both
>> >> > in descending order, if we scan in reverse). Can anyone suggest a way
>> of
>> >> > modifying the keys to support heterogenous sort orders like this? My
>> best
>> >> > idea so far is a monstrosity involving hex-encoded binary data. :)
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Chris Anderson
>> >> http://jchris.mfdz.com
>> >>
>> >
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchris.mfdz.com
>>
>

Re: Sorting views with composite keys

[Nick Johnson <ar...@notdot.net>]

On Sun, Oct 5, 2008 at 7:48 PM, Chris Anderson <jc...@apache.org> wrote:

> yeah you'd have to preprocess the keys with a similar function..
>
> I think the use case is that you'd query based on the front key, and a
> count, so as to get, say the last 10 alphabetical tags for a given
> user.
>
> Come to think of it, I'm not sure what sense there is in doing this.
> Why not just use a regular collation, and if you have more per
> front-key than you can fit in ram, just page through the front key's
> second key in reverse...


Because I don't know at design time how many secondary keys there will be
per primary key, or even how many keys there will be.


>
>
> On Sun, Oct 5, 2008 at 11:35 AM, Ayende Rahien <ay...@ayende.com> wrote:
> > How does querying those works?I mean, the keys wouldn't match, would
> they?
> >
> > On Sun, Oct 5, 2008 at 8:26 PM, Chris Anderson <jc...@apache.org>
> wrote:
> >
> >> Jan posted this to the IRC channel.
> >>
> >> http://friendpaste.com/rc9CcxUW
> >>
> >> I supposed you'd have to write a reverse method for each JS primitive
> >> type, if you wanted to be completely flexible.
> >>
> >>
> >> On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net>
> wrote:
> >> > How would I construct a view with two keys, a and b, sorted first by a
> >> > ascending, then by b descending? Composite keys are easy enough to
> >> generate,
> >> > but the key [a,b] would only permit sorting both in ascending order
> (or
> >> both
> >> > in descending order, if we scan in reverse). Can anyone suggest a way
> of
> >> > modifying the keys to support heterogenous sort orders like this? My
> best
> >> > idea so far is a monstrosity involving hex-encoded binary data. :)
> >> >
> >>
> >>
> >>
> >> --
> >> Chris Anderson
> >> http://jchris.mfdz.com
> >>
> >
>
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>

Re: Sorting views with composite keys

[Chris Anderson <jc...@apache.org>]

yeah you'd have to preprocess the keys with a similar function..

I think the use case is that you'd query based on the front key, and a
count, so as to get, say the last 10 alphabetical tags for a given
user.

Come to think of it, I'm not sure what sense there is in doing this.
Why not just use a regular collation, and if you have more per
front-key than you can fit in ram, just page through the front key's
second key in reverse...

On Sun, Oct 5, 2008 at 11:35 AM, Ayende Rahien <ay...@ayende.com> wrote:
> How does querying those works?I mean, the keys wouldn't match, would they?
>
> On Sun, Oct 5, 2008 at 8:26 PM, Chris Anderson <jc...@apache.org> wrote:
>
>> Jan posted this to the IRC channel.
>>
>> http://friendpaste.com/rc9CcxUW
>>
>> I supposed you'd have to write a reverse method for each JS primitive
>> type, if you wanted to be completely flexible.
>>
>>
>> On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net> wrote:
>> > How would I construct a view with two keys, a and b, sorted first by a
>> > ascending, then by b descending? Composite keys are easy enough to
>> generate,
>> > but the key [a,b] would only permit sorting both in ascending order (or
>> both
>> > in descending order, if we scan in reverse). Can anyone suggest a way of
>> > modifying the keys to support heterogenous sort orders like this? My best
>> > idea so far is a monstrosity involving hex-encoded binary data. :)
>> >
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchris.mfdz.com
>>
>



-- 
Chris Anderson
http://jchris.mfdz.com

Re: Sorting views with composite keys

[Ayende Rahien <ay...@ayende.com>]

How does querying those works?I mean, the keys wouldn't match, would they?

On Sun, Oct 5, 2008 at 8:26 PM, Chris Anderson <jc...@apache.org> wrote:

> Jan posted this to the IRC channel.
>
> http://friendpaste.com/rc9CcxUW
>
> I supposed you'd have to write a reverse method for each JS primitive
> type, if you wanted to be completely flexible.
>
>
> On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net> wrote:
> > How would I construct a view with two keys, a and b, sorted first by a
> > ascending, then by b descending? Composite keys are easy enough to
> generate,
> > but the key [a,b] would only permit sorting both in ascending order (or
> both
> > in descending order, if we scan in reverse). Can anyone suggest a way of
> > modifying the keys to support heterogenous sort orders like this? My best
> > idea so far is a monstrosity involving hex-encoded binary data. :)
> >
>
>
>
> --
> Chris Anderson
> http://jchris.mfdz.com
>

Re: Sorting views with composite keys

[Chris Anderson <jc...@apache.org>]

Jan posted this to the IRC channel.

http://friendpaste.com/rc9CcxUW

I supposed you'd have to write a reverse method for each JS primitive
type, if you wanted to be completely flexible.


On Sun, Oct 5, 2008 at 9:25 AM, Nick Johnson <ar...@notdot.net> wrote:
> How would I construct a view with two keys, a and b, sorted first by a
> ascending, then by b descending? Composite keys are easy enough to generate,
> but the key [a,b] would only permit sorting both in ascending order (or both
> in descending order, if we scan in reverse). Can anyone suggest a way of
> modifying the keys to support heterogenous sort orders like this? My best
> idea so far is a monstrosity involving hex-encoded binary data. :)
>



-- 
Chris Anderson
http://jchris.mfdz.com

Re: Sorting views with composite keys

[Paul Davis <pa...@gmail.com>]

Right now I don't know that this is possible. Other than figuring out
some way of getting your second key to sort descending by having it
sorted ascending. Ie, if it were a timestamp, have it reefrence
something in the future, which is awfully hackish.

There *might* be a method for this with the custom correlation stuff,
but I don't know that this sort of fine grained control would be
allowed.

Paul

On Sun, Oct 5, 2008 at 12:25 PM, Nick Johnson <ar...@notdot.net> wrote:
> How would I construct a view with two keys, a and b, sorted first by a
> ascending, then by b descending? Composite keys are easy enough to generate,
> but the key [a,b] would only permit sorting both in ascending order (or both
> in descending order, if we scan in reverse). Can anyone suggest a way of
> modifying the keys to support heterogenous sort orders like this? My best
> idea so far is a monstrosity involving hex-encoded binary data. :)
>