Order of keys - custom comparator?

[Saptarshi Guha <sa...@gmail.com>]

Hello,
I had wrongly assumed that big endian format of integers(or doubles)
is lexicographically ordered in the same way as they on the number
line, e.g

-1 < 1
but
ff ff ff ff   <= 00 00 00 01

so storing the big endian format as byte arrays would not preserve
order and searches for -5 < rowkey <5 would return wrong results.

Is there during the scan to provide a comparator? So that I can (based
on,say, 4 bytes of prefix) read in the byte key and compare?

Thank you
Saptarshi

Re: Order of keys - custom comparator?

[Ryan Rawson <ry...@gmail.com>]

Good luck out there!

I have not actually heard of needing to have in-order negative
numbers... most of the use cases are storing sequence ids or other
types of things, so negative just never pops up :-)

-ryan

On Mon, Jul 20, 2009 at 10:21 PM, Saptarshi
Guha<sa...@gmail.com> wrote:
> Hello,
> The idea was to insert things like tuples (1,2,3) stored as
>
> length of tuple(int), 1(as int,big endian), 2( as int, B.E), 3 (as
> int, B.E) or in double
>
> Here, if t1,t2  \in R^n, then t1<= t2 if
> a) length(t1) < length(t2)  or
> b)if length(t1)==length(t2), then a lexicographical ordering element wise, thus
> -1 <= (-1,1) <= (1,-1) <= (1,10) <=(2,1)     (Example 1)
>
> This works nicely with positive numbers, but the fly appears with
> negative numbers.
> To make life easier, I'll force users to use positive numbers. There
> is (theoretically) speaking still a countably infinite set to choose
> from! and one can always transform  R to R+ (1 to 1)
>
> Thanks for the confirmation.
> Regards
> Saptarshi
>
>
>
> On Mon, Jul 20, 2009 at 6:52 PM, Ryan Rawson<ry...@gmail.com> wrote:
>> He'd have to use some sign and complement with negation/complement for
>> negative numbers to get the bit pattern to work out...
>>
>> But as for custom comparators, there sadly wont be any, due to the
>> need of compairing keys also in the META table.
>>
>> -ryan
>>
>> On Mon, Jul 20, 2009 at 6:46 PM, Erik Holstad<er...@gmail.com> wrote:
>>> Hey Saptarshi!
>>> I'm not sure what you are trying to do with the order of -5 < rowKey < 5,
>>> but it is not possible to send in a
>>> comparator when scanning, since everything has already been put into HBase.
>>>
>>> And you are right that putting in ints as row keys doesn't keep the order
>>> you are looking for since there is a sign bit.
>>> But what you can do is to add an extra layer on your client that shifts all
>>> your row keys or just use the positive ones.
>>>
>>> Regards Erik
>>>
>>
>

Re: Order of keys - custom comparator?

[Saptarshi Guha <sa...@gmail.com>]

Hello,
The idea was to insert things like tuples (1,2,3) stored as

length of tuple(int), 1(as int,big endian), 2( as int, B.E), 3 (as
int, B.E) or in double

Here, if t1,t2  \in R^n, then t1<= t2 if
a) length(t1) < length(t2)  or
b)if length(t1)==length(t2), then a lexicographical ordering element wise, thus
-1 <= (-1,1) <= (1,-1) <= (1,10) <=(2,1)     (Example 1)

This works nicely with positive numbers, but the fly appears with
negative numbers.
To make life easier, I'll force users to use positive numbers. There
is (theoretically) speaking still a countably infinite set to choose
from! and one can always transform  R to R+ (1 to 1)

Thanks for the confirmation.
Regards
Saptarshi



On Mon, Jul 20, 2009 at 6:52 PM, Ryan Rawson<ry...@gmail.com> wrote:
> He'd have to use some sign and complement with negation/complement for
> negative numbers to get the bit pattern to work out...
>
> But as for custom comparators, there sadly wont be any, due to the
> need of compairing keys also in the META table.
>
> -ryan
>
> On Mon, Jul 20, 2009 at 6:46 PM, Erik Holstad<er...@gmail.com> wrote:
>> Hey Saptarshi!
>> I'm not sure what you are trying to do with the order of -5 < rowKey < 5,
>> but it is not possible to send in a
>> comparator when scanning, since everything has already been put into HBase.
>>
>> And you are right that putting in ints as row keys doesn't keep the order
>> you are looking for since there is a sign bit.
>> But what you can do is to add an extra layer on your client that shifts all
>> your row keys or just use the positive ones.
>>
>> Regards Erik
>>
>

Re: Order of keys - custom comparator?

[Ryan Rawson <ry...@gmail.com>]

He'd have to use some sign and complement with negation/complement for
negative numbers to get the bit pattern to work out...

But as for custom comparators, there sadly wont be any, due to the
need of compairing keys also in the META table.

-ryan

On Mon, Jul 20, 2009 at 6:46 PM, Erik Holstad<er...@gmail.com> wrote:
> Hey Saptarshi!
> I'm not sure what you are trying to do with the order of -5 < rowKey < 5,
> but it is not possible to send in a
> comparator when scanning, since everything has already been put into HBase.
>
> And you are right that putting in ints as row keys doesn't keep the order
> you are looking for since there is a sign bit.
> But what you can do is to add an extra layer on your client that shifts all
> your row keys or just use the positive ones.
>
> Regards Erik
>

Re: Order of keys - custom comparator?

[Erik Holstad <er...@gmail.com>]

Hey Saptarshi!
I'm not sure what you are trying to do with the order of -5 < rowKey < 5,
but it is not possible to send in a
comparator when scanning, since everything has already been put into HBase.

And you are right that putting in ints as row keys doesn't keep the order
you are looking for since there is a sign bit.
But what you can do is to add an extra layer on your client that shifts all
your row keys or just use the positive ones.

Regards Erik