You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by N Kapshoo <nk...@gmail.com> on 2010/06/25 19:42:02 UTC
descending keys
I have a 'long' number that I get by using
HTable.'incrementColumnValue'. This long is used as the qualifier id
on a columnFamily.
However in my listings, I always want the latest number first
(descending order basically).
Currently I call the NavigableMap.descendingKeySet after I do the 'Get'.
Is this the optimal way of doing it? Can I do something that can be
faster? Especially if I start filtering out and getting only first
1000 etc then this wont work.
How should I go about this? Change the key or is there something else
I can do while creating the table, doing a Get etc?
Thanks.
Re: descending keys
Posted by Ryan Rawson <ry...@gmail.com>.
HBase is 8-bit clean, so you can use simple big-endian conversions.
The Bytes class included in HBase has a bunch of examples you can
use/reuse.
The Long.MAX - value advice is also solid. I have used that myself as well.
-ryan
On Fri, Jun 25, 2010 at 12:03 PM, Andrey Stepachev <oc...@gmail.com> wrote:
> and even better use "string sortable long" (code taken from SOLR)
> (for reverse order don't forget to subtrcact value from Long.MAX as in
> message before)
>
> // uses binary representation of an int to build a string of
> // chars that will sort correctly. Only char ranges
> // less than 0xd800 will be used to avoid UCS-16 surrogates.
> // we can use the lowest 15 bits of a char, (or a mask of 0x7fff)
>
> public static int long2bytes(long val, byte[] out, int offset) {
> val += Long.MIN_VALUE;
> out[offset++] = (byte) 0;
> out[offset++] = (byte) (val >>> 60);
> out[offset++] = (byte) (val >>> 53 & 0x7f);
> out[offset++] = (byte) (val >>> 45 & 0xff);
> out[offset++] = (byte) (val >>> 38 & 0x7f);
> out[offset++] = (byte) (val >>> 30 & 0xff);
> out[offset++] = (byte) (val >>> 23 & 0x7f);
> out[offset++] = (byte) (val >>> 15 & 0xff);
> out[offset++] = (byte) (val >>> 8 & 0x7f);
> out[offset] = (byte) (val & 0xff);
> return SORTABLE_LONG_SIZEOF;
> }
>
> public static long bytes2long(byte[] in, int offset, int len) {
> offset++; // skip 1 byte
> long val = ((long) in[offset++] & 0xff) << 60;
> val |= ((long) in[offset++] & 0x7f) << 53;
> val |= ((long) in[offset++] & 0xff) << 45;
> val |= ((long) in[offset++] & 0x7f) << 38;
> val |= ((long) in[offset++] & 0xff) << 30;
> val |= ((long) in[offset++] & 0x7f) << 23;
> val |= ((long) in[offset++] & 0xff) << 15;
> val |= ((long) in[offset++] & 0x7f) << 8;
> val |= ((long) in[offset] & 0xff);
> val -= Long.MIN_VALUE;
> return val;
> }
> 2010/6/25 Andrey Stepachev <oc...@gmail.com>:
>> use Long.MAX - incrementColumnValue() as qualifier
>>
>> 2010/6/25 N Kapshoo <nk...@gmail.com>:
>>> I have a 'long' number that I get by using
>>> HTable.'incrementColumnValue'. This long is used as the qualifier id
>>> on a columnFamily.
>>>
>>> However in my listings, I always want the latest number first
>>> (descending order basically).
>>>
>>> Currently I call the NavigableMap.descendingKeySet after I do the 'Get'.
>>>
>>> Is this the optimal way of doing it? Can I do something that can be
>>> faster? Especially if I start filtering out and getting only first
>>> 1000 etc then this wont work.
>>>
>>> How should I go about this? Change the key or is there something else
>>> I can do while creating the table, doing a Get etc?
>>>
>>> Thanks.
>>>
>>
>
Re: descending keys
Posted by Andrey Stepachev <oc...@gmail.com>.
and even better use "string sortable long" (code taken from SOLR)
(for reverse order don't forget to subtrcact value from Long.MAX as in
message before)
// uses binary representation of an int to build a string of
// chars that will sort correctly. Only char ranges
// less than 0xd800 will be used to avoid UCS-16 surrogates.
// we can use the lowest 15 bits of a char, (or a mask of 0x7fff)
public static int long2bytes(long val, byte[] out, int offset) {
val += Long.MIN_VALUE;
out[offset++] = (byte) 0;
out[offset++] = (byte) (val >>> 60);
out[offset++] = (byte) (val >>> 53 & 0x7f);
out[offset++] = (byte) (val >>> 45 & 0xff);
out[offset++] = (byte) (val >>> 38 & 0x7f);
out[offset++] = (byte) (val >>> 30 & 0xff);
out[offset++] = (byte) (val >>> 23 & 0x7f);
out[offset++] = (byte) (val >>> 15 & 0xff);
out[offset++] = (byte) (val >>> 8 & 0x7f);
out[offset] = (byte) (val & 0xff);
return SORTABLE_LONG_SIZEOF;
}
public static long bytes2long(byte[] in, int offset, int len) {
offset++; // skip 1 byte
long val = ((long) in[offset++] & 0xff) << 60;
val |= ((long) in[offset++] & 0x7f) << 53;
val |= ((long) in[offset++] & 0xff) << 45;
val |= ((long) in[offset++] & 0x7f) << 38;
val |= ((long) in[offset++] & 0xff) << 30;
val |= ((long) in[offset++] & 0x7f) << 23;
val |= ((long) in[offset++] & 0xff) << 15;
val |= ((long) in[offset++] & 0x7f) << 8;
val |= ((long) in[offset] & 0xff);
val -= Long.MIN_VALUE;
return val;
}
2010/6/25 Andrey Stepachev <oc...@gmail.com>:
> use Long.MAX - incrementColumnValue() as qualifier
>
> 2010/6/25 N Kapshoo <nk...@gmail.com>:
>> I have a 'long' number that I get by using
>> HTable.'incrementColumnValue'. This long is used as the qualifier id
>> on a columnFamily.
>>
>> However in my listings, I always want the latest number first
>> (descending order basically).
>>
>> Currently I call the NavigableMap.descendingKeySet after I do the 'Get'.
>>
>> Is this the optimal way of doing it? Can I do something that can be
>> faster? Especially if I start filtering out and getting only first
>> 1000 etc then this wont work.
>>
>> How should I go about this? Change the key or is there something else
>> I can do while creating the table, doing a Get etc?
>>
>> Thanks.
>>
>
Re: descending keys
Posted by Andrey Stepachev <oc...@gmail.com>.
use Long.MAX - incrementColumnValue() as qualifier
2010/6/25 N Kapshoo <nk...@gmail.com>:
> I have a 'long' number that I get by using
> HTable.'incrementColumnValue'. This long is used as the qualifier id
> on a columnFamily.
>
> However in my listings, I always want the latest number first
> (descending order basically).
>
> Currently I call the NavigableMap.descendingKeySet after I do the 'Get'.
>
> Is this the optimal way of doing it? Can I do something that can be
> faster? Especially if I start filtering out and getting only first
> 1000 etc then this wont work.
>
> How should I go about this? Change the key or is there something else
> I can do while creating the table, doing a Get etc?
>
> Thanks.
>