You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by 金砖 <ji...@wacai.com> on 2016/05/30 11:29:45 UTC
about int types of phoenix
hi, ALL:
Recently I'm importing existing hbase table to phoenix and I found
out phoenix toBytes
method(org.apache.phoenix.schema.types.PInteger.toBytes) is different to
hbase(org.apache.hadoop.hbase.util.Bytes.toBytes).
And document says only support hbase positive values.
Why not just use hbase to bytes? And hbase negative values is supported?
Re: about int types of phoenix
Posted by 金砖 <ji...@wacai.com>.
Thanks for you explanation William, now I understand.
Really appreciated!
\u5728 2016\u5e7405\u670831\u65e5 09:41, William \u5199\u9053:
> Hi, \u91d1\u7816
> For signed data types, phoenix will flip the first bit to guarantee
> the expected sort order for binary comparison. For example:
> 100, its binary: 0x00 00 00 64
> -100, its binary: 0xFF FF FF 9C
> As we all know, 100 is greater than -100, but their sort order by
> binary comparison is 0xFF FF FF 9C > 0x00 00 00 64.
> So we cannot simply store the original bytes to hbase, we have to
> flip the first byte (the sign byte) so that their binary sort order is
> as expected.
> 100 -> 0x00 00 00 64 -> flip the first byte: 0x80 00 00 64
> -100 -> 0xFF FF FF 9C -> flip the first byte: 0x7F FF FF 9C
> Actually we store 0x80 00 00 64 for 100, 0x7F FF FF 9C for -100, so
> the binary comparison result is the same as the original values.
>
> The code is in PInteger.IntCodec#decodeInt(), encodeInt(), as well
> as other numeric data types like PLong, PShort, etc.
> As long as you are using hbase to store signed values, this is one
> issue you must look into, no matter you are using phoenix or not.
>
> For unsigned data types, phoenix does the same thing as
> Bytes.toBytes. So when you import existing hbase table to phoenix, you
> should flip the first byte for signed data and do nothing for unsigned
> data.
> I hope this will help you solve your problem.
>
> Thanks.
> William.
>
> At 2016-05-30 22:24:15, "Christian Hellstr�m" <ps...@gmail.com>
> wrote:
>
> Because HBase does not store metadata, which Phoenix needs to be a
> true SQL skin.
>
> On 30 May 2016 1:30 p.m., "\u91d1\u7816" <jinzhuan@wacai.com
> <ma...@wacai.com>> wrote:
>
> hi, ALL:
> Recently I'm importing existing hbase table to phoenix and
> I found out phoenix toBytes
> method(org.apache.phoenix.schema.types.PInteger.toBytes) is
> different to hbase(org.apache.hadoop.hbase.util.Bytes.toBytes).
>
> And document says only support hbase positive values.
>
> Why not just use hbase to bytes? And hbase negative values is
> supported?
>
>
>
Re:Re: about int types of phoenix
Posted by William <yh...@163.com>.
Hi, 金砖
For signed data types, phoenix will flip the first bit to guarantee the expected sort order for binary comparison. For example:
100, its binary: 0x00 00 00 64
-100, its binary: 0xFF FF FF 9C
As we all know, 100 is greater than -100, but their sort order by binary comparison is 0xFF FF FF 9C > 0x00 00 00 64.
So we cannot simply store the original bytes to hbase, we have to flip the first byte (the sign byte) so that their binary sort order is as expected.
100 -> 0x00 00 00 64 -> flip the first byte: 0x80 00 00 64
-100 -> 0xFF FF FF 9C -> flip the first byte: 0x7F FF FF 9C
Actually we store 0x80 00 00 64 for 100, 0x7F FF FF 9C for -100, so the binary comparison result is the same as the original values.
The code is in PInteger.IntCodec#decodeInt(), encodeInt(), as well as other numeric data types like PLong, PShort, etc.
As long as you are using hbase to store signed values, this is one issue you must look into, no matter you are using phoenix or not.
For unsigned data types, phoenix does the same thing as Bytes.toBytes. So when you import existing hbase table to phoenix, you should flip the first byte for signed data and do nothing for unsigned data.
I hope this will help you solve your problem.
Thanks.
William.
At 2016-05-30 22:24:15, "Christian Hellström" <ps...@gmail.com> wrote:
Because HBase does not store metadata, which Phoenix needs to be a true SQL skin.
On 30 May 2016 1:30 p.m., "金砖" <ji...@wacai.com> wrote:
hi, ALL:
Recently I'm importing existing hbase table to phoenix and I found out phoenix toBytes method(org.apache.phoenix.schema.types.PInteger.toBytes) is different to hbase(org.apache.hadoop.hbase.util.Bytes.toBytes).
And document says only support hbase positive values.
Why not just use hbase to bytes? And hbase negative values is supported?
Re: about int types of phoenix
Posted by Christian Hellström <ps...@gmail.com>.
Because HBase does not store metadata, which Phoenix needs to be a true SQL
skin.
On 30 May 2016 1:30 p.m., "金砖" <ji...@wacai.com> wrote:
> hi, ALL:
> Recently I'm importing existing hbase table to phoenix and I found out
> phoenix toBytes method(org.apache.phoenix.schema.types.PInteger.toBytes) is
> different to hbase(org.apache.hadoop.hbase.util.Bytes.toBytes).
>
> And document says only support hbase positive values.
>
> Why not just use hbase to bytes? And hbase negative values is supported?
>
>