Re: Drill + HBase/MapR-DB and float/double/....

[Tugdual Grall <tu...@gmail.com>]

Hello,

Thanks this was the issue, easy fix!

Tug

On Fri, Sep 30, 2016 at 8:49 PM, Aditya <ad...@gmail.com> wrote:

> The unexpected output is because you are using the *little endian*
> decoders in the convert_from() functions while HBase Bytes class uses big
> endian encoding.
>
> That is why int value 10 (0x0000000A) shows up as 167772160 (0xA000000).
>
> Using the *big endian* decoders should give you the expected output.
>
> select
>  convert_from(t.`default`.`int` , 'INT_BE') ,
>  convert_from(t.`default`.`double`, 'DOUBLE_BE' ) ,
>  convert_from(t.`default`.`float`, 'FLOAT_BE')
> from dfs.`demos/simple_table` t
>
>
>
> On Fri, Sep 30, 2016 at 12:01 AM, Tugdual Grall <tu...@gmail.com> wrote:
>
>> Hello,
>>
>> Thanks for your answer, but I did test that (sorry forgot to mention) and
>> it does not work.
>> When I use the cast I have the following error:
>>
>> --
>> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
>> NumberFormatException: Fragment 0:0 [Error Id:
>> fe0bf062-5219-443a-b874-19355fe0bbeb on maprdemo:31010]
>> ---
>>
>> Today, I am converting all the data into String in my Java application and
>> use the cast on String, that does work. (but I do not like this)
>>
>>
>> I think that your code is done for data that are store in String in MapR
>> Db
>> Cells, where I want to store data in Java native types.
>>
>> Tug
>>
>> On Thu, Sep 29, 2016 at 7:54 PM, Abhishek Girish <
>> abhishek.girish@gmail.com>
>> wrote:
>>
>> > Hey Tug,
>> >
>> > You can query a HBase / MapR-DB Binary table as follows:
>> >
>> > > select cast(t.cf1.c1 as INTEGER), cast(t.cf1.c2 as BIGINT),
>> > cast(t.cf1.c3 as DOUBLE), cast(t.cf1.c4 as FLOAT), cast(t.cf1.c5 as
>> BIGINT)
>> > from mfs.`/t1` t;
>> > +---------+---------+----------+---------+---------+
>> > | EXPR$0  | EXPR$1  |  EXPR$2  | EXPR$3  | EXPR$4  |
>> > +---------+---------+----------+---------+---------+
>> > | 10      | 22      | 123.123  | 99.99   | 1111    |
>> > +---------+---------+----------+---------+---------+
>> > 1 row selected (0.198 seconds)
>> >
>> > OR
>> >
>> > > select convert_from(t.cf1.c1, 'UTF8'), convert_from(t.cf1.c2, 'UTF8'),
>> > convert_from(t.cf1.c3, 'UTF8'), convert_from(t.cf1.c4, 'UTF8'),
>> > convert_from(t.cf1.c5, 'UTF8') from mfs.`/t1` t;
>> > +---------+---------+----------+---------+---------+
>> > | EXPR$0  | EXPR$1  |  EXPR$2  | EXPR$3  | EXPR$4  |
>> > +---------+---------+----------+---------+---------+
>> > | 10      | 22      | 123.123  | 99.99   | 1111    |
>> > +---------+---------+----------+---------+---------+
>> > 1 row selected (0.151 seconds)
>> >
>> >
>> > One thing which I'm not clear is if storing values with type is
>> supported.
>> > I mean "99.99f" vs just "99.99" for float. I only got the latter one to
>> > work. With Drill explicit casts, I don't think that should be an issue
>> > though.
>> >
>> > On Thu, Sep 29, 2016 at 6:51 AM, Tugdual Grall <tu...@gmail.com>
>> wrote:
>> >
>> >> Hello,
>> >>
>> >> My HBase (MapR-DB to be exact) application is saving values as
>> numerical
>> >> data types as follow:
>> >>
>> >>
>> >> ====
>> >>     int intValue = 10;
>> >>     long longValue = 22l;
>> >>     double doubleValue = 123.123;
>> >>     float floatValue = 99.99f;
>> >>     BigDecimal bigDecimal = new BigDecimal(1111);
>> >>
>> >>
>> >>     put.addColumn( Bytes.toBytes("default") , Bytes.toBytes("int"),
>> >> Bytes.toBytes( intValue )  );
>> >>     put.addColumn( Bytes.toBytes("default") , Bytes.toBytes("long"),
>> >> Bytes.toBytes( longValue )  );
>> >>     put.addColumn( Bytes.toBytes("default") , Bytes.toBytes("double"),
>> >> Bytes.toBytes( doubleValue )  );
>> >>     put.addColumn( Bytes.toBytes("default") , Bytes.toBytes("float"),
>> >> Bytes.toBytes( floatValue )  );
>> >>     put.addColumn( Bytes.toBytes("default") ,
>> Bytes.toBytes("bigDecimal"),
>> >> Bytes.toBytes( bigDecimal )  );
>> >> ====
>> >>
>> >>
>> >> I am trying to use the convert_from drill function to get the values
>> out
>> >> of
>> >> my database, but it does not work the way I was expecting...
>> >>
>> >> select
>> >>  convert_from(t.`default`.`int` , 'INT') ,
>> >>  convert_from(t.`default`.`double`, 'DOUBLE' ) ,
>> >>  convert_from(t.`default`.`float`, 'FLOAT')
>> >> from dfs.`demos/simple_table` t
>> >> +------------+------------------------+---------------+
>> >> |   EXPR$0   |         EXPR$1         |    EXPR$2     |
>> >> +------------+------------------------+---------------+
>> >> | 167772160  | 2.79722325089736E-167  | -5.782555E20  |
>> >> +------------+------------------------+---------------+
>> >>
>> >>
>> >> 1- What is the proper way to query HBase/MapR-DB to get the proper
>> value?
>> >>
>> >> 2- Can I use such datatypes, or I need to store everything as String?
>> >>
>> >> Regards
>> >> Tug
>> >>
>> >
>> >
>>
>
>