Pig 0.9.2 and HBaseStorage and HBaseBinaryConverter caster of a map

[Neil Yalowitz <ne...@gmail.com>]

Is it possible to load data from an HBase table with Pig 0.9.2 and using
HBaseBinaryConverter -caster on a map of values that are floats?

The O'Reilly Pig book states that it can't be done (page 168), but I'm not
sure if that book is up-to-date.



I can successfully load a single cell float value by calling the
column-qualifier explicitly (such as mycf:myqual1) like this:

raw = LOAD 'hbase://mytable'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
'mycf:myqual1', '-loadKey=true -caster=HBaseBinaryConverter'
) AS (id:chararray, myqual1:float);


The output from the above query is the desired behavior, a human-readable
float.  Then I attempted to use a caster with a map like this:


raw = LOAD 'hbase://mytable'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
'mycf:*', '-loadKey=true -limit=1 -caster=HBaseBinaryConverter'
) AS (id:chararray, mycf_map:map[float]);


This does not appear to work.  The map [key1#val1, key2#val2, key3#val3] is
returned as encoded characters (represented here with ???):

mykey [myqual1#????, myqual2#????, myqual3#????]




Is this impossible?

Neil Yalowitz

Re: Pig 0.9.2 and HBaseStorage and HBaseBinaryConverter caster of a map

[Dmitriy Ryaboy <dv...@gmail.com>]

I think casters are currently not recursive for maps.

On Fri, Apr 13, 2012 at 3:55 PM, Neil Yalowitz <ne...@gmail.com> wrote:
> Is it possible to load data from an HBase table with Pig 0.9.2 and using
> HBaseBinaryConverter -caster on a map of values that are floats?
>
> The O'Reilly Pig book states that it can't be done (page 168), but I'm not
> sure if that book is up-to-date.
>
>
>
> I can successfully load a single cell float value by calling the
> column-qualifier explicitly (such as mycf:myqual1) like this:
>
> raw = LOAD 'hbase://mytable'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
> 'mycf:myqual1', '-loadKey=true -caster=HBaseBinaryConverter'
> ) AS (id:chararray, myqual1:float);
>
>
> The output from the above query is the desired behavior, a human-readable
> float.  Then I attempted to use a caster with a map like this:
>
>
> raw = LOAD 'hbase://mytable'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(
> 'mycf:*', '-loadKey=true -limit=1 -caster=HBaseBinaryConverter'
> ) AS (id:chararray, mycf_map:map[float]);
>
>
> This does not appear to work.  The map [key1#val1, key2#val2, key3#val3] is
> returned as encoded characters (represented here with ???):
>
> mykey [myqual1#????, myqual2#????, myqual3#????]
>
>
>
>
> Is this impossible?
>
> Neil Yalowitz