HBase aggregation for sum

[james isaac <ja...@gmail.com>]

Hi,

I'm having some trouble doing aggregation on a particular column in HBase.

This is the snippet of code I tried:

 Configuration config = HBaseConfiguration.create();
 AggregationClient aggregationClient = new AggregationClient(config);

 Scan scan = new Scan();
 scan.addColumn(Bytes.toBytes("drs"), Bytes.toBytes("count"));

 ColumnInterpreter<Long, Long> ci = new LongColumnInterpreter();

 Long sum = aggregationClient.sum(Bytes.toBytes("DEMO_CALCULATIONS"),
ci , scan);
 System.out.println(sum);

sum returns a value of null. The aggregationClient API works fine if I do a
rowcount.

I was trying to follow the directions in
http://michaelmorello.blogspot.in/2012/01/row-count-hbase-aggregation-example.html

Could there be a problem with me using a LongColumnInterpreter when the
'count' field was an int? What am I missing in here?


Regards,

James

Re: HBase aggregation for sum

[Ted Yu <yu...@gmail.com>]

bq. Could there be a problem with me using a LongColumnInterpreter
when the 'count'
field was an int?

Right. Take a look at the check in the beginning of
LongColumnInterpreter.getValue():

  public Long getValue(byte[] colFamily, byte[] colQualifier, Cell kv)

      throws IOException {

    if (kv == null || kv.getValueLength() != Bytes.SIZEOF_LONG)

      return null;

There is no stock ColumnInterpreter for int in 0.94, please create one.

Cheers



On Thu, Dec 19, 2013 at 1:05 AM, james isaac <ja...@gmail.com> wrote:

> Hi,
>
> I'm having some trouble doing aggregation on a particular column in HBase.
>
> This is the snippet of code I tried:
>
>  Configuration config = HBaseConfiguration.create();
>  AggregationClient aggregationClient = new AggregationClient(config);
>
>  Scan scan = new Scan();
>  scan.addColumn(Bytes.toBytes("drs"), Bytes.toBytes("count"));
>
>  ColumnInterpreter<Long, Long> ci = new LongColumnInterpreter();
>
>  Long sum = aggregationClient.sum(Bytes.toBytes("DEMO_CALCULATIONS"),
> ci , scan);
>  System.out.println(sum);
>
> sum returns a value of null. The aggregationClient API works fine if I do a
> rowcount.
>
> I was trying to follow the directions in
>
> http://michaelmorello.blogspot.in/2012/01/row-count-hbase-aggregation-example.html
>
> Could there be a problem with me using a LongColumnInterpreter when the
> 'count' field was an int? What am I missing in here?
>
>
> Regards,
>
> James
>

Re: HBase aggregation for sum

[Matteo Bertozzi <th...@gmail.com>]

On Thu, Dec 19, 2013 at 9:05 AM, james isaac <ja...@gmail.com> wrote:

> Could there be a problem with me using a LongColumnInterpreter when the
> 'count' field was an int? What am I missing in here?
>

An "int" is serialized as 4 bytes, a "long" as 8 bytes...

The size of the value is checked and expected to be 8 bytes (a long) by the
LongColumnInterpreter, otherwise null is returned
https://github.com/apache/hbase/blob/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/LongColumnInterpreter.java#L44

(the blog post mentions that only long are supported: "Note that this
version is designed to work on data whose type is Long. If you plan to use
other data types you should provide your own column interpreter")