HBase get from within UDF vs. PIG FILTER

[Norbert Burger <no...@gmail.com>]

I have a need within a larger Pig script to pull just a few records from an
Hbase table.  I know the exact key, so it'd be trivial with a get() from a
UDF.  Another alternative is use to a custom LOAD/FILTER combo, but this
would involve filtering off all but 3 of about 50 records.

>From a performance angle, punting over to the UDF is faster, right?
Although it seems to break the model of only using UDFs when necessary...

How are others handling this situation?

Norbert

Re: HBase get from within UDF vs. PIG FILTER

[Raghu Angadi <an...@gmail.com>]

UDF could be faster some of the accesses. We do use a lookup UDF in some of
the scripts.

Looking up 6% of  the rows might be a bit high for some tables.

Raghu.
On Fri, Aug 19, 2011 at 9:16 AM, Norbert Burger <no...@gmail.com>wrote:

> I have a need within a larger Pig script to pull just a few records from an
> Hbase table.  I know the exact key, so it'd be trivial with a get() from a
> UDF.  Another alternative is use to a custom LOAD/FILTER combo, but this
> would involve filtering off all but 3 of about 50 records.
>
> From a performance angle, punting over to the UDF is faster, right?
> Although it seems to break the model of only using UDFs when necessary...
>
> How are others handling this situation?
>
> Norbert
>