You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by 某因幡 <te...@gmail.com> on 2012/08/27 12:42:56 UTC
PIG with HBase Coprocessors
Hi,
I've created a co-processor which will insert more columns in the
result by overriding preGet and postScannerNext.
And it is registered as a system co-processor, works with hbase shell,
both get and scan.
When I tried to access those columns in PIG, it simply returns null
for the column.
Is there something more to be done?
--
language: Chinese, Japanese, English
Re: PIG with HBase Coprocessors
Posted by yu...@gmail.com.
Did you find some clue from region server logs ?
You can pastebin snippet and show the link here.
On Aug 28, 2012, at 6:54 PM, 某因幡 <te...@gmail.com> wrote:
> Ping?
>
> 2012/8/28 某因幡 <te...@gmail.com>:
>> Thanks for your quick reply.
>> The co-processor looks like:
>> public void postGet(final ObserverContext<RegionCoprocessorEnvironment> e,
>> final Get get, final List<KeyValue> results)
>> {
>> if table is X
>> get some columns from table Y
>> add these columns to results
>> }
>> And similar for postScannerNext().
>> This works in HBase JRuby shell using get/scan plus heavy performance loss.
>>
>> And the versions are:
>> Hadoop 1.0.3
>> HBase 0.92.1
>> Pig 0.9.2
>>
>>
>> 2012/8/27 Ted Yu <yu...@gmail.com>:
>>> Allow me to refer to previous discussion:
>>> http://mail-archives.apache.org/mod_mbox/hbase-user/201203.mbox/%3CCABsY1jQ8+OiLh7SYkXZ8iO=NosY8khz7iYs+6W4u6SxCpJ5s9g@mail.gmail.com%3E
>>>
>>> If the above doesn't answer your question, please give us more details
>>> about the versions of HBase and PIG you're using.
>>>
>>> On Mon, Aug 27, 2012 at 3:42 AM, 某因幡 <te...@gmail.com> wrote:
>>>
>>>> Hi,
>>>> I've created a co-processor which will insert more columns in the
>>>> result by overriding preGet and postScannerNext.
>>>> And it is registered as a system co-processor, works with hbase shell,
>>>> both get and scan.
>>>> When I tried to access those columns in PIG, it simply returns null
>>>> for the column.
>>>> Is there something more to be done?
>>>>
>>>>
>>>> --
>>>> language: Chinese, Japanese, English
>>>>
>>
>>
>>
>> --
>> language: Chinese, Japanese, English
>
>
>
> --
> language: Chinese, Japanese, English
Re: PIG with HBase Coprocessors
Posted by 某因幡 <te...@gmail.com>.
Ping?
2012/8/28 某因幡 <te...@gmail.com>:
> Thanks for your quick reply.
> The co-processor looks like:
> public void postGet(final ObserverContext<RegionCoprocessorEnvironment> e,
> final Get get, final List<KeyValue> results)
> {
> if table is X
> get some columns from table Y
> add these columns to results
> }
> And similar for postScannerNext().
> This works in HBase JRuby shell using get/scan plus heavy performance loss.
>
> And the versions are:
> Hadoop 1.0.3
> HBase 0.92.1
> Pig 0.9.2
>
>
> 2012/8/27 Ted Yu <yu...@gmail.com>:
>> Allow me to refer to previous discussion:
>> http://mail-archives.apache.org/mod_mbox/hbase-user/201203.mbox/%3CCABsY1jQ8+OiLh7SYkXZ8iO=NosY8khz7iYs+6W4u6SxCpJ5s9g@mail.gmail.com%3E
>>
>> If the above doesn't answer your question, please give us more details
>> about the versions of HBase and PIG you're using.
>>
>> On Mon, Aug 27, 2012 at 3:42 AM, 某因幡 <te...@gmail.com> wrote:
>>
>>> Hi,
>>> I've created a co-processor which will insert more columns in the
>>> result by overriding preGet and postScannerNext.
>>> And it is registered as a system co-processor, works with hbase shell,
>>> both get and scan.
>>> When I tried to access those columns in PIG, it simply returns null
>>> for the column.
>>> Is there something more to be done?
>>>
>>>
>>> --
>>> language: Chinese, Japanese, English
>>>
>
>
>
> --
> language: Chinese, Japanese, English
--
language: Chinese, Japanese, English
Re: PIG with HBase Coprocessors
Posted by 某因幡 <te...@gmail.com>.
Thanks for your quick reply.
The co-processor looks like:
public void postGet(final ObserverContext<RegionCoprocessorEnvironment> e,
final Get get, final List<KeyValue> results)
{
if table is X
get some columns from table Y
add these columns to results
}
And similar for postScannerNext().
This works in HBase JRuby shell using get/scan plus heavy performance loss.
And the versions are:
Hadoop 1.0.3
HBase 0.92.1
Pig 0.9.2
2012/8/27 Ted Yu <yu...@gmail.com>:
> Allow me to refer to previous discussion:
> http://mail-archives.apache.org/mod_mbox/hbase-user/201203.mbox/%3CCABsY1jQ8+OiLh7SYkXZ8iO=NosY8khz7iYs+6W4u6SxCpJ5s9g@mail.gmail.com%3E
>
> If the above doesn't answer your question, please give us more details
> about the versions of HBase and PIG you're using.
>
> On Mon, Aug 27, 2012 at 3:42 AM, 某因幡 <te...@gmail.com> wrote:
>
>> Hi,
>> I've created a co-processor which will insert more columns in the
>> result by overriding preGet and postScannerNext.
>> And it is registered as a system co-processor, works with hbase shell,
>> both get and scan.
>> When I tried to access those columns in PIG, it simply returns null
>> for the column.
>> Is there something more to be done?
>>
>>
>> --
>> language: Chinese, Japanese, English
>>
--
language: Chinese, Japanese, English
Re: PIG with HBase Coprocessors
Posted by Ted Yu <yu...@gmail.com>.
Allow me to refer to previous discussion:
http://mail-archives.apache.org/mod_mbox/hbase-user/201203.mbox/%3CCABsY1jQ8+OiLh7SYkXZ8iO=NosY8khz7iYs+6W4u6SxCpJ5s9g@mail.gmail.com%3E
If the above doesn't answer your question, please give us more details
about the versions of HBase and PIG you're using.
On Mon, Aug 27, 2012 at 3:42 AM, 某因幡 <te...@gmail.com> wrote:
> Hi,
> I've created a co-processor which will insert more columns in the
> result by overriding preGet and postScannerNext.
> And it is registered as a system co-processor, works with hbase shell,
> both get and scan.
> When I tried to access those columns in PIG, it simply returns null
> for the column.
> Is there something more to be done?
>
>
> --
> language: Chinese, Japanese, English
>