You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Weiping Qu <qu...@informatik.uni-kl.de> on 2013/04/29 22:08:15 UTC
the version of HBase supported by HBaseStorage
Dear all,
I am currently using HBaseStorage to load and store data between HBase
and Pig.
I have the Pig of the newest version 0.11.1.
I worked with hbase-0.90.6
But I found that HBaseStorage in pig 0.11.1 cannot be used for hbase
with version 0.94.4 or above which is used for phoenix.
The error message is found as follows.
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal
error. com/google/protobuf/Message
Thank you in advance!
Kind Regards,
Weiping
Re: the version of HBase supported by HBaseStorage
Posted by Bill Graham <bi...@gmail.com>.
It's hard to tell the problem from the error pasted, but version 0.94 of
HBase requires protobuf 2.4.x. I've seen errors that start like the one
you've pasted when a 2.3.x version of protobuf is in the classpath.
On Mon, Apr 29, 2013 at 1:49 PM, Weiping Qu <qu...@informatik.uni-kl.de> wrote:
> Oh, cool!
> Thanks for the help.
> I would like to give it a try. :)
>
> Thanks,
> Weiping
>
> +James (Phoenix lead)
>>
>> AFAIK, data inserted into hbase for phoenix reads needs to be phoenix
>> encoded. We wrote a StoreFunc to write data into hbase with data being
>> phoenix encoded. You can give it a try
>> https://github.com/**forcedotcom/phoenix/blob/**master/src/main/java/com/
>> **salesforce/phoenix/pig/**PhoenixHBaseStorage.java<https://github.com/forcedotcom/phoenix/blob/master/src/main/java/com/salesforce/phoenix/pig/PhoenixHBaseStorage.java>
>>
>> Please note this is in the early phase of development, feel free to open
>> issues on https://github.com/**forcedotcom/phoenix/issues<https://github.com/forcedotcom/phoenix/issues>if you find any.
>>
>> Thanks,
>> Prashant
>>
>>
>>
>>
>>
>> On Mon, Apr 29, 2013 at 1:37 PM, Weiping Qu <qu...@informatik.uni-kl.de>
>> wrote:
>>
>> Dear Prashant,
>>>
>>> Firstly thank you very much for your quick reply.
>>>
>>> I would just like to try the new SQL layer above HBase (called Phoenix)
>>> which claims that it outperforms Hive.
>>> So far phoenix has only two releases which support Hbase 0.94.2/3 and
>>> 0.94.4 or above, respectively.
>>> I had carefully included these hbase jars in pig's lib directory
>>> respectively for each try, but all failed. :(
>>> As you mentioned that HBaseStorage currently only supports 0.94.1 by
>>> default, then I think I should give it up now and wait for future
>>> release.
>>>
>>> Or just use Hive than Phoenix.
>>>
>>> Thank you again.
>>> Best,
>>> Weiping
>>>
>>> Hi Weiping,
>>>>
>>>> Pig 0.11.1 uses hbase 0.94.1 by default. Are you using the right jars?
>>>>
>>>>
>>>> On Mon, Apr 29, 2013 at 1:08 PM, Weiping Qu <qu...@informatik.uni-kl.de>
>>>> wrote:
>>>>
>>>> Dear all,
>>>>
>>>>> I am currently using HBaseStorage to load and store data between HBase
>>>>> and
>>>>> Pig.
>>>>> I have the Pig of the newest version 0.11.1.
>>>>> I worked with hbase-0.90.6
>>>>> But I found that HBaseStorage in pig 0.11.1 cannot be used for hbase
>>>>> with
>>>>> version 0.94.4 or above which is used for phoenix.
>>>>>
>>>>> The error message is found as follows.
>>>>> ERROR org.apache.pig.tools.grunt.******Grunt - ERROR 2998: Unhandled
>>>>> internal
>>>>>
>>>>> error. com/google/protobuf/Message
>>>>>
>>>>>
>>>>> Thank you in advance!
>>>>> Kind Regards,
>>>>> Weiping
>>>>>
>>>>>
>>>>>
>>>>>
>
--
*Note that I'm no longer using my Yahoo! email address. Please email me at
billgraham@gmail.com going forward.*
Re: the version of HBase supported by HBaseStorage
Posted by Weiping Qu <qu...@informatik.uni-kl.de>.
Oh, cool!
Thanks for the help.
I would like to give it a try. :)
Thanks,
Weiping
> +James (Phoenix lead)
>
> AFAIK, data inserted into hbase for phoenix reads needs to be phoenix
> encoded. We wrote a StoreFunc to write data into hbase with data being
> phoenix encoded. You can give it a try
> https://github.com/forcedotcom/phoenix/blob/master/src/main/java/com/salesforce/phoenix/pig/PhoenixHBaseStorage.java
>
> Please note this is in the early phase of development, feel free to open
> issues on https://github.com/forcedotcom/phoenix/issues if you find any.
>
> Thanks,
> Prashant
>
>
>
>
>
> On Mon, Apr 29, 2013 at 1:37 PM, Weiping Qu <qu...@informatik.uni-kl.de> wrote:
>
>> Dear Prashant,
>>
>> Firstly thank you very much for your quick reply.
>>
>> I would just like to try the new SQL layer above HBase (called Phoenix)
>> which claims that it outperforms Hive.
>> So far phoenix has only two releases which support Hbase 0.94.2/3 and
>> 0.94.4 or above, respectively.
>> I had carefully included these hbase jars in pig's lib directory
>> respectively for each try, but all failed. :(
>> As you mentioned that HBaseStorage currently only supports 0.94.1 by
>> default, then I think I should give it up now and wait for future release.
>>
>> Or just use Hive than Phoenix.
>>
>> Thank you again.
>> Best,
>> Weiping
>>
>>> Hi Weiping,
>>>
>>> Pig 0.11.1 uses hbase 0.94.1 by default. Are you using the right jars?
>>>
>>>
>>> On Mon, Apr 29, 2013 at 1:08 PM, Weiping Qu <qu...@informatik.uni-kl.de>
>>> wrote:
>>>
>>> Dear all,
>>>> I am currently using HBaseStorage to load and store data between HBase
>>>> and
>>>> Pig.
>>>> I have the Pig of the newest version 0.11.1.
>>>> I worked with hbase-0.90.6
>>>> But I found that HBaseStorage in pig 0.11.1 cannot be used for hbase with
>>>> version 0.94.4 or above which is used for phoenix.
>>>>
>>>> The error message is found as follows.
>>>> ERROR org.apache.pig.tools.grunt.****Grunt - ERROR 2998: Unhandled
>>>> internal
>>>>
>>>> error. com/google/protobuf/Message
>>>>
>>>>
>>>> Thank you in advance!
>>>> Kind Regards,
>>>> Weiping
>>>>
>>>>
>>>>
Re: the version of HBase supported by HBaseStorage
Posted by Prashant Kommireddi <pr...@gmail.com>.
+James (Phoenix lead)
AFAIK, data inserted into hbase for phoenix reads needs to be phoenix
encoded. We wrote a StoreFunc to write data into hbase with data being
phoenix encoded. You can give it a try
https://github.com/forcedotcom/phoenix/blob/master/src/main/java/com/salesforce/phoenix/pig/PhoenixHBaseStorage.java
Please note this is in the early phase of development, feel free to open
issues on https://github.com/forcedotcom/phoenix/issues if you find any.
Thanks,
Prashant
On Mon, Apr 29, 2013 at 1:37 PM, Weiping Qu <qu...@informatik.uni-kl.de> wrote:
> Dear Prashant,
>
> Firstly thank you very much for your quick reply.
>
> I would just like to try the new SQL layer above HBase (called Phoenix)
> which claims that it outperforms Hive.
> So far phoenix has only two releases which support Hbase 0.94.2/3 and
> 0.94.4 or above, respectively.
> I had carefully included these hbase jars in pig's lib directory
> respectively for each try, but all failed. :(
> As you mentioned that HBaseStorage currently only supports 0.94.1 by
> default, then I think I should give it up now and wait for future release.
>
> Or just use Hive than Phoenix.
>
> Thank you again.
> Best,
> Weiping
>
>> Hi Weiping,
>>
>> Pig 0.11.1 uses hbase 0.94.1 by default. Are you using the right jars?
>>
>>
>> On Mon, Apr 29, 2013 at 1:08 PM, Weiping Qu <qu...@informatik.uni-kl.de>
>> wrote:
>>
>> Dear all,
>>>
>>> I am currently using HBaseStorage to load and store data between HBase
>>> and
>>> Pig.
>>> I have the Pig of the newest version 0.11.1.
>>> I worked with hbase-0.90.6
>>> But I found that HBaseStorage in pig 0.11.1 cannot be used for hbase with
>>> version 0.94.4 or above which is used for phoenix.
>>>
>>> The error message is found as follows.
>>> ERROR org.apache.pig.tools.grunt.****Grunt - ERROR 2998: Unhandled
>>> internal
>>>
>>> error. com/google/protobuf/Message
>>>
>>>
>>> Thank you in advance!
>>> Kind Regards,
>>> Weiping
>>>
>>>
>>>
>
Re: the version of HBase supported by HBaseStorage
Posted by Weiping Qu <qu...@informatik.uni-kl.de>.
Dear Prashant,
Firstly thank you very much for your quick reply.
I would just like to try the new SQL layer above HBase (called Phoenix)
which claims that it outperforms Hive.
So far phoenix has only two releases which support Hbase 0.94.2/3 and
0.94.4 or above, respectively.
I had carefully included these hbase jars in pig's lib directory
respectively for each try, but all failed. :(
As you mentioned that HBaseStorage currently only supports 0.94.1 by
default, then I think I should give it up now and wait for future release.
Or just use Hive than Phoenix.
Thank you again.
Best,
Weiping
> Hi Weiping,
>
> Pig 0.11.1 uses hbase 0.94.1 by default. Are you using the right jars?
>
>
> On Mon, Apr 29, 2013 at 1:08 PM, Weiping Qu <qu...@informatik.uni-kl.de> wrote:
>
>> Dear all,
>>
>> I am currently using HBaseStorage to load and store data between HBase and
>> Pig.
>> I have the Pig of the newest version 0.11.1.
>> I worked with hbase-0.90.6
>> But I found that HBaseStorage in pig 0.11.1 cannot be used for hbase with
>> version 0.94.4 or above which is used for phoenix.
>>
>> The error message is found as follows.
>> ERROR org.apache.pig.tools.grunt.**Grunt - ERROR 2998: Unhandled internal
>> error. com/google/protobuf/Message
>>
>>
>> Thank you in advance!
>> Kind Regards,
>> Weiping
>>
>>
Re: the version of HBase supported by HBaseStorage
Posted by Prashant Kommireddi <pr...@gmail.com>.
Hi Weiping,
Pig 0.11.1 uses hbase 0.94.1 by default. Are you using the right jars?
On Mon, Apr 29, 2013 at 1:08 PM, Weiping Qu <qu...@informatik.uni-kl.de> wrote:
> Dear all,
>
> I am currently using HBaseStorage to load and store data between HBase and
> Pig.
> I have the Pig of the newest version 0.11.1.
> I worked with hbase-0.90.6
> But I found that HBaseStorage in pig 0.11.1 cannot be used for hbase with
> version 0.94.4 or above which is used for phoenix.
>
> The error message is found as follows.
> ERROR org.apache.pig.tools.grunt.**Grunt - ERROR 2998: Unhandled internal
> error. com/google/protobuf/Message
>
>
> Thank you in advance!
> Kind Regards,
> Weiping
>
>