You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by yonghu <yo...@gmail.com> on 2012/11/08 16:40:58 UTC

How can pig load data into HBase

Dear all,

I use pig 0.10.0, hadoop 1.0.3 and hbase 0.94.1. My configuration mode
is pseudo-distribution. When I use
org.apache.pig.backend.hadoop.hbase.HBaseStorage() method to load data
into Hbase, I got the following errors:
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. Not a host:port pair: ï¿½
9854@hans-laptophans-laptop,60020,1352383501956

java.lang.IllegalArgumentException: Not a host:port pair: ï¿½
9854@hans-laptophans-laptop,60020,1352383501956
	at org.apache.hadoop.hbase.HServerAddress.<init>(HServerAddress.java:60)
	at org.apache.hadoop.hbase.zookeeper.RootRegionTracker.dataToHServerAddress(RootRegionTracker.java:82)
	at org.apache.hadoop.hbase.zookeeper.RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:575)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:684)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:586)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:684)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:590)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
	at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:196)
	at org.apache.pig.backend.hadoop.hbase.HBaseStorage.getOutputFormat(HBaseStorage.java:630)
	at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80)
	at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)
	at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
	at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
	at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
	at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
	at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
	at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
	at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:293)
	at org.apache.pig.PigServer.compilePp(PigServer.java:1316)
	at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1253)
	at org.apache.pig.PigServer.execute(PigServer.java:1245)
	at org.apache.pig.PigServer.executeBatch(PigServer.java:362)
	at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
	at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
	at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
	at org.apache.pig.Main.run(Main.java:555)
	at org.apache.pig.Main.main(Main.java:111)
================================================================================
Can anyone tell me how to solve this problem?

regards!

Yong

Re: How can pig load data into HBase

Posted by yonghu <yo...@gmail.com>.

Hi Cheolsoo,

Thanks for your response! It's very important infor.

Yong

On Thu, Nov 8, 2012 at 6:21 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Yong,
>
> ERROR 2999: Unexpected internal error. Not a host:port pair: ï¿½
>  9854@hans-laptophans-laptop,60020,1352383501956
>
> This is because you're using the standalone "pig.jar" that bundles hbase
> 0.90 with hbase 0.94. Please find more details in the following mailing
> thread:
> http://search-hadoop.com/m/sSEnxElsww1&subj=Re+IllegalArgumentException+Not+a+host+port+pair+Pig+0+10+0+with+HBase
>
> This has been an issue to many users, so we made a change to stop bundling
> hbase in pig.jar as part of
> (PIG-2885<https://issues.apache.org/jira/browse/PIG-2885>).
> The patch is committed to trunk now, so this should be no longer issue from
> pig 0.12.
>
> Thanks,
> Cheolsoo
>
> On Thu, Nov 8, 2012 at 7:40 AM, yonghu <yo...@gmail.com> wrote:
>
>> Dear all,
>>
>> I use pig 0.10.0, hadoop 1.0.3 and hbase 0.94.1. My configuration mode
>> is pseudo-distribution. When I use
>> org.apache.pig.backend.hadoop.hbase.HBaseStorage() method to load data
>> into Hbase, I got the following errors:
>> Pig Stack Trace
>> ---------------
>> ERROR 2999: Unexpected internal error. Not a host:port pair: ï¿½
>>  9854@hans-laptophans-laptop,60020,1352383501956
>>
>> java.lang.IllegalArgumentException: Not a host:port pair: ï¿½
>>  9854@hans-laptophans-laptop,60020,1352383501956
>>         at
>> org.apache.hadoop.hbase.HServerAddress.<init>(HServerAddress.java:60)
>>         at
>> org.apache.hadoop.hbase.zookeeper.RootRegionTracker.dataToHServerAddress(RootRegionTracker.java:82)
>>         at
>> org.apache.hadoop.hbase.zookeeper.RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:575)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:684)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:586)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:684)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:590)
>>         at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
>>         at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
>>         at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
>>         at
>> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:196)
>>         at
>> org.apache.pig.backend.hadoop.hbase.HBaseStorage.getOutputFormat(HBaseStorage.java:630)
>>         at
>> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80)
>>         at
>> org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)
>>         at
>> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
>>         at
>> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
>>         at
>> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
>>         at
>> org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
>>         at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
>>         at
>> org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
>>         at
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:293)
>>         at org.apache.pig.PigServer.compilePp(PigServer.java:1316)
>>         at
>> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1253)
>>         at org.apache.pig.PigServer.execute(PigServer.java:1245)
>>         at org.apache.pig.PigServer.executeBatch(PigServer.java:362)
>>         at
>> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
>>         at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
>>         at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>>         at org.apache.pig.Main.run(Main.java:555)
>>         at org.apache.pig.Main.main(Main.java:111)
>>
>> ================================================================================
>> Can anyone tell me how to solve this problem?
>>
>> regards!
>>
>> Yong
>>

Re: How can pig load data into HBase

Posted by Cheolsoo Park <ch...@cloudera.com>.

Hi Yong,

ERROR 2999: Unexpected internal error. Not a host:port pair: ï¿½
 9854@hans-laptophans-laptop,60020,1352383501956

This is because you're using the standalone "pig.jar" that bundles hbase
0.90 with hbase 0.94. Please find more details in the following mailing
thread:
http://search-hadoop.com/m/sSEnxElsww1&subj=Re+IllegalArgumentException+Not+a+host+port+pair+Pig+0+10+0+with+HBase

This has been an issue to many users, so we made a change to stop bundling
hbase in pig.jar as part of
(PIG-2885<https://issues.apache.org/jira/browse/PIG-2885>).
The patch is committed to trunk now, so this should be no longer issue from
pig 0.12.

Thanks,
Cheolsoo

On Thu, Nov 8, 2012 at 7:40 AM, yonghu <yo...@gmail.com> wrote:

> Dear all,
>
> I use pig 0.10.0, hadoop 1.0.3 and hbase 0.94.1. My configuration mode
> is pseudo-distribution. When I use
> org.apache.pig.backend.hadoop.hbase.HBaseStorage() method to load data
> into Hbase, I got the following errors:
> Pig Stack Trace
> ---------------
> ERROR 2999: Unexpected internal error. Not a host:port pair: ï¿½
>  9854@hans-laptophans-laptop,60020,1352383501956
>
> java.lang.IllegalArgumentException: Not a host:port pair: ï¿½
>  9854@hans-laptophans-laptop,60020,1352383501956
>         at
> org.apache.hadoop.hbase.HServerAddress.<init>(HServerAddress.java:60)
>         at
> org.apache.hadoop.hbase.zookeeper.RootRegionTracker.dataToHServerAddress(RootRegionTracker.java:82)
>         at
> org.apache.hadoop.hbase.zookeeper.RootRegionTracker.waitRootRegionLocation(RootRegionTracker.java:73)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:575)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:684)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:586)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:684)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:590)
>         at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:555)
>         at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
>         at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
>         at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:196)
>         at
> org.apache.pig.backend.hadoop.hbase.HBaseStorage.getOutputFormat(HBaseStorage.java:630)
>         at
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:80)
>         at
> org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)
>         at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
>         at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
>         at
> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
>         at
> org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
>         at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
>         at
> org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
>         at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:293)
>         at org.apache.pig.PigServer.compilePp(PigServer.java:1316)
>         at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1253)
>         at org.apache.pig.PigServer.execute(PigServer.java:1245)
>         at org.apache.pig.PigServer.executeBatch(PigServer.java:362)
>         at
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:132)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>         at org.apache.pig.Main.run(Main.java:555)
>         at org.apache.pig.Main.main(Main.java:111)
>
> ================================================================================
> Can anyone tell me how to solve this problem?
>
> regards!
>
> Yong
>