You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Shashikant Kulkarni (JIRA)" <ji...@apache.org> on 2015/11/25 01:07:11 UTC
[jira] [Resolved] (PIG-4743) PIG HBaseStorage with HBase not
working
[ https://issues.apache.org/jira/browse/PIG-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shashikant Kulkarni resolved PIG-4743.
--------------------------------------
Resolution: Fixed
Release Note:
After compiling the PIG 0.15 against HBase 1.1.2, I am able to execute the PIG script successfully using HBaseStorage.
I also downgraded the Hadoop version from 2.6.1 to 2.5.1 because of version compatibility issues with HBase.
See the previous comments on this task for how to fix it.
> PIG HBaseStorage with HBase not working
> ---------------------------------------
>
> Key: PIG-4743
> URL: https://issues.apache.org/jira/browse/PIG-4743
> Project: Pig
> Issue Type: Bug
> Environment: CentOS 6.7
> Hadoop 2.6.1
> HBase 1.1.2
> PIG 0.15.0
> JDK 1.7.0_21
> Reporter: Shashikant Kulkarni
>
> I am using Hadoop version 2.6.1, HBase version 1.1.2 and PIG version 0.15.0
> I have also set the required environment variables in /home/hadoop/.bashrc file as follows
> ========================
> # .bashrc
> # Source global definitions
> if [ -f /etc/bashrc ]; then
> . /etc/bashrc
> fi
> # User specific aliases and functions
> export HADOOP_HOME=/opt/hadoop/hadoop
> export HADOOP_INSTALL=$HADOOP_HOME
> export HADOOP_MAPRED_HOME=$HADOOP_HOME
> export HADOOP_COMMON_HOME=$HADOOP_HOME
> export HADOOP_HDFS_HOME=$HADOOP_HOME
> export YARN_HOME=$HADOOP_HOME
> export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
> export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
> ### Flume Home Directory
> export FLUME_HOME=/usr/lib/flume/apache-flume
> export FLUME_CONF_DIR=$FLUME_HOME/conf
> export FLUME_CLASSPATH=$FLUME_CONF_DIR
> export PATH=$PATH:$FLUME_HOME/bin
> ### HBase Home Directory
> export HBASE_HOME=/usr/lib/hbase
> export HBASE_CONF_DIR=$HBASE_HOME/conf
> export HBASE_CLASSPATH=$HBASE_CONF_DIR
> export PATH=$PATH:$HBASE_HOME/bin
> ###Hive Home Directory
> export HIVE_HOME=/usr/lib/hive/apache-hive
> export HIVE_CONF_DIR=$HIVE_HOME/conf
> export HIVE_CLASSPATH=$HIVE_CONF_DIR
> export PATH=$PATH:$HIVE_HOME/bin
> ###HCatalog Home Directory
> export HCAT_HOME=$HIVE_HOME/hcatalog
> export HCAT_CONF_DIR=$HCAT_HOME/etc/hcatalog:$HCAT_HOME/etc/webhcat
> export HCAT_CLASSPATH=$HCAT_CONF_DIR
> export PATH=$PATH:$HCAT_HOME/bin:$HCAT_HOME/sbin
> ###PIG Home Directory
> export PIG_HOME=/usr/lib/pig/pig-0.15.0
> export PIG_CONF_DIR=$PIG_HOME/conf
> export PIG_CLASSPATH=$PIG_CONF_DIR:$PIG_HOME/pig-0.15.0-core-h2.jar:$HBASE_HOME
> export PATH=$PATH:$PIG_HOME/bin
> ==========================
> When I try to execute the following script
> raw = LOAD 'hbase://test_table' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage ('test_cf:NAME') AS (NAME:chararray);
> I get following error message
> Pig Stack Trace
> ---------------
> ERROR 1200: Pig script failed to parse:
> <file sample1.pig, line 1, column 6> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[test_cf:NAME]'
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Pig script failed to parse:
> <file sample1.pig, line 1, column 6> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[test_cf:NAME]'
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1748)
> at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1443)
> at org.apache.pig.PigServer.parseAndBuild(PigServer.java:387)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:412)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
> at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> at org.apache.pig.Main.run(Main.java:631)
> at org.apache.pig.Main.main(Main.java:177)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> Caused by: Failed to parse: Pig script failed to parse:
> <file sample1.pig, line 1, column 6> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[test_cf:NAME]'
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1735)
> ... 15 more
> Caused by:
> <file sample1.pig, line 1, column 6> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[test_cf:NAME]'
> at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:897)
> at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3568)
> at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)
> at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
> at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
> at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
> ... 16 more
> Caused by: java.lang.RuntimeException: could not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[test_cf:NAME]'
> at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:772)
> at org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:885)
> ... 22 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:740)
> ... 23 more
> Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
> at org.apache.pig.backend.hadoop.hbase.HBaseStorage.initScan(HBaseStorage.java:427)
> at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:368)
> at org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:239)
> ... 28 more
> ================================================================================
> The hbase-client-1.1.2.jar is in path of PIG which contains the org.apache.hadoop.hbase.client.Scan class.
> Can someone please help if I am missing the configuration somewhere? Am I using the compatible versions of PIG and HBase.
> I also tried all the suggestion found on internet to fix this problem, including compiling PIG, copying Jars, Registering Jars with PIG, etc. Nothing is working for me.
> Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)