You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by bajeesh rahman <ba...@gmail.com> on 2012/06/05 11:48:44 UTC
Error in pig execution when loading data from HBase
Hi,
I executed Pig java program loading data from HBase in local mode in
Eclipse IDE correctly . But when executing through command mode with the
following commands its showing error.
Input :-
pigServer.registerQuery("L2 = LOAD 'hbase://HBaseTable1' " +
"USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(" +
" ' Family1:fname Family1:lname ' )" +
"AS (fname :chararray,lname :chararray);");
Commands given are(Executing in Linux):-
$ javac -cp pig.jar MyPigProgram.java
$ java -cp pig.jar:. MyPigProgram
Error obtained :-
12/06/05 14:56:30 INFO mapReduceLayer.JobControlCompiler:
mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/zookeeper/ZooKeeper
at
org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:322)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:369)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:256)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
at
org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
at org.apache.pig.PigServer.storeEx(PigServer.java:885)
at org.apache.pig.PigServer.store(PigServer.java:827)
at org.apache.pig.PigServer.store(PigServer.java:795)
at ClusterFinder.runIdQuery(ClusterFinder.java:97)
at ClusterFinder.main(ClusterFinder.java:9)
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ZooKeeper
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 11 more
Any Idea?????
regards
bajeesh rahman
RE: Error in pig execution when loading data from HBase
Posted by ba...@wipro.com.
Fixed issue by exporting :
export CLASSPATH=${PIG_CLASSPATH}:${HBASE_CLASSPATH}:${HADOOP_CLASSPATH}
Thanks all..
From: Bajeesh Rahman (WI01 - Solutions)
Sent: Tuesday, June 05, 2012 6:46 PM
To: 'Subir S'; mapreduce-user@hadoop.apache.org; user@pig.apache.org; bajeeshrahman@gmail.com
Subject: RE: Error in pig execution when loading data from HBase
Hi Subir,
Thank you for your reply.
I have set all environmental variables as specified, then also getting same error.
Pig version used is Pig-0.8.1
From: Subir S [mailto:subir.sasikumar@gmail.com]
Sent: Tuesday, June 05, 2012 6:17 PM
To: mapreduce-user@hadoop.apache.org; user@pig.apache.org; bajeeshrahman@gmail.com
Subject: Re: Error in pig execution when loading data from HBase
This is a pig mailing list question. +pig list for better response.
Which version of Pig you are using.
You miss the Hbase jars in PIG_CLASSPATH as per logs below.
Try:
export HBASE_CONF_DIR=$HBASE_HOME/conf
export HBASE_CLASSPATH="`/usr/bin/hbase classpath`:$HBASE_HOME/conf/hbase-site.xml:$HBASE_CONF_DIR"
On Tue, Jun 5, 2012 at 3:18 PM, bajeesh rahman <ba...@gmail.com>> wrote:
Hi,
I executed Pig java program loading data from HBase in local mode in Eclipse IDE correctly . But when executing through command mode with the following commands its showing error.
Input :-
pigServer.registerQuery("L2 = LOAD 'hbase://HBaseTable1' " +
"USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(" +
" ' Family1:fname Family1:lname ' )" +
"AS (fname :chararray,lname :chararray);");
Commands given are(Executing in Linux):-
$ javac -cp pig.jar MyPigProgram.java
$ java -cp pig.jar:. MyPigProgram
Error obtained :-
12/06/05 14:56:30 INFO mapReduceLayer.JobControlCompiler: mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/zookeeper/ZooKeeper
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:322)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:369)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:256)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
at org.apache.pig.PigServer.storeEx(PigServer.java:885)
at org.apache.pig.PigServer.store(PigServer.java:827)
at org.apache.pig.PigServer.store(PigServer.java:795)
at ClusterFinder.runIdQuery(ClusterFinder.java:97)
at ClusterFinder.main(ClusterFinder.java:9)
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ZooKeeper
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 11 more
Any Idea?????
regards
bajeesh rahman
Please do not print this email unless it is absolutely necessary.
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
www.wipro.com
RE: Error in pig execution when loading data from HBase
Posted by ba...@wipro.com.
Hi Subir,
Thank you for your reply.
I have set all environmental variables as specified, then also getting same error.
Pig version used is Pig-0.8.1
From: Subir S [mailto:subir.sasikumar@gmail.com]
Sent: Tuesday, June 05, 2012 6:17 PM
To: mapreduce-user@hadoop.apache.org; user@pig.apache.org; bajeeshrahman@gmail.com
Subject: Re: Error in pig execution when loading data from HBase
This is a pig mailing list question. +pig list for better response.
Which version of Pig you are using.
You miss the Hbase jars in PIG_CLASSPATH as per logs below.
Try:
export HBASE_CONF_DIR=$HBASE_HOME/conf
export HBASE_CLASSPATH="`/usr/bin/hbase classpath`:$HBASE_HOME/conf/hbase-site.xml:$HBASE_CONF_DIR"
On Tue, Jun 5, 2012 at 3:18 PM, bajeesh rahman <ba...@gmail.com>> wrote:
Hi,
I executed Pig java program loading data from HBase in local mode in Eclipse IDE correctly . But when executing through command mode with the following commands its showing error.
Input :-
pigServer.registerQuery("L2 = LOAD 'hbase://HBaseTable1' " +
"USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(" +
" ' Family1:fname Family1:lname ' )" +
"AS (fname :chararray,lname :chararray);");
Commands given are(Executing in Linux):-
$ javac -cp pig.jar MyPigProgram.java
$ java -cp pig.jar:. MyPigProgram
Error obtained :-
12/06/05 14:56:30 INFO mapReduceLayer.JobControlCompiler: mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/zookeeper/ZooKeeper
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:322)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:369)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:256)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
at org.apache.pig.PigServer.storeEx(PigServer.java:885)
at org.apache.pig.PigServer.store(PigServer.java:827)
at org.apache.pig.PigServer.store(PigServer.java:795)
at ClusterFinder.runIdQuery(ClusterFinder.java:97)
at ClusterFinder.main(ClusterFinder.java:9)
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ZooKeeper
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 11 more
Any Idea?????
regards
bajeesh rahman
Please do not print this email unless it is absolutely necessary.
The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.
WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.
www.wipro.com
Re: Error in pig execution when loading data from HBase
Posted by Subir S <su...@gmail.com>.
This is a pig mailing list question. +pig list for better response.
Which version of Pig you are using.
You miss the Hbase jars in PIG_CLASSPATH as per logs below.
Try:
export HBASE_CONF_DIR=$HBASE_HOME/conf
export HBASE_CLASSPATH="`/usr/bin/hbase
classpath`:$HBASE_HOME/conf/hbase-site.xml:$HBASE_CONF_DIR"
On Tue, Jun 5, 2012 at 3:18 PM, bajeesh rahman <ba...@gmail.com>wrote:
> Hi,
>
>
>
> I executed Pig java program loading data from HBase in local mode in
> Eclipse IDE correctly . But when executing through command mode with the
> following commands its showing error.
>
>
>
> Input :-
>
> pigServer.registerQuery("L2 = LOAD 'hbase://HBaseTable1' " +
>
> "USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(" +
>
> " ' Family1:fname Family1:lname ' )" +
>
> "AS (fname :chararray,lname :chararray);");
>
>
>
> Commands given are(Executing in Linux):-
>
>
>
> $ javac -cp pig.jar MyPigProgram.java
>
> $ java -cp pig.jar:. MyPigProgram
>
>
>
>
>
> Error obtained :-
>
>
>
> 12/06/05 14:56:30 INFO mapReduceLayer.JobControlCompiler:
> mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/zookeeper/ZooKeeper
>
> at
> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:322)
>
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:369)
>
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:256)
>
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
>
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
>
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
>
> at org.apache.pig.PigServer.storeEx(PigServer.java:885)
>
> at org.apache.pig.PigServer.store(PigServer.java:827)
>
> at org.apache.pig.PigServer.store(PigServer.java:795)
>
> at ClusterFinder.runIdQuery(ClusterFinder.java:97)
>
> at ClusterFinder.main(ClusterFinder.java:9)
>
> Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ZooKeeper
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>
> ... 11 more
>
>
>
>
>
> Any Idea?????
>
>
>
>
> regards
> bajeesh rahman
>
Re: Error in pig execution when loading data from HBase
Posted by Subir S <su...@gmail.com>.
This is a pig mailing list question. +pig list for better response.
Which version of Pig you are using.
You miss the Hbase jars in PIG_CLASSPATH as per logs below.
Try:
export HBASE_CONF_DIR=$HBASE_HOME/conf
export HBASE_CLASSPATH="`/usr/bin/hbase
classpath`:$HBASE_HOME/conf/hbase-site.xml:$HBASE_CONF_DIR"
On Tue, Jun 5, 2012 at 3:18 PM, bajeesh rahman <ba...@gmail.com>wrote:
> Hi,
>
>
>
> I executed Pig java program loading data from HBase in local mode in
> Eclipse IDE correctly . But when executing through command mode with the
> following commands its showing error.
>
>
>
> Input :-
>
> pigServer.registerQuery("L2 = LOAD 'hbase://HBaseTable1' " +
>
> "USING org.apache.pig.backend.hadoop.hbase.HBaseStorage(" +
>
> " ' Family1:fname Family1:lname ' )" +
>
> "AS (fname :chararray,lname :chararray);");
>
>
>
> Commands given are(Executing in Linux):-
>
>
>
> $ javac -cp pig.jar MyPigProgram.java
>
> $ java -cp pig.jar:. MyPigProgram
>
>
>
>
>
> Error obtained :-
>
>
>
> 12/06/05 14:56:30 INFO mapReduceLayer.JobControlCompiler:
> mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/zookeeper/ZooKeeper
>
> at
> org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:322)
>
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:369)
>
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:256)
>
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:147)
>
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:382)
>
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1209)
>
> at org.apache.pig.PigServer.storeEx(PigServer.java:885)
>
> at org.apache.pig.PigServer.store(PigServer.java:827)
>
> at org.apache.pig.PigServer.store(PigServer.java:795)
>
> at ClusterFinder.runIdQuery(ClusterFinder.java:97)
>
> at ClusterFinder.main(ClusterFinder.java:9)
>
> Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ZooKeeper
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>
> ... 11 more
>
>
>
>
>
> Any Idea?????
>
>
>
>
> regards
> bajeesh rahman
>