You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Shahab Yunus <sh...@gmail.com> on 2013/05/30 00:24:13 UTC
Exception (possibly) due to type casting between bytearray to charray
in HBaseStorage
Hello,
When loading data from a HBase table in Pig, using HBaseStorage, if I
specify the type of the fields as chararray, I get an exception:
2013-05-29 16:18:56,557 INFO
org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
running child : java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/mapreduce/TableInputFormat
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
events = LOAD 'hbase://events'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
info:field2 info:field3')
AS (field1: chararray, field2:chararray, field3:chararray);
It seems to work if I use bytearray instead of chararray (or no type at
all.)
AS (field1: bytearray, field2:bytearray, field3:bytearray);
or
AS (field1, field2, field3);
My question is that why do we get the ClassNotFoundException for
TableInpuFormat which is kind of misleading? Plus, is it even type casting
really the issue? Wouldn't automatic byte-to-char conversion happen?
This issue has been discussed earlier also but it doesn't seem any
conclusion was reached. Plus so far, I have not seen any problem regarding
HBase libs anywhere:
http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
Thanks.
Regards,
Shahab
Re: Exception (possibly) due to type casting between bytearray to
charray in HBaseStorage
Posted by Shahab Yunus <sh...@gmail.com>.
Praveen,
Including those worked! Thanks.
But I have some questions:
1-Why do we even need to do this? If in a cluster, same nodes are being
used as DataNodes/TaskTrackers/RegionServers then shouldn't these libs be
on the classpath already? These are pretty basic libs? Had it been the case
that we have different RegionServer nodes from TaskTrackers then I could
understand. Or even if the same node/server is hosting
RegionServer/DataNode/TaskTracker, the TT daemon, when started, did not
have hbase jars on its classpath?
2-Why hbase security jar? Why not the main one? Is the security name
misleading in the context that it is not 'only' for hbase security but
hbase 'with' security?
3-Why zookeeper jar though? Is it because the hbase jar that is actually
needed here depends, downstream in the dependency chain?
4-Lastly, is there a way where we cannot hardcode these registered
libraries' path?
Or am I missing something here? Thanks for any explanation.
Regards,
Shahab
On Fri, May 31, 2013 at 7:44 AM, shashwat shriparv <
dwivedishashwat@gmail.com> wrote:
> export HBASE_CLASS= jarfiles
> export HADOOP_CLASS= jarfiles
>
> *Thanks & Regards *
>
> ∞
> Shashwat Shriparv
>
>
>
> On Fri, May 31, 2013 at 4:49 PM, Shahab Yunus <shahab.yunus@gmail.com
> >wrote:
>
> > Thanks, I will try and provide an update. Out of curiosity, did you ever
> > resolved it?
> >
> > Regards,
> > Shahab
> >
> >
> > On Fri, May 31, 2013 at 4:17 AM, Praveen Bysani <praveen.iiith@gmail.com
> > >wrote:
> >
> > > Yes, it could be. Try registering these jar files
> > >
> > >
> >
> /opt/cloudera/parcels/CDH-<version>/lib/hbase/hbase-<version>-security.jar
> > >
> /opt/cloudera/parcels/CDH-<version>/lib/zookeeper/zookeeper-<version>.jar
> > >
> > > On 30 May 2013 20:14, Shahab Yunus <sh...@gmail.com> wrote:
> > >
> > > > I am not explicitly registering any of these jars in the script. The
> > > > cluster was setup through standard Cloudera installation (4.2.0).
> > Should
> > > I?
> > > > Is that the issue?
> > > >
> > > > Regards,
> > > > Shahab
> > > >
> > > >
> > > > On Wed, May 29, 2013 at 11:26 PM, Praveen Bysani <
> > > praveen.iiith@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Are you registering the hbase and zookeepr jar files in your pig
> > > script ?
> > > > >
> > > > > On 30 May 2013 06:24, Shahab Yunus <sh...@gmail.com> wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > When loading data from a HBase table in Pig, using HBaseStorage,
> > if I
> > > > > > specify the type of the fields as chararray, I get an exception:
> > > > > >
> > > > > > 2013-05-29 16:18:56,557 INFO
> > > > > > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
> > > > > > truncater with mapRetainSize=-1 and reduceRetainSize=-1
> > > > > > 2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child:
> Error
> > > > > > running child : java.lang.NoClassDefFoundError:
> > > > > > org/apache/hadoop/hbase/mapreduce/TableInputFormat
> > > > > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > > > > at
> > > java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> > > > > > at
> java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> > > > > > at
> > > > > >
> > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > > > > > at
> > > java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > > > > > at
> > java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > > > > > at java.security.AccessController.doPrivileged(Native
> > Method)
> > > > > > at
> > java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > > > > > at
> > > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > > > > > at java.lang.Class.forName0(Native Method)
> > > > > > at java.lang.Class.forName(Class.java:247)
> > > > > >
> > > > > >
> > > > > > events = LOAD 'hbase://events'
> > > > > > USING
> org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
> > > > > > info:field2 info:field3')
> > > > > > AS (field1: chararray, field2:chararray, field3:chararray);
> > > > > > It seems to work if I use bytearray instead of chararray (or no
> > type
> > > at
> > > > > > all.)
> > > > > > AS (field1: bytearray, field2:bytearray, field3:bytearray);
> > > > > >
> > > > > > or
> > > > > >
> > > > > > AS (field1, field2, field3);
> > > > > >
> > > > > > My question is that why do we get the ClassNotFoundException for
> > > > > > TableInpuFormat which is kind of misleading? Plus, is it even
> type
> > > > > casting
> > > > > > really the issue? Wouldn't automatic byte-to-char conversion
> > happen?
> > > > > >
> > > > > > This issue has been discussed earlier also but it doesn't seem
> any
> > > > > > conclusion was reached. Plus so far, I have not seen any problem
> > > > > regarding
> > > > > > HBase libs anywhere:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > Regards,
> > > > > > Shahab
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Regards,
> > > > > Praveen Bysani
> > > > > http://www.praveenbysani.com
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Praveen Bysani
> > > http://www.praveenbysani.com
> > >
> >
>
Re: Exception (possibly) due to type casting between bytearray to
charray in HBaseStorage
Posted by shashwat shriparv <dw...@gmail.com>.
export HBASE_CLASS= jarfiles
export HADOOP_CLASS= jarfiles
*Thanks & Regards *
∞
Shashwat Shriparv
On Fri, May 31, 2013 at 4:49 PM, Shahab Yunus <sh...@gmail.com>wrote:
> Thanks, I will try and provide an update. Out of curiosity, did you ever
> resolved it?
>
> Regards,
> Shahab
>
>
> On Fri, May 31, 2013 at 4:17 AM, Praveen Bysani <praveen.iiith@gmail.com
> >wrote:
>
> > Yes, it could be. Try registering these jar files
> >
> >
> /opt/cloudera/parcels/CDH-<version>/lib/hbase/hbase-<version>-security.jar
> > /opt/cloudera/parcels/CDH-<version>/lib/zookeeper/zookeeper-<version>.jar
> >
> > On 30 May 2013 20:14, Shahab Yunus <sh...@gmail.com> wrote:
> >
> > > I am not explicitly registering any of these jars in the script. The
> > > cluster was setup through standard Cloudera installation (4.2.0).
> Should
> > I?
> > > Is that the issue?
> > >
> > > Regards,
> > > Shahab
> > >
> > >
> > > On Wed, May 29, 2013 at 11:26 PM, Praveen Bysani <
> > praveen.iiith@gmail.com
> > > >wrote:
> > >
> > > > Hi,
> > > >
> > > > Are you registering the hbase and zookeepr jar files in your pig
> > script ?
> > > >
> > > > On 30 May 2013 06:24, Shahab Yunus <sh...@gmail.com> wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > When loading data from a HBase table in Pig, using HBaseStorage,
> if I
> > > > > specify the type of the fields as chararray, I get an exception:
> > > > >
> > > > > 2013-05-29 16:18:56,557 INFO
> > > > > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
> > > > > truncater with mapRetainSize=-1 and reduceRetainSize=-1
> > > > > 2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
> > > > > running child : java.lang.NoClassDefFoundError:
> > > > > org/apache/hadoop/hbase/mapreduce/TableInputFormat
> > > > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > > > at
> > java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> > > > > at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> > > > > at
> > > > >
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > > > > at
> > java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > > > > at
> java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > > > > at java.security.AccessController.doPrivileged(Native
> Method)
> > > > > at
> java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > > > > at
> > > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > > > > at java.lang.Class.forName0(Native Method)
> > > > > at java.lang.Class.forName(Class.java:247)
> > > > >
> > > > >
> > > > > events = LOAD 'hbase://events'
> > > > > USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
> > > > > info:field2 info:field3')
> > > > > AS (field1: chararray, field2:chararray, field3:chararray);
> > > > > It seems to work if I use bytearray instead of chararray (or no
> type
> > at
> > > > > all.)
> > > > > AS (field1: bytearray, field2:bytearray, field3:bytearray);
> > > > >
> > > > > or
> > > > >
> > > > > AS (field1, field2, field3);
> > > > >
> > > > > My question is that why do we get the ClassNotFoundException for
> > > > > TableInpuFormat which is kind of misleading? Plus, is it even type
> > > > casting
> > > > > really the issue? Wouldn't automatic byte-to-char conversion
> happen?
> > > > >
> > > > > This issue has been discussed earlier also but it doesn't seem any
> > > > > conclusion was reached. Plus so far, I have not seen any problem
> > > > regarding
> > > > > HBase libs anywhere:
> > > > >
> > > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Regards,
> > > > > Shahab
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Praveen Bysani
> > > > http://www.praveenbysani.com
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Praveen Bysani
> > http://www.praveenbysani.com
> >
>
Re: Exception (possibly) due to type casting between bytearray to
charray in HBaseStorage
Posted by Shahab Yunus <sh...@gmail.com>.
Thanks, I will try and provide an update. Out of curiosity, did you ever
resolved it?
Regards,
Shahab
On Fri, May 31, 2013 at 4:17 AM, Praveen Bysani <pr...@gmail.com>wrote:
> Yes, it could be. Try registering these jar files
>
> /opt/cloudera/parcels/CDH-<version>/lib/hbase/hbase-<version>-security.jar
> /opt/cloudera/parcels/CDH-<version>/lib/zookeeper/zookeeper-<version>.jar
>
> On 30 May 2013 20:14, Shahab Yunus <sh...@gmail.com> wrote:
>
> > I am not explicitly registering any of these jars in the script. The
> > cluster was setup through standard Cloudera installation (4.2.0). Should
> I?
> > Is that the issue?
> >
> > Regards,
> > Shahab
> >
> >
> > On Wed, May 29, 2013 at 11:26 PM, Praveen Bysani <
> praveen.iiith@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > Are you registering the hbase and zookeepr jar files in your pig
> script ?
> > >
> > > On 30 May 2013 06:24, Shahab Yunus <sh...@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > When loading data from a HBase table in Pig, using HBaseStorage, if I
> > > > specify the type of the fields as chararray, I get an exception:
> > > >
> > > > 2013-05-29 16:18:56,557 INFO
> > > > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
> > > > truncater with mapRetainSize=-1 and reduceRetainSize=-1
> > > > 2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
> > > > running child : java.lang.NoClassDefFoundError:
> > > > org/apache/hadoop/hbase/mapreduce/TableInputFormat
> > > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > > at
> java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> > > > at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> > > > at
> > > >
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > > > at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > > > at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > > > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > > > at java.security.AccessController.doPrivileged(Native Method)
> > > > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > > > at
> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > > > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > > > at java.lang.Class.forName0(Native Method)
> > > > at java.lang.Class.forName(Class.java:247)
> > > >
> > > >
> > > > events = LOAD 'hbase://events'
> > > > USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
> > > > info:field2 info:field3')
> > > > AS (field1: chararray, field2:chararray, field3:chararray);
> > > > It seems to work if I use bytearray instead of chararray (or no type
> at
> > > > all.)
> > > > AS (field1: bytearray, field2:bytearray, field3:bytearray);
> > > >
> > > > or
> > > >
> > > > AS (field1, field2, field3);
> > > >
> > > > My question is that why do we get the ClassNotFoundException for
> > > > TableInpuFormat which is kind of misleading? Plus, is it even type
> > > casting
> > > > really the issue? Wouldn't automatic byte-to-char conversion happen?
> > > >
> > > > This issue has been discussed earlier also but it doesn't seem any
> > > > conclusion was reached. Plus so far, I have not seen any problem
> > > regarding
> > > > HBase libs anywhere:
> > > >
> > > >
> > >
> >
> http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
> > > >
> > > > Thanks.
> > > >
> > > > Regards,
> > > > Shahab
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Praveen Bysani
> > > http://www.praveenbysani.com
> > >
> >
>
>
>
> --
> Regards,
> Praveen Bysani
> http://www.praveenbysani.com
>
Re: Exception (possibly) due to type casting between bytearray to
charray in HBaseStorage
Posted by Praveen Bysani <pr...@gmail.com>.
Yes, it could be. Try registering these jar files
/opt/cloudera/parcels/CDH-<version>/lib/hbase/hbase-<version>-security.jar
/opt/cloudera/parcels/CDH-<version>/lib/zookeeper/zookeeper-<version>.jar
On 30 May 2013 20:14, Shahab Yunus <sh...@gmail.com> wrote:
> I am not explicitly registering any of these jars in the script. The
> cluster was setup through standard Cloudera installation (4.2.0). Should I?
> Is that the issue?
>
> Regards,
> Shahab
>
>
> On Wed, May 29, 2013 at 11:26 PM, Praveen Bysani <praveen.iiith@gmail.com
> >wrote:
>
> > Hi,
> >
> > Are you registering the hbase and zookeepr jar files in your pig script ?
> >
> > On 30 May 2013 06:24, Shahab Yunus <sh...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > When loading data from a HBase table in Pig, using HBaseStorage, if I
> > > specify the type of the fields as chararray, I get an exception:
> > >
> > > 2013-05-29 16:18:56,557 INFO
> > > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
> > > truncater with mapRetainSize=-1 and reduceRetainSize=-1
> > > 2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
> > > running child : java.lang.NoClassDefFoundError:
> > > org/apache/hadoop/hbase/mapreduce/TableInputFormat
> > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> > > at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> > > at
> > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > > at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > > at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > > at java.lang.Class.forName0(Native Method)
> > > at java.lang.Class.forName(Class.java:247)
> > >
> > >
> > > events = LOAD 'hbase://events'
> > > USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
> > > info:field2 info:field3')
> > > AS (field1: chararray, field2:chararray, field3:chararray);
> > > It seems to work if I use bytearray instead of chararray (or no type at
> > > all.)
> > > AS (field1: bytearray, field2:bytearray, field3:bytearray);
> > >
> > > or
> > >
> > > AS (field1, field2, field3);
> > >
> > > My question is that why do we get the ClassNotFoundException for
> > > TableInpuFormat which is kind of misleading? Plus, is it even type
> > casting
> > > really the issue? Wouldn't automatic byte-to-char conversion happen?
> > >
> > > This issue has been discussed earlier also but it doesn't seem any
> > > conclusion was reached. Plus so far, I have not seen any problem
> > regarding
> > > HBase libs anywhere:
> > >
> > >
> >
> http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
> > >
> > > Thanks.
> > >
> > > Regards,
> > > Shahab
> > >
> >
> >
> >
> > --
> > Regards,
> > Praveen Bysani
> > http://www.praveenbysani.com
> >
>
--
Regards,
Praveen Bysani
http://www.praveenbysani.com
Re: Exception (possibly) due to type casting between bytearray to
charray in HBaseStorage
Posted by Shahab Yunus <sh...@gmail.com>.
I am not explicitly registering any of these jars in the script. The
cluster was setup through standard Cloudera installation (4.2.0). Should I?
Is that the issue?
Regards,
Shahab
On Wed, May 29, 2013 at 11:26 PM, Praveen Bysani <pr...@gmail.com>wrote:
> Hi,
>
> Are you registering the hbase and zookeepr jar files in your pig script ?
>
> On 30 May 2013 06:24, Shahab Yunus <sh...@gmail.com> wrote:
>
> > Hello,
> >
> > When loading data from a HBase table in Pig, using HBaseStorage, if I
> > specify the type of the fields as chararray, I get an exception:
> >
> > 2013-05-29 16:18:56,557 INFO
> > org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
> > truncater with mapRetainSize=-1 and reduceRetainSize=-1
> > 2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
> > running child : java.lang.NoClassDefFoundError:
> > org/apache/hadoop/hbase/mapreduce/TableInputFormat
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> > at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:247)
> >
> >
> > events = LOAD 'hbase://events'
> > USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
> > info:field2 info:field3')
> > AS (field1: chararray, field2:chararray, field3:chararray);
> > It seems to work if I use bytearray instead of chararray (or no type at
> > all.)
> > AS (field1: bytearray, field2:bytearray, field3:bytearray);
> >
> > or
> >
> > AS (field1, field2, field3);
> >
> > My question is that why do we get the ClassNotFoundException for
> > TableInpuFormat which is kind of misleading? Plus, is it even type
> casting
> > really the issue? Wouldn't automatic byte-to-char conversion happen?
> >
> > This issue has been discussed earlier also but it doesn't seem any
> > conclusion was reached. Plus so far, I have not seen any problem
> regarding
> > HBase libs anywhere:
> >
> >
> http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
> >
> > Thanks.
> >
> > Regards,
> > Shahab
> >
>
>
>
> --
> Regards,
> Praveen Bysani
> http://www.praveenbysani.com
>
Re: Exception (possibly) due to type casting between bytearray to
charray in HBaseStorage
Posted by Praveen Bysani <pr...@gmail.com>.
Hi,
Are you registering the hbase and zookeepr jar files in your pig script ?
On 30 May 2013 06:24, Shahab Yunus <sh...@gmail.com> wrote:
> Hello,
>
> When loading data from a HBase table in Pig, using HBaseStorage, if I
> specify the type of the fields as chararray, I get an exception:
>
> 2013-05-29 16:18:56,557 INFO
> org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
> truncater with mapRetainSize=-1 and reduceRetainSize=-1
> 2013-05-29 16:18:56,560 FATAL org.apache.hadoop.mapred.Child: Error
> running child : java.lang.NoClassDefFoundError:
> org/apache/hadoop/hbase/mapreduce/TableInputFormat
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
>
>
> events = LOAD 'hbase://events'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:field1
> info:field2 info:field3')
> AS (field1: chararray, field2:chararray, field3:chararray);
> It seems to work if I use bytearray instead of chararray (or no type at
> all.)
> AS (field1: bytearray, field2:bytearray, field3:bytearray);
>
> or
>
> AS (field1, field2, field3);
>
> My question is that why do we get the ClassNotFoundException for
> TableInpuFormat which is kind of misleading? Plus, is it even type casting
> really the issue? Wouldn't automatic byte-to-char conversion happen?
>
> This issue has been discussed earlier also but it doesn't seem any
> conclusion was reached. Plus so far, I have not seen any problem regarding
> HBase libs anywhere:
>
> http://search-hadoop.com/m/KUU6m1oZMEi1&subj=Re+Unable+to+typecast+fields+loaded+from+HBase
>
> Thanks.
>
> Regards,
> Shahab
>
--
Regards,
Praveen Bysani
http://www.praveenbysani.com