You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by bharath vissapragada <bh...@gmail.com> on 2009/07/24 12:00:01 UTC
Implementing TableMap interface
Hi all,
I wanted to implement TableMap interface so that "map" function can emit
<Text,Text>.. I wrote the code as follows
-------------------------------------------------------------------------------------------------------
package org.apache.hadoop.hbase.mapred;
import java.io.IOException;
import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
import org.apache.hadoop.hbase.io.RowResult;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;
public class MyTableMap
extends MapReduceBase
implements TableMap<ImmutableBytesWritable, RowResult> {
/** constructor */
public MyTableMap() {
super();
}
@SuppressWarnings("unchecked")
public static void initJob(String table, String columns,
Class<? extends TableMap> mapper, JobConf job) {
TableMapReduceUtil.initTableMapJob(table, columns, mapper,
Text.class,
Text.class, job);
}
public void map(ImmutableBytesWritable key, RowResult value,
OutputCollector<Text,Text> output,
@SuppressWarnings("unused") Reporter reporter) throws IOException {
output.collect(new Text("ss"), new Text("pp"));
}
}
When i compile this code .. i get the following error ....
MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is not
abstract and does not override abstract method
map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
in org.apache.hadoop.mapred.Mapper
public class MyTableMap
^
Can anyone help me out!!
Re: Implementing TableMap interface
Posted by Ninad Raut <hb...@gmail.com>.
Or else you can try this if u want to run something from the IDE
Configuration conf = StudyRepository.getConf(studyId);
JobShell jobShell = new JobShell(conf);
String args[] = new String[]{"/root/hadoop-0.19.0/EchoOhce.jar",
"com.javaworld.mapreduce.EchoOhce" , "/user/root/data", "output"};
int result = jobShell.run(args);
if (result == 0) {
return true;
} else {
return false;
}
On Fri, Jul 24, 2009 at 8:06 PM, bharath vissapragada <
bharathvissapragada1990@gmail.com> wrote:
> Thanks J-D for giving this useful advise ... I have figured out my error ..
> and i'll try to follow what you have said in the previous mail .
>
> Thanks!
>
> 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
>
> > This is not a very java-way to do it. For example if you have let's
> > say 20 mapreduce jobs and you keep them in a version control system,
> > you will have to copy over the classes every time they change into the
> > hbase path... is that a good practice?
> >
> > The exception you get says "wrong name", a google search will tell you
> > how to solve that.
> >
> > So yes I say that you must provide your own jar, this is how it works
> > actually. Please have a look at the MR tutorial in the Hadoop doc
> > http://hadoop.apache.org/common/docs/r0.19.1/mapred_tutorial.html
> >
> > If you only want to run your job in local mode and run it by bin/hbase
> > then you should put your jar either in the lib folder or in the HBase
> > classpath in conf/hbase-env.sh if you want to keep it somewhere else.
> >
> > Hope this helps,
> >
> > J-D
> >
> > On Fri, Jul 24, 2009 at 9:29 AM, bharath
> > vissapragada<bh...@gmail.com> wrote:
> > > Actually u might have seen that i have implemented the TableMap
> interface
> > > using MyTableMap class so that map can return <Text,Text> pairs .. i
> have
> > > kept the package name to be so and so .., otherwise it is not
> recognising
> > > TableMap interface and throwing an error .. such a package name implies
> > that
> > > hbase will find the the class from <packagename as a
> > path>/MyTableMap.class
> > > ... so what you say is should we create such a jar and provide it while
> > > running the program?
> > >
> > > Initially what i did was .. i added the class file to that path in
> > hbase-jar
> > > and the program compiled succesfully ...
> > > but while running the class file using "bin/hbase <MRprogram>" it threw
> > an
> > > error as follows ...
> > >
> > > Exception in thread "main" java.lang.NoClassDefFoundError: MyTableMap
> > (wrong
> > > name: org/apache/hadoop/hbase/mapred/MyTableMap)
> > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > at java.lang.ClassLoader.defineClass(ClassLoader.java:637)
> > > at
> > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > > at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
> > > at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> > > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:336)
> > > Could not find the main class: MyTableMap. Program will exit.
> > >
> > > Hoping for a reply!
> > >
> > > 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> > >
> > >> I really wonder why you would want to do that... Because you don't
> > >> want to import the classes from that package? Anyways you don't have
> > >> to be in that jar to be in that package, it should still resolve the
> > >> full name of your class as long as it is somewhere in the Hadoop
> > >> classpath. Thus, you have to add the HBase jar in Hadoop classpath and
> > >> when you run your job you have to provide your own jar.
> > >>
> > >> I hope this helps.
> > >>
> > >> J-D
> > >>
> > >> 2009/7/24 john smith <js...@gmail.com>:
> > >> > Thanks for replying Daniel,
> > >> >
> > >> > The link you have provided gives the info about adding a new jar to
> > the
> > >> > classpath .. Thats fine ...
> > >> > Suppose consider the above program written by Bharath ..
> > >> >
> > >> > It contains ...
> > >> >
> > >> > package "org.apache.hadoop.hbase.mapred" ..
> > >> >
> > >> > This means that the new class should be in that jar in that
> particular
> > >> > folder .. I think this is different to adding a new jar to
> classpath
> > ..
> > >> >
> > >> > any comments?
> > >> >
> > >> > 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> > >> >
> > >> >> Please see the doc
> > >> >>
> > >> >>
> > >>
> >
> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapred/package-summary.html#classpath
> > >> >>
> > >> >> J-D
> > >> >>
> > >> >> 2009/7/24 john smith <js...@gmail.com>:
> > >> >> > Yes i too have the same problem .. Can anyone tell me in detail
> how
> > to
> > >> >> > add new classes to the existing hbase jar or do we have a
> different
> > >> >> > method to include our own classes in the program ..
> > >> >> >
> > >> >> > On 7/24/09, bharath vissapragada <
> > bharathvissapragada1990@gmail.com>
> > >> >> wrote:
> > >> >> >> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or
> > is
> > >> >> there
> > >> >> >> some other procedure to use it in my program ... because when i
> > >> updated
> > >> >> >> hbase-x.x.jar .. it gave me NoClassDefFoundError while running
> my
> > >> >> program!!
> > >> >> >>
> > >> >> >>
> > >> >> >>
> > >> >> >> 2009/7/24 Doğacan Güney <do...@gmail.com>
> > >> >> >>
> > >> >> >>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
> > >> >> >>>
> > >> >> >>> Hi all,
> > >> >> >>>>
> > >> >> >>>> I wanted to implement TableMap interface so that "map"
> function
> > can
> > >> >> emit
> > >> >> >>>> <Text,Text>.. I wrote the code as follows
> > >> >> >>>>
> > >> >> >>>>
> > >> >>
> > >>
> >
> -------------------------------------------------------------------------------------------------------
> > >> >> >>>> package org.apache.hadoop.hbase.mapred;
> > >> >> >>>>
> > >> >> >>>> import java.io.IOException;
> > >> >> >>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> > >> >> >>>> import org.apache.hadoop.hbase.io.RowResult;
> > >> >> >>>> import org.apache.hadoop.io.Text;
> > >> >> >>>> import org.apache.hadoop.mapred.JobConf;
> > >> >> >>>> import org.apache.hadoop.mapred.MapReduceBase;
> > >> >> >>>> import org.apache.hadoop.mapred.OutputCollector;
> > >> >> >>>> import org.apache.hadoop.mapred.Reporter;
> > >> >> >>>>
> > >> >> >>>> public class MyTableMap
> > >> >> >>>> extends MapReduceBase
> > >> >> >>>> implements TableMap<ImmutableBytesWritable, RowResult> {
> > >> >> >>>>
> > >> >> >>>> /** constructor */
> > >> >> >>>> public MyTableMap() {
> > >> >> >>>> super();
> > >> >> >>>> }
> > >> >> >>>> @SuppressWarnings("unchecked")
> > >> >> >>>> public static void initJob(String table, String columns,
> > >> >> >>>> Class<? extends TableMap> mapper, JobConf job) {
> > >> >> >>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
> > >> >> >>>> Text.class,
> > >> >> >>>> Text.class, job);
> > >> >> >>>> }
> > >> >> >>>> public void map(ImmutableBytesWritable key, RowResult value,
> > >> >> >>>> OutputCollector<Text,Text> output,
> > >> >> >>>> @SuppressWarnings("unused") Reporter reporter) throws
> > >> IOException
> > >> >> {
> > >> >> >>>> output.collect(new Text("ss"), new Text("pp"));
> > >> >> >>>> }
> > >> >> >>>> }
> > >> >> >>>>
> > >> >> >>>> When i compile this code .. i get the following error ....
> > >> >> >>>>
> > >> >> >>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap
> is
> > >> not
> > >> >> >>>> abstract and does not override abstract method
> > >> >> >>>>
> > >> >> >>>>
> > >> >>
> > >>
> >
> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
> > >> >> >>>> in org.apache.hadoop.mapred.Mapper
> > >> >> >>>> public class MyTableMap
> > >> >> >>>> ^
> > >> >> >>>> Can anyone help me out!!
> > >> >> >>>>
> > >> >> >>>
> > >> >> >>> Your class should implement TableMap<Text, Text> not
> > TableMap<IBW,
> > >> RR>.
> > >> >> >>>
> > >> >> >>> --
> > >> >> >>> Doğacan Güney
> > >> >> >>
> > >> >> >
> > >> >>
> > >> >
> > >>
> > >
> >
>
Re: Implementing TableMap interface
Posted by bharath vissapragada <bh...@gmail.com>.
Thanks J-D for giving this useful advise ... I have figured out my error ..
and i'll try to follow what you have said in the previous mail .
Thanks!
2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> This is not a very java-way to do it. For example if you have let's
> say 20 mapreduce jobs and you keep them in a version control system,
> you will have to copy over the classes every time they change into the
> hbase path... is that a good practice?
>
> The exception you get says "wrong name", a google search will tell you
> how to solve that.
>
> So yes I say that you must provide your own jar, this is how it works
> actually. Please have a look at the MR tutorial in the Hadoop doc
> http://hadoop.apache.org/common/docs/r0.19.1/mapred_tutorial.html
>
> If you only want to run your job in local mode and run it by bin/hbase
> then you should put your jar either in the lib folder or in the HBase
> classpath in conf/hbase-env.sh if you want to keep it somewhere else.
>
> Hope this helps,
>
> J-D
>
> On Fri, Jul 24, 2009 at 9:29 AM, bharath
> vissapragada<bh...@gmail.com> wrote:
> > Actually u might have seen that i have implemented the TableMap interface
> > using MyTableMap class so that map can return <Text,Text> pairs .. i have
> > kept the package name to be so and so .., otherwise it is not recognising
> > TableMap interface and throwing an error .. such a package name implies
> that
> > hbase will find the the class from <packagename as a
> path>/MyTableMap.class
> > ... so what you say is should we create such a jar and provide it while
> > running the program?
> >
> > Initially what i did was .. i added the class file to that path in
> hbase-jar
> > and the program compiled succesfully ...
> > but while running the class file using "bin/hbase <MRprogram>" it threw
> an
> > error as follows ...
> >
> > Exception in thread "main" java.lang.NoClassDefFoundError: MyTableMap
> (wrong
> > name: org/apache/hadoop/hbase/mapred/MyTableMap)
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:637)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
> > at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:336)
> > Could not find the main class: MyTableMap. Program will exit.
> >
> > Hoping for a reply!
> >
> > 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> >
> >> I really wonder why you would want to do that... Because you don't
> >> want to import the classes from that package? Anyways you don't have
> >> to be in that jar to be in that package, it should still resolve the
> >> full name of your class as long as it is somewhere in the Hadoop
> >> classpath. Thus, you have to add the HBase jar in Hadoop classpath and
> >> when you run your job you have to provide your own jar.
> >>
> >> I hope this helps.
> >>
> >> J-D
> >>
> >> 2009/7/24 john smith <js...@gmail.com>:
> >> > Thanks for replying Daniel,
> >> >
> >> > The link you have provided gives the info about adding a new jar to
> the
> >> > classpath .. Thats fine ...
> >> > Suppose consider the above program written by Bharath ..
> >> >
> >> > It contains ...
> >> >
> >> > package "org.apache.hadoop.hbase.mapred" ..
> >> >
> >> > This means that the new class should be in that jar in that particular
> >> > folder .. I think this is different to adding a new jar to classpath
> ..
> >> >
> >> > any comments?
> >> >
> >> > 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> >> >
> >> >> Please see the doc
> >> >>
> >> >>
> >>
> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapred/package-summary.html#classpath
> >> >>
> >> >> J-D
> >> >>
> >> >> 2009/7/24 john smith <js...@gmail.com>:
> >> >> > Yes i too have the same problem .. Can anyone tell me in detail how
> to
> >> >> > add new classes to the existing hbase jar or do we have a different
> >> >> > method to include our own classes in the program ..
> >> >> >
> >> >> > On 7/24/09, bharath vissapragada <
> bharathvissapragada1990@gmail.com>
> >> >> wrote:
> >> >> >> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or
> is
> >> >> there
> >> >> >> some other procedure to use it in my program ... because when i
> >> updated
> >> >> >> hbase-x.x.jar .. it gave me NoClassDefFoundError while running my
> >> >> program!!
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> 2009/7/24 Doğacan Güney <do...@gmail.com>
> >> >> >>
> >> >> >>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
> >> >> >>>
> >> >> >>> Hi all,
> >> >> >>>>
> >> >> >>>> I wanted to implement TableMap interface so that "map" function
> can
> >> >> emit
> >> >> >>>> <Text,Text>.. I wrote the code as follows
> >> >> >>>>
> >> >> >>>>
> >> >>
> >>
> -------------------------------------------------------------------------------------------------------
> >> >> >>>> package org.apache.hadoop.hbase.mapred;
> >> >> >>>>
> >> >> >>>> import java.io.IOException;
> >> >> >>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> >> >> >>>> import org.apache.hadoop.hbase.io.RowResult;
> >> >> >>>> import org.apache.hadoop.io.Text;
> >> >> >>>> import org.apache.hadoop.mapred.JobConf;
> >> >> >>>> import org.apache.hadoop.mapred.MapReduceBase;
> >> >> >>>> import org.apache.hadoop.mapred.OutputCollector;
> >> >> >>>> import org.apache.hadoop.mapred.Reporter;
> >> >> >>>>
> >> >> >>>> public class MyTableMap
> >> >> >>>> extends MapReduceBase
> >> >> >>>> implements TableMap<ImmutableBytesWritable, RowResult> {
> >> >> >>>>
> >> >> >>>> /** constructor */
> >> >> >>>> public MyTableMap() {
> >> >> >>>> super();
> >> >> >>>> }
> >> >> >>>> @SuppressWarnings("unchecked")
> >> >> >>>> public static void initJob(String table, String columns,
> >> >> >>>> Class<? extends TableMap> mapper, JobConf job) {
> >> >> >>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
> >> >> >>>> Text.class,
> >> >> >>>> Text.class, job);
> >> >> >>>> }
> >> >> >>>> public void map(ImmutableBytesWritable key, RowResult value,
> >> >> >>>> OutputCollector<Text,Text> output,
> >> >> >>>> @SuppressWarnings("unused") Reporter reporter) throws
> >> IOException
> >> >> {
> >> >> >>>> output.collect(new Text("ss"), new Text("pp"));
> >> >> >>>> }
> >> >> >>>> }
> >> >> >>>>
> >> >> >>>> When i compile this code .. i get the following error ....
> >> >> >>>>
> >> >> >>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is
> >> not
> >> >> >>>> abstract and does not override abstract method
> >> >> >>>>
> >> >> >>>>
> >> >>
> >>
> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
> >> >> >>>> in org.apache.hadoop.mapred.Mapper
> >> >> >>>> public class MyTableMap
> >> >> >>>> ^
> >> >> >>>> Can anyone help me out!!
> >> >> >>>>
> >> >> >>>
> >> >> >>> Your class should implement TableMap<Text, Text> not
> TableMap<IBW,
> >> RR>.
> >> >> >>>
> >> >> >>> --
> >> >> >>> Doğacan Güney
> >> >> >>
> >> >> >
> >> >>
> >> >
> >>
> >
>
Re: Implementing TableMap interface
Posted by Jean-Daniel Cryans <jd...@apache.org>.
This is not a very java-way to do it. For example if you have let's
say 20 mapreduce jobs and you keep them in a version control system,
you will have to copy over the classes every time they change into the
hbase path... is that a good practice?
The exception you get says "wrong name", a google search will tell you
how to solve that.
So yes I say that you must provide your own jar, this is how it works
actually. Please have a look at the MR tutorial in the Hadoop doc
http://hadoop.apache.org/common/docs/r0.19.1/mapred_tutorial.html
If you only want to run your job in local mode and run it by bin/hbase
then you should put your jar either in the lib folder or in the HBase
classpath in conf/hbase-env.sh if you want to keep it somewhere else.
Hope this helps,
J-D
On Fri, Jul 24, 2009 at 9:29 AM, bharath
vissapragada<bh...@gmail.com> wrote:
> Actually u might have seen that i have implemented the TableMap interface
> using MyTableMap class so that map can return <Text,Text> pairs .. i have
> kept the package name to be so and so .., otherwise it is not recognising
> TableMap interface and throwing an error .. such a package name implies that
> hbase will find the the class from <packagename as a path>/MyTableMap.class
> ... so what you say is should we create such a jar and provide it while
> running the program?
>
> Initially what i did was .. i added the class file to that path in hbase-jar
> and the program compiled succesfully ...
> but while running the class file using "bin/hbase <MRprogram>" it threw an
> error as follows ...
>
> Exception in thread "main" java.lang.NoClassDefFoundError: MyTableMap (wrong
> name: org/apache/hadoop/hbase/mapred/MyTableMap)
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:637)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
> at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:336)
> Could not find the main class: MyTableMap. Program will exit.
>
> Hoping for a reply!
>
> 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
>
>> I really wonder why you would want to do that... Because you don't
>> want to import the classes from that package? Anyways you don't have
>> to be in that jar to be in that package, it should still resolve the
>> full name of your class as long as it is somewhere in the Hadoop
>> classpath. Thus, you have to add the HBase jar in Hadoop classpath and
>> when you run your job you have to provide your own jar.
>>
>> I hope this helps.
>>
>> J-D
>>
>> 2009/7/24 john smith <js...@gmail.com>:
>> > Thanks for replying Daniel,
>> >
>> > The link you have provided gives the info about adding a new jar to the
>> > classpath .. Thats fine ...
>> > Suppose consider the above program written by Bharath ..
>> >
>> > It contains ...
>> >
>> > package "org.apache.hadoop.hbase.mapred" ..
>> >
>> > This means that the new class should be in that jar in that particular
>> > folder .. I think this is different to adding a new jar to classpath ..
>> >
>> > any comments?
>> >
>> > 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
>> >
>> >> Please see the doc
>> >>
>> >>
>> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapred/package-summary.html#classpath
>> >>
>> >> J-D
>> >>
>> >> 2009/7/24 john smith <js...@gmail.com>:
>> >> > Yes i too have the same problem .. Can anyone tell me in detail how to
>> >> > add new classes to the existing hbase jar or do we have a different
>> >> > method to include our own classes in the program ..
>> >> >
>> >> > On 7/24/09, bharath vissapragada <bh...@gmail.com>
>> >> wrote:
>> >> >> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or is
>> >> there
>> >> >> some other procedure to use it in my program ... because when i
>> updated
>> >> >> hbase-x.x.jar .. it gave me NoClassDefFoundError while running my
>> >> program!!
>> >> >>
>> >> >>
>> >> >>
>> >> >> 2009/7/24 Doğacan Güney <do...@gmail.com>
>> >> >>
>> >> >>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
>> >> >>>
>> >> >>> Hi all,
>> >> >>>>
>> >> >>>> I wanted to implement TableMap interface so that "map" function can
>> >> emit
>> >> >>>> <Text,Text>.. I wrote the code as follows
>> >> >>>>
>> >> >>>>
>> >>
>> -------------------------------------------------------------------------------------------------------
>> >> >>>> package org.apache.hadoop.hbase.mapred;
>> >> >>>>
>> >> >>>> import java.io.IOException;
>> >> >>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
>> >> >>>> import org.apache.hadoop.hbase.io.RowResult;
>> >> >>>> import org.apache.hadoop.io.Text;
>> >> >>>> import org.apache.hadoop.mapred.JobConf;
>> >> >>>> import org.apache.hadoop.mapred.MapReduceBase;
>> >> >>>> import org.apache.hadoop.mapred.OutputCollector;
>> >> >>>> import org.apache.hadoop.mapred.Reporter;
>> >> >>>>
>> >> >>>> public class MyTableMap
>> >> >>>> extends MapReduceBase
>> >> >>>> implements TableMap<ImmutableBytesWritable, RowResult> {
>> >> >>>>
>> >> >>>> /** constructor */
>> >> >>>> public MyTableMap() {
>> >> >>>> super();
>> >> >>>> }
>> >> >>>> @SuppressWarnings("unchecked")
>> >> >>>> public static void initJob(String table, String columns,
>> >> >>>> Class<? extends TableMap> mapper, JobConf job) {
>> >> >>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
>> >> >>>> Text.class,
>> >> >>>> Text.class, job);
>> >> >>>> }
>> >> >>>> public void map(ImmutableBytesWritable key, RowResult value,
>> >> >>>> OutputCollector<Text,Text> output,
>> >> >>>> @SuppressWarnings("unused") Reporter reporter) throws
>> IOException
>> >> {
>> >> >>>> output.collect(new Text("ss"), new Text("pp"));
>> >> >>>> }
>> >> >>>> }
>> >> >>>>
>> >> >>>> When i compile this code .. i get the following error ....
>> >> >>>>
>> >> >>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is
>> not
>> >> >>>> abstract and does not override abstract method
>> >> >>>>
>> >> >>>>
>> >>
>> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
>> >> >>>> in org.apache.hadoop.mapred.Mapper
>> >> >>>> public class MyTableMap
>> >> >>>> ^
>> >> >>>> Can anyone help me out!!
>> >> >>>>
>> >> >>>
>> >> >>> Your class should implement TableMap<Text, Text> not TableMap<IBW,
>> RR>.
>> >> >>>
>> >> >>> --
>> >> >>> Doğacan Güney
>> >> >>
>> >> >
>> >>
>> >
>>
>
Re: Implementing TableMap interface
Posted by bharath vissapragada <bh...@gmail.com>.
Actually u might have seen that i have implemented the TableMap interface
using MyTableMap class so that map can return <Text,Text> pairs .. i have
kept the package name to be so and so .., otherwise it is not recognising
TableMap interface and throwing an error .. such a package name implies that
hbase will find the the class from <packagename as a path>/MyTableMap.class
... so what you say is should we create such a jar and provide it while
running the program?
Initially what i did was .. i added the class file to that path in hbase-jar
and the program compiled succesfully ...
but while running the class file using "bin/hbase <MRprogram>" it threw an
error as follows ...
Exception in thread "main" java.lang.NoClassDefFoundError: MyTableMap (wrong
name: org/apache/hadoop/hbase/mapred/MyTableMap)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:637)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:336)
Could not find the main class: MyTableMap. Program will exit.
Hoping for a reply!
2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> I really wonder why you would want to do that... Because you don't
> want to import the classes from that package? Anyways you don't have
> to be in that jar to be in that package, it should still resolve the
> full name of your class as long as it is somewhere in the Hadoop
> classpath. Thus, you have to add the HBase jar in Hadoop classpath and
> when you run your job you have to provide your own jar.
>
> I hope this helps.
>
> J-D
>
> 2009/7/24 john smith <js...@gmail.com>:
> > Thanks for replying Daniel,
> >
> > The link you have provided gives the info about adding a new jar to the
> > classpath .. Thats fine ...
> > Suppose consider the above program written by Bharath ..
> >
> > It contains ...
> >
> > package "org.apache.hadoop.hbase.mapred" ..
> >
> > This means that the new class should be in that jar in that particular
> > folder .. I think this is different to adding a new jar to classpath ..
> >
> > any comments?
> >
> > 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> >
> >> Please see the doc
> >>
> >>
> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapred/package-summary.html#classpath
> >>
> >> J-D
> >>
> >> 2009/7/24 john smith <js...@gmail.com>:
> >> > Yes i too have the same problem .. Can anyone tell me in detail how to
> >> > add new classes to the existing hbase jar or do we have a different
> >> > method to include our own classes in the program ..
> >> >
> >> > On 7/24/09, bharath vissapragada <bh...@gmail.com>
> >> wrote:
> >> >> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or is
> >> there
> >> >> some other procedure to use it in my program ... because when i
> updated
> >> >> hbase-x.x.jar .. it gave me NoClassDefFoundError while running my
> >> program!!
> >> >>
> >> >>
> >> >>
> >> >> 2009/7/24 Doğacan Güney <do...@gmail.com>
> >> >>
> >> >>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
> >> >>>
> >> >>> Hi all,
> >> >>>>
> >> >>>> I wanted to implement TableMap interface so that "map" function can
> >> emit
> >> >>>> <Text,Text>.. I wrote the code as follows
> >> >>>>
> >> >>>>
> >>
> -------------------------------------------------------------------------------------------------------
> >> >>>> package org.apache.hadoop.hbase.mapred;
> >> >>>>
> >> >>>> import java.io.IOException;
> >> >>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> >> >>>> import org.apache.hadoop.hbase.io.RowResult;
> >> >>>> import org.apache.hadoop.io.Text;
> >> >>>> import org.apache.hadoop.mapred.JobConf;
> >> >>>> import org.apache.hadoop.mapred.MapReduceBase;
> >> >>>> import org.apache.hadoop.mapred.OutputCollector;
> >> >>>> import org.apache.hadoop.mapred.Reporter;
> >> >>>>
> >> >>>> public class MyTableMap
> >> >>>> extends MapReduceBase
> >> >>>> implements TableMap<ImmutableBytesWritable, RowResult> {
> >> >>>>
> >> >>>> /** constructor */
> >> >>>> public MyTableMap() {
> >> >>>> super();
> >> >>>> }
> >> >>>> @SuppressWarnings("unchecked")
> >> >>>> public static void initJob(String table, String columns,
> >> >>>> Class<? extends TableMap> mapper, JobConf job) {
> >> >>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
> >> >>>> Text.class,
> >> >>>> Text.class, job);
> >> >>>> }
> >> >>>> public void map(ImmutableBytesWritable key, RowResult value,
> >> >>>> OutputCollector<Text,Text> output,
> >> >>>> @SuppressWarnings("unused") Reporter reporter) throws
> IOException
> >> {
> >> >>>> output.collect(new Text("ss"), new Text("pp"));
> >> >>>> }
> >> >>>> }
> >> >>>>
> >> >>>> When i compile this code .. i get the following error ....
> >> >>>>
> >> >>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is
> not
> >> >>>> abstract and does not override abstract method
> >> >>>>
> >> >>>>
> >>
> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
> >> >>>> in org.apache.hadoop.mapred.Mapper
> >> >>>> public class MyTableMap
> >> >>>> ^
> >> >>>> Can anyone help me out!!
> >> >>>>
> >> >>>
> >> >>> Your class should implement TableMap<Text, Text> not TableMap<IBW,
> RR>.
> >> >>>
> >> >>> --
> >> >>> Doğacan Güney
> >> >>
> >> >
> >>
> >
>
Re: Implementing TableMap interface
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I really wonder why you would want to do that... Because you don't
want to import the classes from that package? Anyways you don't have
to be in that jar to be in that package, it should still resolve the
full name of your class as long as it is somewhere in the Hadoop
classpath. Thus, you have to add the HBase jar in Hadoop classpath and
when you run your job you have to provide your own jar.
I hope this helps.
J-D
2009/7/24 john smith <js...@gmail.com>:
> Thanks for replying Daniel,
>
> The link you have provided gives the info about adding a new jar to the
> classpath .. Thats fine ...
> Suppose consider the above program written by Bharath ..
>
> It contains ...
>
> package "org.apache.hadoop.hbase.mapred" ..
>
> This means that the new class should be in that jar in that particular
> folder .. I think this is different to adding a new jar to classpath ..
>
> any comments?
>
> 2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
>
>> Please see the doc
>>
>> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapred/package-summary.html#classpath
>>
>> J-D
>>
>> 2009/7/24 john smith <js...@gmail.com>:
>> > Yes i too have the same problem .. Can anyone tell me in detail how to
>> > add new classes to the existing hbase jar or do we have a different
>> > method to include our own classes in the program ..
>> >
>> > On 7/24/09, bharath vissapragada <bh...@gmail.com>
>> wrote:
>> >> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or is
>> there
>> >> some other procedure to use it in my program ... because when i updated
>> >> hbase-x.x.jar .. it gave me NoClassDefFoundError while running my
>> program!!
>> >>
>> >>
>> >>
>> >> 2009/7/24 Doğacan Güney <do...@gmail.com>
>> >>
>> >>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
>> >>>
>> >>> Hi all,
>> >>>>
>> >>>> I wanted to implement TableMap interface so that "map" function can
>> emit
>> >>>> <Text,Text>.. I wrote the code as follows
>> >>>>
>> >>>>
>> -------------------------------------------------------------------------------------------------------
>> >>>> package org.apache.hadoop.hbase.mapred;
>> >>>>
>> >>>> import java.io.IOException;
>> >>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
>> >>>> import org.apache.hadoop.hbase.io.RowResult;
>> >>>> import org.apache.hadoop.io.Text;
>> >>>> import org.apache.hadoop.mapred.JobConf;
>> >>>> import org.apache.hadoop.mapred.MapReduceBase;
>> >>>> import org.apache.hadoop.mapred.OutputCollector;
>> >>>> import org.apache.hadoop.mapred.Reporter;
>> >>>>
>> >>>> public class MyTableMap
>> >>>> extends MapReduceBase
>> >>>> implements TableMap<ImmutableBytesWritable, RowResult> {
>> >>>>
>> >>>> /** constructor */
>> >>>> public MyTableMap() {
>> >>>> super();
>> >>>> }
>> >>>> @SuppressWarnings("unchecked")
>> >>>> public static void initJob(String table, String columns,
>> >>>> Class<? extends TableMap> mapper, JobConf job) {
>> >>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
>> >>>> Text.class,
>> >>>> Text.class, job);
>> >>>> }
>> >>>> public void map(ImmutableBytesWritable key, RowResult value,
>> >>>> OutputCollector<Text,Text> output,
>> >>>> @SuppressWarnings("unused") Reporter reporter) throws IOException
>> {
>> >>>> output.collect(new Text("ss"), new Text("pp"));
>> >>>> }
>> >>>> }
>> >>>>
>> >>>> When i compile this code .. i get the following error ....
>> >>>>
>> >>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is not
>> >>>> abstract and does not override abstract method
>> >>>>
>> >>>>
>> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>>> public class MyTableMap
>> >>>> ^
>> >>>> Can anyone help me out!!
>> >>>>
>> >>>
>> >>> Your class should implement TableMap<Text, Text> not TableMap<IBW, RR>.
>> >>>
>> >>> --
>> >>> Doğacan Güney
>> >>
>> >
>>
>
Re: Implementing TableMap interface
Posted by john smith <js...@gmail.com>.
Thanks for replying Daniel,
The link you have provided gives the info about adding a new jar to the
classpath .. Thats fine ...
Suppose consider the above program written by Bharath ..
It contains ...
package "org.apache.hadoop.hbase.mapred" ..
This means that the new class should be in that jar in that particular
folder .. I think this is different to adding a new jar to classpath ..
any comments?
2009/7/24 Jean-Daniel Cryans <jd...@apache.org>
> Please see the doc
>
> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapred/package-summary.html#classpath
>
> J-D
>
> 2009/7/24 john smith <js...@gmail.com>:
> > Yes i too have the same problem .. Can anyone tell me in detail how to
> > add new classes to the existing hbase jar or do we have a different
> > method to include our own classes in the program ..
> >
> > On 7/24/09, bharath vissapragada <bh...@gmail.com>
> wrote:
> >> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or is
> there
> >> some other procedure to use it in my program ... because when i updated
> >> hbase-x.x.jar .. it gave me NoClassDefFoundError while running my
> program!!
> >>
> >>
> >>
> >> 2009/7/24 Doğacan Güney <do...@gmail.com>
> >>
> >>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
> >>>
> >>> Hi all,
> >>>>
> >>>> I wanted to implement TableMap interface so that "map" function can
> emit
> >>>> <Text,Text>.. I wrote the code as follows
> >>>>
> >>>>
> -------------------------------------------------------------------------------------------------------
> >>>> package org.apache.hadoop.hbase.mapred;
> >>>>
> >>>> import java.io.IOException;
> >>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> >>>> import org.apache.hadoop.hbase.io.RowResult;
> >>>> import org.apache.hadoop.io.Text;
> >>>> import org.apache.hadoop.mapred.JobConf;
> >>>> import org.apache.hadoop.mapred.MapReduceBase;
> >>>> import org.apache.hadoop.mapred.OutputCollector;
> >>>> import org.apache.hadoop.mapred.Reporter;
> >>>>
> >>>> public class MyTableMap
> >>>> extends MapReduceBase
> >>>> implements TableMap<ImmutableBytesWritable, RowResult> {
> >>>>
> >>>> /** constructor */
> >>>> public MyTableMap() {
> >>>> super();
> >>>> }
> >>>> @SuppressWarnings("unchecked")
> >>>> public static void initJob(String table, String columns,
> >>>> Class<? extends TableMap> mapper, JobConf job) {
> >>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
> >>>> Text.class,
> >>>> Text.class, job);
> >>>> }
> >>>> public void map(ImmutableBytesWritable key, RowResult value,
> >>>> OutputCollector<Text,Text> output,
> >>>> @SuppressWarnings("unused") Reporter reporter) throws IOException
> {
> >>>> output.collect(new Text("ss"), new Text("pp"));
> >>>> }
> >>>> }
> >>>>
> >>>> When i compile this code .. i get the following error ....
> >>>>
> >>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is not
> >>>> abstract and does not override abstract method
> >>>>
> >>>>
> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
> >>>> in org.apache.hadoop.mapred.Mapper
> >>>> public class MyTableMap
> >>>> ^
> >>>> Can anyone help me out!!
> >>>>
> >>>
> >>> Your class should implement TableMap<Text, Text> not TableMap<IBW, RR>.
> >>>
> >>> --
> >>> Doğacan Güney
> >>
> >
>
Re: Implementing TableMap interface
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Please see the doc
http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/mapred/package-summary.html#classpath
J-D
2009/7/24 john smith <js...@gmail.com>:
> Yes i too have the same problem .. Can anyone tell me in detail how to
> add new classes to the existing hbase jar or do we have a different
> method to include our own classes in the program ..
>
> On 7/24/09, bharath vissapragada <bh...@gmail.com> wrote:
>> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or is there
>> some other procedure to use it in my program ... because when i updated
>> hbase-x.x.jar .. it gave me NoClassDefFoundError while running my program!!
>>
>>
>>
>> 2009/7/24 Doğacan Güney <do...@gmail.com>
>>
>>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
>>>
>>> Hi all,
>>>>
>>>> I wanted to implement TableMap interface so that "map" function can emit
>>>> <Text,Text>.. I wrote the code as follows
>>>>
>>>> -------------------------------------------------------------------------------------------------------
>>>> package org.apache.hadoop.hbase.mapred;
>>>>
>>>> import java.io.IOException;
>>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
>>>> import org.apache.hadoop.hbase.io.RowResult;
>>>> import org.apache.hadoop.io.Text;
>>>> import org.apache.hadoop.mapred.JobConf;
>>>> import org.apache.hadoop.mapred.MapReduceBase;
>>>> import org.apache.hadoop.mapred.OutputCollector;
>>>> import org.apache.hadoop.mapred.Reporter;
>>>>
>>>> public class MyTableMap
>>>> extends MapReduceBase
>>>> implements TableMap<ImmutableBytesWritable, RowResult> {
>>>>
>>>> /** constructor */
>>>> public MyTableMap() {
>>>> super();
>>>> }
>>>> @SuppressWarnings("unchecked")
>>>> public static void initJob(String table, String columns,
>>>> Class<? extends TableMap> mapper, JobConf job) {
>>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
>>>> Text.class,
>>>> Text.class, job);
>>>> }
>>>> public void map(ImmutableBytesWritable key, RowResult value,
>>>> OutputCollector<Text,Text> output,
>>>> @SuppressWarnings("unused") Reporter reporter) throws IOException {
>>>> output.collect(new Text("ss"), new Text("pp"));
>>>> }
>>>> }
>>>>
>>>> When i compile this code .. i get the following error ....
>>>>
>>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is not
>>>> abstract and does not override abstract method
>>>>
>>>> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
>>>> in org.apache.hadoop.mapred.Mapper
>>>> public class MyTableMap
>>>> ^
>>>> Can anyone help me out!!
>>>>
>>>
>>> Your class should implement TableMap<Text, Text> not TableMap<IBW, RR>.
>>>
>>> --
>>> Doğacan Güney
>>
>
Re: Implementing TableMap interface
Posted by john smith <js...@gmail.com>.
Yes i too have the same problem .. Can anyone tell me in detail how to
add new classes to the existing hbase jar or do we have a different
method to include our own classes in the program ..
On 7/24/09, bharath vissapragada <bh...@gmail.com> wrote:
> Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or is there
> some other procedure to use it in my program ... because when i updated
> hbase-x.x.jar .. it gave me NoClassDefFoundError while running my program!!
>
>
>
> 2009/7/24 Doğacan Güney <do...@gmail.com>
>
>> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
>>
>> Hi all,
>>>
>>> I wanted to implement TableMap interface so that "map" function can emit
>>> <Text,Text>.. I wrote the code as follows
>>>
>>> -------------------------------------------------------------------------------------------------------
>>> package org.apache.hadoop.hbase.mapred;
>>>
>>> import java.io.IOException;
>>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
>>> import org.apache.hadoop.hbase.io.RowResult;
>>> import org.apache.hadoop.io.Text;
>>> import org.apache.hadoop.mapred.JobConf;
>>> import org.apache.hadoop.mapred.MapReduceBase;
>>> import org.apache.hadoop.mapred.OutputCollector;
>>> import org.apache.hadoop.mapred.Reporter;
>>>
>>> public class MyTableMap
>>> extends MapReduceBase
>>> implements TableMap<ImmutableBytesWritable, RowResult> {
>>>
>>> /** constructor */
>>> public MyTableMap() {
>>> super();
>>> }
>>> @SuppressWarnings("unchecked")
>>> public static void initJob(String table, String columns,
>>> Class<? extends TableMap> mapper, JobConf job) {
>>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
>>> Text.class,
>>> Text.class, job);
>>> }
>>> public void map(ImmutableBytesWritable key, RowResult value,
>>> OutputCollector<Text,Text> output,
>>> @SuppressWarnings("unused") Reporter reporter) throws IOException {
>>> output.collect(new Text("ss"), new Text("pp"));
>>> }
>>> }
>>>
>>> When i compile this code .. i get the following error ....
>>>
>>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is not
>>> abstract and does not override abstract method
>>>
>>> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
>>> in org.apache.hadoop.mapred.Mapper
>>> public class MyTableMap
>>> ^
>>> Can anyone help me out!!
>>>
>>
>> Your class should implement TableMap<Text, Text> not TableMap<IBW, RR>.
>>
>> --
>> Doğacan Güney
>
Re: Implementing TableMap interface
Posted by bharath vissapragada <bh...@gmail.com>.
Thanks it worked fine .. Do i need to update hbase-x.x.jar ?? or is there
some other procedure to use it in my program ... because when i updated
hbase-x.x.jar .. it gave me NoClassDefFoundError while running my program!!
2009/7/24 Doğacan Güney <do...@gmail.com>
> n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
>
> Hi all,
>>
>> I wanted to implement TableMap interface so that "map" function can emit
>> <Text,Text>.. I wrote the code as follows
>>
>> -------------------------------------------------------------------------------------------------------
>> package org.apache.hadoop.hbase.mapred;
>>
>> import java.io.IOException;
>> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
>> import org.apache.hadoop.hbase.io.RowResult;
>> import org.apache.hadoop.io.Text;
>> import org.apache.hadoop.mapred.JobConf;
>> import org.apache.hadoop.mapred.MapReduceBase;
>> import org.apache.hadoop.mapred.OutputCollector;
>> import org.apache.hadoop.mapred.Reporter;
>>
>> public class MyTableMap
>> extends MapReduceBase
>> implements TableMap<ImmutableBytesWritable, RowResult> {
>>
>> /** constructor */
>> public MyTableMap() {
>> super();
>> }
>> @SuppressWarnings("unchecked")
>> public static void initJob(String table, String columns,
>> Class<? extends TableMap> mapper, JobConf job) {
>> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
>> Text.class,
>> Text.class, job);
>> }
>> public void map(ImmutableBytesWritable key, RowResult value,
>> OutputCollector<Text,Text> output,
>> @SuppressWarnings("unused") Reporter reporter) throws IOException {
>> output.collect(new Text("ss"), new Text("pp"));
>> }
>> }
>>
>> When i compile this code .. i get the following error ....
>>
>> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is not
>> abstract and does not override abstract method
>>
>> map(org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.hbase.io.ImmutableBytesWritable,org.apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
>> in org.apache.hadoop.mapred.Mapper
>> public class MyTableMap
>> ^
>> Can anyone help me out!!
>>
>
> Your class should implement TableMap<Text, Text> not TableMap<IBW, RR>.
>
> --
> Doğacan Güney
Re: Implementing TableMap interface
Posted by Doğacan Güney <do...@gmail.com>.
n Jul 24, 2009, at 1:00 PM, bharath vissapragada wrote:
> Hi all,
>
> I wanted to implement TableMap interface so that "map" function can
> emit
> <Text,Text>.. I wrote the code as follows
> -------------------------------------------------------------------------------------------------------
> package org.apache.hadoop.hbase.mapred;
>
> import java.io.IOException;
> import org.apache.hadoop.hbase.io.ImmutableBytesWritable;
> import org.apache.hadoop.hbase.io.RowResult;
> import org.apache.hadoop.io.Text;
> import org.apache.hadoop.mapred.JobConf;
> import org.apache.hadoop.mapred.MapReduceBase;
> import org.apache.hadoop.mapred.OutputCollector;
> import org.apache.hadoop.mapred.Reporter;
>
> public class MyTableMap
> extends MapReduceBase
> implements TableMap<ImmutableBytesWritable, RowResult> {
>
> /** constructor */
> public MyTableMap() {
> super();
> }
> @SuppressWarnings("unchecked")
> public static void initJob(String table, String columns,
> Class<? extends TableMap> mapper, JobConf job) {
> TableMapReduceUtil.initTableMapJob(table, columns, mapper,
> Text.class,
> Text.class, job);
> }
> public void map(ImmutableBytesWritable key, RowResult value,
> OutputCollector<Text,Text> output,
> @SuppressWarnings("unused") Reporter reporter) throws
> IOException {
> output.collect(new Text("ss"), new Text("pp"));
> }
> }
>
> When i compile this code .. i get the following error ....
>
> MyTableMap.java:35: org.apache.hadoop.hbase.mapred.MyTableMap is not
> abstract and does not override abstract method
> map
> (org
> .apache
> .hadoop
> .hbase
> .io
> .ImmutableBytesWritable
> ,org
> .apache
> .hadoop
> .hbase
> .io
> .RowResult
> ,org
> .apache
> .hadoop
> .mapred
> .OutputCollector
> <
> org
> .apache
> .hadoop
> .hbase
> .io
> .ImmutableBytesWritable
> ,org
> .apache.hadoop.hbase.io.RowResult>,org.apache.hadoop.mapred.Reporter)
> in org.apache.hadoop.mapred.Mapper
> public class MyTableMap
> ^
> Can anyone help me out!!
Your class should implement TableMap<Text, Text> not TableMap<IBW, RR>.
--
Doğacan Güney