You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by ch huang <ju...@gmail.com> on 2014/05/21 13:07:03 UTC
how to get the number of total rows of a table
hi,maillist:
i use count 'table' in hbase shell,but it very slow,how can i
get rows of a table quickly?
Re: how to get the number of total rows of a table
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
You are missing the jars in the classpath.
Try something like this:
export CLASSPATH=`hbase classpath`:`hadoop classpath`:.
2014-05-21 20:44 GMT-04:00 ch huang <ju...@gmail.com>:
> # sudo -u hbase hadoop jar /usr/lib/hbase/hbase.jar rowcounter
> -Dmapred.map.tasks.speculative.execution=false
> -Dhbase.client.scanner.caching=100 dsp_cookie_mapping
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/Job
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
> at java.lang.Class.getMethod0(Class.java:2774)
> at java.lang.Class.getMethod(Class.java:1663)
> at
>
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.<init>(ProgramDriver.java:60)
> at
> org.apache.hadoop.util.ProgramDriver.addClass(ProgramDriver.java:103)
> at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:36)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.Job
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 12 more
> # rpm -qa|grep hbase
> hbase-regionserver-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
> hbase-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
>
>
>
> On Wed, May 21, 2014 at 7:18 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
> > You can use the MR method:
> >
> > hadoop jar /home/hbase/hbase-0.94.3/hbase-0.94.19.jar rowcounter
> > -Dmapred.map.tasks.speculative.execution=false
> > -Dhbase.client.scanner.caching=100 work_proposed
> >
> > JM
> >
> >
> > 2014-05-21 7:07 GMT-04:00 ch huang <ju...@gmail.com>:
> >
> > > hi,maillist:
> > > i use count 'table' in hbase shell,but it very slow,how can
> i
> > > get rows of a table quickly?
> > >
> >
>
Re: how to get the number of total rows of a table
Posted by ch huang <ju...@gmail.com>.
it is run as a MR job
On Thu, May 22, 2014 at 8:54 AM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Is it correctly kicking the MR job? Or is it running local only? If former,
> then it's perfect!
>
>
>
> 2014-05-21 20:51 GMT-04:00 ch huang <ju...@gmail.com>:
>
> > i use this command ,it works fine
> >
> > # hbase org.apache.hadoop.hbase.mapreduce.RowCounter dsp_cookie_mapping
> >
> >
> > On Thu, May 22, 2014 at 8:44 AM, ch huang <ju...@gmail.com> wrote:
> >
> > > # sudo -u hbase hadoop jar /usr/lib/hbase/hbase.jar rowcounter
> > > -Dmapred.map.tasks.speculative.execution=false
> > > -Dhbase.client.scanner.caching=100 dsp_cookie_mapping
> > > Exception in thread "main" java.lang.NoClassDefFoundError:
> > > org/apache/hadoop/mapreduce/Job
> > > at java.lang.Class.getDeclaredMethods0(Native Method)
> > > at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
> > > at java.lang.Class.getMethod0(Class.java:2774)
> > > at java.lang.Class.getMethod(Class.java:1663)
> > > at
> > >
> >
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.<init>(ProgramDriver.java:60)
> > > at
> > > org.apache.hadoop.util.ProgramDriver.addClass(ProgramDriver.java:103)
> > > at
> org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:36)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at java.lang.reflect.Method.invoke(Method.java:606)
> > > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> > > Caused by: java.lang.ClassNotFoundException:
> > > org.apache.hadoop.mapreduce.Job
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > ... 12 more
> > > # rpm -qa|grep hbase
> > > hbase-regionserver-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
> > > hbase-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
> > >
> > >
> > >
> > > On Wed, May 21, 2014 at 7:18 PM, Jean-Marc Spaggiari <
> > > jean-marc@spaggiari.org> wrote:
> > >
> > >> You can use the MR method:
> > >>
> > >> hadoop jar /home/hbase/hbase-0.94.3/hbase-0.94.19.jar rowcounter
> > >> -Dmapred.map.tasks.speculative.execution=false
> > >> -Dhbase.client.scanner.caching=100 work_proposed
> > >>
> > >> JM
> > >>
> > >>
> > >> 2014-05-21 7:07 GMT-04:00 ch huang <ju...@gmail.com>:
> > >>
> > >> > hi,maillist:
> > >> > i use count 'table' in hbase shell,but it very slow,how
> > can i
> > >> > get rows of a table quickly?
> > >> >
> > >>
> > >
> > >
> >
>
Re: how to get the number of total rows of a table
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Is it correctly kicking the MR job? Or is it running local only? If former,
then it's perfect!
2014-05-21 20:51 GMT-04:00 ch huang <ju...@gmail.com>:
> i use this command ,it works fine
>
> # hbase org.apache.hadoop.hbase.mapreduce.RowCounter dsp_cookie_mapping
>
>
> On Thu, May 22, 2014 at 8:44 AM, ch huang <ju...@gmail.com> wrote:
>
> > # sudo -u hbase hadoop jar /usr/lib/hbase/hbase.jar rowcounter
> > -Dmapred.map.tasks.speculative.execution=false
> > -Dhbase.client.scanner.caching=100 dsp_cookie_mapping
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > org/apache/hadoop/mapreduce/Job
> > at java.lang.Class.getDeclaredMethods0(Native Method)
> > at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
> > at java.lang.Class.getMethod0(Class.java:2774)
> > at java.lang.Class.getMethod(Class.java:1663)
> > at
> >
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.<init>(ProgramDriver.java:60)
> > at
> > org.apache.hadoop.util.ProgramDriver.addClass(ProgramDriver.java:103)
> > at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:36)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.hadoop.mapreduce.Job
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > ... 12 more
> > # rpm -qa|grep hbase
> > hbase-regionserver-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
> > hbase-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
> >
> >
> >
> > On Wed, May 21, 2014 at 7:18 PM, Jean-Marc Spaggiari <
> > jean-marc@spaggiari.org> wrote:
> >
> >> You can use the MR method:
> >>
> >> hadoop jar /home/hbase/hbase-0.94.3/hbase-0.94.19.jar rowcounter
> >> -Dmapred.map.tasks.speculative.execution=false
> >> -Dhbase.client.scanner.caching=100 work_proposed
> >>
> >> JM
> >>
> >>
> >> 2014-05-21 7:07 GMT-04:00 ch huang <ju...@gmail.com>:
> >>
> >> > hi,maillist:
> >> > i use count 'table' in hbase shell,but it very slow,how
> can i
> >> > get rows of a table quickly?
> >> >
> >>
> >
> >
>
Re: how to get the number of total rows of a table
Posted by ch huang <ju...@gmail.com>.
i use this command ,it works fine
# hbase org.apache.hadoop.hbase.mapreduce.RowCounter dsp_cookie_mapping
On Thu, May 22, 2014 at 8:44 AM, ch huang <ju...@gmail.com> wrote:
> # sudo -u hbase hadoop jar /usr/lib/hbase/hbase.jar rowcounter
> -Dmapred.map.tasks.speculative.execution=false
> -Dhbase.client.scanner.caching=100 dsp_cookie_mapping
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/Job
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
> at java.lang.Class.getMethod0(Class.java:2774)
> at java.lang.Class.getMethod(Class.java:1663)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.<init>(ProgramDriver.java:60)
> at
> org.apache.hadoop.util.ProgramDriver.addClass(ProgramDriver.java:103)
> at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:36)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.Job
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 12 more
> # rpm -qa|grep hbase
> hbase-regionserver-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
> hbase-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
>
>
>
> On Wed, May 21, 2014 at 7:18 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
>> You can use the MR method:
>>
>> hadoop jar /home/hbase/hbase-0.94.3/hbase-0.94.19.jar rowcounter
>> -Dmapred.map.tasks.speculative.execution=false
>> -Dhbase.client.scanner.caching=100 work_proposed
>>
>> JM
>>
>>
>> 2014-05-21 7:07 GMT-04:00 ch huang <ju...@gmail.com>:
>>
>> > hi,maillist:
>> > i use count 'table' in hbase shell,but it very slow,how can i
>> > get rows of a table quickly?
>> >
>>
>
>
Re: how to get the number of total rows of a table
Posted by ch huang <ju...@gmail.com>.
# sudo -u hbase hadoop jar /usr/lib/hbase/hbase.jar rowcounter
-Dmapred.map.tasks.speculative.execution=false
-Dhbase.client.scanner.caching=100 dsp_cookie_mapping
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/Job
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
at java.lang.Class.getMethod0(Class.java:2774)
at java.lang.Class.getMethod(Class.java:1663)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.<init>(ProgramDriver.java:60)
at
org.apache.hadoop.util.ProgramDriver.addClass(ProgramDriver.java:103)
at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.Job
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 12 more
# rpm -qa|grep hbase
hbase-regionserver-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
hbase-0.94.6+132-1.cdh4.4.0.p0.17.el6.x86_64
On Wed, May 21, 2014 at 7:18 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> You can use the MR method:
>
> hadoop jar /home/hbase/hbase-0.94.3/hbase-0.94.19.jar rowcounter
> -Dmapred.map.tasks.speculative.execution=false
> -Dhbase.client.scanner.caching=100 work_proposed
>
> JM
>
>
> 2014-05-21 7:07 GMT-04:00 ch huang <ju...@gmail.com>:
>
> > hi,maillist:
> > i use count 'table' in hbase shell,but it very slow,how can i
> > get rows of a table quickly?
> >
>
Re: how to get the number of total rows of a table
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
You can use the MR method:
hadoop jar /home/hbase/hbase-0.94.3/hbase-0.94.19.jar rowcounter
-Dmapred.map.tasks.speculative.execution=false
-Dhbase.client.scanner.caching=100 work_proposed
JM
2014-05-21 7:07 GMT-04:00 ch huang <ju...@gmail.com>:
> hi,maillist:
> i use count 'table' in hbase shell,but it very slow,how can i
> get rows of a table quickly?
>
Re: how to get the number of total rows of a table
Posted by Ted Yu <yu...@gmail.com>.
Take a look at http://hbase.apache.org/book/ops_mgt.html#rowcounter
Cheers
On May 21, 2014, at 4:07 AM, ch huang <ju...@gmail.com> wrote:
> hi,maillist:
> i use count 'table' in hbase shell,but it very slow,how can i
> get rows of a table quickly?