You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by dh...@tcs.com on 2014/11/28 07:03:41 UTC

Unable to run map reduce in HBase

Hi, 

I am importing a csv file into HBase using the command bin/hbase 
org.apache.hadoop.hbase.mapreduce.ImportTsv 

When I execute the this map reduce program I am getting the following 
error. I am using Hadoop 2.4.1 and HBase 0.98.8-hadoop2

I have set export JAVA_OPTS="-Xms1024m -Xmx10240m" in .bashrc, the server 
has 32 GB of RAM.

2014-11-28 18:56:44,029 INFO [IPC Server listener on 56283] 
org.apache.hadoop.ipc.Server: IPC Server listener on 56283: starting
2014-11-28 18:56:44,031 FATAL [main] 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:693)
        at org.apache.hadoop.ipc.Server.start(Server.java:2392)
        at 
org.apache.hadoop.mapred.TaskAttemptListenerImpl.startRpcServer(TaskAttemptListenerImpl.java:137)
        at 
org.apache.hadoop.mapred.TaskAttemptListenerImpl.serviceStart(TaskAttemptListenerImpl.java:107)
        at 
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at 
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
        at 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1071)
        at 
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
        at 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1456)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
        at 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1452)
        at 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1385)
2014-11-28 18:56:44,034 INFO [main] org.apache.hadoop.util.ExitUtil: 
Exiting with status 1

Any help is very much appreciated.

Thanks & Regards
Dhamodharan Ramalingam
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain 
confidential or privileged information. If you are 
not the intended recipient, any dissemination, use, 
review, distribution, printing or copying of the 
information contained in this e-mail message 
and/or attachments to it are strictly prohibited. If 
you have received this communication in error, 
please notify us by reply e-mail or telephone and 
immediately and permanently delete the message 
and any attachments. Thank you

Re: Unable to run map reduce in HBase

Posted by Vineet Mishra <cl...@gmail.com>.
Hi,

It seems your job is not getting enough memory as required, can you try
adding these properties to your configuration and run schedule the job once
again.

mapred.cluster.map.memory.mb=2048
mapred.cluster.reduce.memory.mb=2048

Hope that solve your problem.

Cheers!

On Fri, Nov 28, 2014 at 11:33 AM, <dh...@tcs.com> wrote:

> Hi,
>
> I am importing a csv file into HBase using the command bin/hbase
> org.apache.hadoop.hbase.mapreduce.ImportTsv
>
> When I execute the this map reduce program I am getting the following
> error. I am using Hadoop 2.4.1 and HBase 0.98.8-hadoop2
>
> I have set export JAVA_OPTS="-Xms1024m -Xmx10240m" in .bashrc, the server
> has 32 GB of RAM.
>
> 2014-11-28 18:56:44,029 INFO [IPC Server listener on 56283]
> org.apache.hadoop.ipc.Server: IPC Server listener on 56283: starting
> 2014-11-28 18:56:44,031 FATAL [main]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
> java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:693)
>         at org.apache.hadoop.ipc.Server.start(Server.java:2392)
>         at
>
> org.apache.hadoop.mapred.TaskAttemptListenerImpl.startRpcServer(TaskAttemptListenerImpl.java:137)
>         at
>
> org.apache.hadoop.mapred.TaskAttemptListenerImpl.serviceStart(TaskAttemptListenerImpl.java:107)
>         at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>         at
>
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
>         at
>
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1071)
>         at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>         at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1456)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>         at
>
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1452)
>         at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1385)
> 2014-11-28 18:56:44,034 INFO [main] org.apache.hadoop.util.ExitUtil:
> Exiting with status 1
>
> Any help is very much appreciated.
>
> Thanks & Regards
> Dhamodharan Ramalingam
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>