You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Chandrashekhar Kotekar <sh...@gmail.com> on 2015/05/02 11:56:56 UTC

HBase copyTable stuck at map 100% reduce 0%

Hi,

I am copying table from primary cluster to backup cluster using copyTable
command but m-r job spawned by this command is stuck at map "100% reduce
0%".

Command used is :  hbase org.apache.hadoop.hbase.mapreduce.CopyTable
-Dhbase.client.scanner.caching=100 --peer.adr=target-nn1:/hbase

Here is some part of  the logs where job is stuck since last 15 minutes.

15/05/02 09:49:17 INFO mapreduce.JobSubmitter: number of splits:1
15/05/02 09:49:17 INFO Configuration.deprecation: io.bytes.per.checksum is
deprecated. Instead, use dfs.bytes-per-checksum
15/05/02 09:49:17 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1424936551928_0181
15/05/02 09:49:18 INFO impl.YarnClientImpl: Submitted application
application_1424936551928_0181
15/05/02 09:49:18 INFO mapreduce.Job: The url to track the job:
http://resource-manager:8088/proxy/application_1424936551928_0181/
15/05/02 09:49:18 INFO mapreduce.Job: Running job: job_1424936551928_0181
15/05/02 09:49:27 INFO mapreduce.Job: Job job_1424936551928_0181 running in
uber mode : false
15/05/02 09:49:27 INFO mapreduce.Job:  map 0% reduce 0%
15/05/02 09:49:40 INFO mapreduce.Job:  map 100% reduce 0%


Does anyone knows why this m-r job gets hang at reduce 0%? I am not able to
see m-r jobs logs also because this cluster is in VPC, can't reach yarn job
web UI.

Regards,
Chandrash3khar Kotekar
Mobile - +91 8600011455

Re: HBase copyTable stuck at map 100% reduce 0%

Posted by Ted Yu <yu...@gmail.com>.
Here is usage of CopyTable:

    System.err.println(" $ bin/hbase " +
        "org.apache.hadoop.hbase.mapreduce.CopyTable
--starttime=1265875194289 --endtime=1265878794289 " +
        "--peer.adr=server1,server2,server3:2181:/hbase
--families=myOldCf:myNewCf,cf2,cf3 TestTable ");

Your peer.adr specification doesn't seem to follow the usage.

Are you able to execute yarn command against the cluster ?

Which hbase release are you using ?

Cheers

On Sat, May 2, 2015 at 2:56 AM, Chandrashekhar Kotekar <
shekhar.kotekar@gmail.com> wrote:

> Hi,
>
> I am copying table from primary cluster to backup cluster using copyTable
> command but m-r job spawned by this command is stuck at map "100% reduce
> 0%".
>
> Command used is :  hbase org.apache.hadoop.hbase.mapreduce.CopyTable
> -Dhbase.client.scanner.caching=100 --peer.adr=target-nn1:/hbase
>
> Here is some part of  the logs where job is stuck since last 15 minutes.
>
> 15/05/02 09:49:17 INFO mapreduce.JobSubmitter: number of splits:1
> 15/05/02 09:49:17 INFO Configuration.deprecation: io.bytes.per.checksum is
> deprecated. Instead, use dfs.bytes-per-checksum
> 15/05/02 09:49:17 INFO mapreduce.JobSubmitter: Submitting tokens for job:
> job_1424936551928_0181
> 15/05/02 09:49:18 INFO impl.YarnClientImpl: Submitted application
> application_1424936551928_0181
> 15/05/02 09:49:18 INFO mapreduce.Job: The url to track the job:
> http://resource-manager:8088/proxy/application_1424936551928_0181/
> 15/05/02 09:49:18 INFO mapreduce.Job: Running job: job_1424936551928_0181
> 15/05/02 09:49:27 INFO mapreduce.Job: Job job_1424936551928_0181 running in
> uber mode : false
> 15/05/02 09:49:27 INFO mapreduce.Job:  map 0% reduce 0%
> 15/05/02 09:49:40 INFO mapreduce.Job:  map 100% reduce 0%
>
>
> Does anyone knows why this m-r job gets hang at reduce 0%? I am not able to
> see m-r jobs logs also because this cluster is in VPC, can't reach yarn job
> web UI.
>
> Regards,
> Chandrash3khar Kotekar
> Mobile - +91 8600011455
>