You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2014/07/23 20:19:41 UTC
[jira] [Resolved] (MAPREDUCE-678) distcp -m option does not work
[ https://issues.apache.org/jira/browse/MAPREDUCE-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer resolved MAPREDUCE-678.
----------------------------------------
Resolution: Cannot Reproduce
> distcp -m option does not work
> -------------------------------
>
> Key: MAPREDUCE-678
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-678
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Suhas Gogate
>
> In spite of -m 10 , number of launched map tasks are 15.
> bin/hadoop distcp -Dmapred.job.queue.name=xxxx -i -p -update -m 10 -delete hftp://xxxx.yyyy.com:50070/user/xxxx/QA_incremental hdfs://aaaa.yyyy.com:8020/user/xxxxx/QA_incremental'
> 09/06/29 19:14:24 INFO tools.DistCp: srcPaths=[hftp://xxxxx.yyyy.com:50070/user/xxxx/QA_incremental]
> 09/06/29 19:14:24 INFO tools.DistCp: destPath=hdfs://zzzz.yyyy.com:8020/user/xxxx/QA_incremental
> 09/06/29 19:14:34 INFO tools.DistCp: sourcePathsCount=62
> 09/06/29 19:14:34 INFO tools.DistCp: filesToCopyCount=15
> 09/06/29 19:14:34 INFO tools.DistCp: bytesToCopyCount=18.0g
> 09/06/29 19:14:37 INFO mapred.JobClient: Running job: job_200906290541_1523
> 09/06/29 19:14:38 INFO mapred.JobClient: map 0% reduce 0%
> 09/06/29 19:15:22 INFO mapred.JobClient: map 2% reduce 0%
> 09/06/29 19:26:57 INFO mapred.JobClient: map 9% reduce 0%
> 09/06/29 19:36:02 INFO mapred.JobClient: map 16% reduce 0%
> 09/06/29 19:36:46 INFO mapred.JobClient: map 22% reduce 0%
> 09/06/29 19:37:25 INFO mapred.JobClient: map 29% reduce 0%
> 09/06/29 19:38:28 INFO mapred.JobClient: map 36% reduce 0%
> 09/06/29 19:39:41 INFO mapred.JobClient: map 42% reduce 0%
> 09/06/29 19:40:08 INFO mapred.JobClient: map 46% reduce 0%
> 09/06/29 19:44:08 INFO mapred.JobClient: map 53% reduce 0%
> 09/06/29 19:44:19 INFO mapred.JobClient: map 60% reduce 0%
> 09/06/29 19:44:39 INFO mapred.JobClient: map 66% reduce 0%
> 09/06/29 19:45:04 INFO mapred.JobClient: map 73% reduce 0%
> 09/06/29 19:46:08 INFO mapred.JobClient: map 80% reduce 0%
> 09/06/29 19:46:24 INFO mapred.JobClient: map 86% reduce 0%
> 09/06/29 19:46:38 INFO mapred.JobClient: map 93% reduce 0%
> 09/06/29 19:48:00 INFO mapred.JobClient: map 100% reduce 0%
> 09/06/29 19:48:19 INFO mapred.JobClient: Job complete: job_200906290541_1523
> 09/06/29 19:48:19 INFO mapred.JobClient: Counters: 10
> 09/06/29 19:48:19 INFO mapred.JobClient: Job Counters
> 09/06/29 19:48:19 INFO mapred.JobClient: Launched map tasks=15
> 09/06/29 19:48:19 INFO mapred.JobClient: FileSystemCounters
> 09/06/29 19:48:19 INFO mapred.JobClient: HDFS_BYTES_READ=7742
> 09/06/29 19:48:19 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=19294982191
> 09/06/29 19:48:19 INFO mapred.JobClient: distcp
> 09/06/29 19:48:19 INFO mapred.JobClient: Files copied=15
> 09/06/29 19:48:19 INFO mapred.JobClient: Bytes copied=19294982191
> 09/06/29 19:48:19 INFO mapred.JobClient: Bytes expected=19294982191
> 09/06/29 19:48:19 INFO mapred.JobClient: Map-Reduce Framework
> 09/06/29 19:48:19 INFO mapred.JobClient: Map input records=16
> 09/06/29 19:48:19 INFO mapred.JobClient: Spilled Records=0
> 09/06/29 19:48:19 INFO mapred.JobClient: Map input bytes=3190
> 09/06/29 19:48:19 INFO mapred.JobClient: Map output records=0
--
This message was sent by Atlassian JIRA
(v6.2#6252)