You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2012/09/06 21:39:07 UTC
[jira] [Created] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Sandy Ryza created HADOOP-8775:
----------------------------------
Summary: MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
Key: HADOOP-8775
URL: https://issues.apache.org/jira/browse/HADOOP-8775
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Sandy Ryza
The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HADOOP-8775:
-----------------------------------
Assignee: Sandy Ryza
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: diff2.txt
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HADOOP-8775) MR2 distcp permits non-positive value
to -bandwidth option which causes job never to complete
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed HADOOP-8775.
---------------------------------
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.2-alpha, 0.23.5
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Sandy Ryza (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandy Ryza updated HADOOP-8775:
-------------------------------
Fix Version/s: 2.2.0-alpha
Status: Patch Available (was: Open)
I made it throw an IllegalArgumentException, as it would do if the bandwidth entered were not an integer.
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Fix For: 2.2.0-alpha
>
> Attachments: diff2.txt
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HADOOP-8775:
-----------------------------------
Fix Version/s: (was: 2.0.1-alpha)
2.2.0-alpha
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.2.0-alpha
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451338#comment-13451338 ]
Hudson commented on HADOOP-8775:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1190 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1190/])
HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza. (Revision 1382119)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382119
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.2-alpha
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HADOOP-8775:
-----------------------------------
Fix Version/s: (was: 2.2.0-alpha)
The patch looks pretty good to me, but it should be pretty easy to add a little test case for this. See TestOptionsParser for some examples. There's even one in there which tests the bandwidth option.
Also, please don't set the "fix versions" field until the JIRA is actually committed. Use the "target versions" field to express what versions you'd like to see this JIRA committed to.
Also, per the [how to contribute wiki|http://wiki.apache.org/hadoop/HowToContribute], please name patch files using the form "HADOOP-8775.patch" instead of "diff2.txt".
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Attachments: diff2.txt
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450888#comment-13450888 ]
Hudson commented on HADOOP-8775:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2771 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2771/])
HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza. (Revision 1382119)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382119
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.1-alpha
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450275#comment-13450275 ]
Hadoop QA commented on HADOOP-8775:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12544146/HADOOP-8775.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-tools/hadoop-distcp.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1421//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1421//console
This message is automatically generated.
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450012#comment-13450012 ]
Hadoop QA commented on HADOOP-8775:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12544100/diff2.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-tools/hadoop-distcp.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1417//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1417//console
This message is automatically generated.
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Fix For: 2.2.0-alpha
>
> Attachments: diff2.txt
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Sandy Ryza (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandy Ryza updated HADOOP-8775:
-------------------------------
Attachment: HADOOP-8775.patch
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450238#comment-13450238 ]
Aaron T. Myers commented on HADOOP-8775:
----------------------------------------
Thanks a lot for addressing my comments, Sandy. The latest patch looks good to me. +1 pending a clean Jenkins report.
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450909#comment-13450909 ]
Hudson commented on HADOOP-8775:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2732 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2732/])
HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza. (Revision 1382119)
Result = FAILURE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382119
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.2.0-alpha
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Sandy Ryza (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandy Ryza updated HADOOP-8775:
-------------------------------
Attachment: diff2.txt
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Attachments: diff2.txt
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers updated HADOOP-8775:
-----------------------------------
Resolution: Fixed
Fix Version/s: 2.0.1-alpha
Status: Resolved (was: Patch Available)
I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Sandy.
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.1-alpha
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451331#comment-13451331 ]
Hudson commented on HADOOP-8775:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #1159 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1159/])
HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza. (Revision 1382119)
Result = FAILURE
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382119
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.2-alpha
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-8775:
----------------------------------------
Fix Version/s: 0.23.5
I pulled this into branch-0.23
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.2-alpha, 0.23.5
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450886#comment-13450886 ]
Hudson commented on HADOOP-8775:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #2708 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2708/])
HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza. (Revision 1382119)
Result = SUCCESS
atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382119
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
* /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.1-alpha
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467665#comment-13467665 ]
Hudson commented on HADOOP-8775:
--------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #392 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/392/])
svn merge -c 1382119 FIXES: HADOOP-8775. MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete. Contributed by Sandy Ryza. (Revision 1392519)
Result = UNSTABLE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392519
Files :
* /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
* /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Fix For: 2.0.2-alpha, 0.23.5
>
> Attachments: diff2.txt, HADOOP-8775.patch
>
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8775) MR2 distcp permits non-positive
value to -bandwidth option which causes job never to complete
Posted by "Sandy Ryza (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sandy Ryza updated HADOOP-8775:
-------------------------------
Description:
The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
{noformat}
bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
{noformat}
was:
The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-8775
> URL: https://issues.apache.org/jira/browse/HADOOP-8775
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha
> Reporter: Sandy Ryza
>
> The likelihood that someone would want to enter a non-positive value for -bandwidth seems really low. However, the job would never complete if a non-positive value was specified. It'd just get stuck at map 100%. Luckily, a positive value would always lead to the job completing.
> {noformat}
> bash-4.1$ hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
> 12/05/23 15:53:01 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsiz\
> e', sourceFileListing=null, sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir], targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead, use mapreduce.task.io.sort.mb
> 12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated. Instead, use mapreduce.task.io.sort.factor
> 12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
> 12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
> 12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
> 12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application application_1337808305464_0014 to ResourceManager at c1204.hal.cloudera.com/172.29.98.195:8040
> 12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job: http://auto0:8088/proxy/application_1337808305464_0014/
> 12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
> 12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
> 12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in uber mode : false
> 12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
> 12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
> 12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira