You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Hongyuan Li (JIRA)" <ji...@apache.org> on 2019/02/21 09:13:00 UTC
[jira] [Updated] (MAPREDUCE-7186) JobSubmitter#checkSpecs doesn not
handle the situation when reduceNums is less than zero.
[ https://issues.apache.org/jira/browse/MAPREDUCE-7186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hongyuan Li updated MAPREDUCE-7186:
-----------------------------------
Description:
JobSubmitter#checkSpecs doesn not judge whether reduceNums is less than zero.
{code:java}
private void checkSpecs(Job job) throws ClassNotFoundException,
InterruptedException, IOException {
JobConf jConf = (JobConf)job.getConfiguration();
// Check the output specification
if (jConf.getNumReduceTasks() == 0 ?
jConf.getUseNewMapper() : jConf.getUseNewReducer()) {
org.apache.hadoop.mapreduce.OutputFormat<?, ?> output =
ReflectionUtils.newInstance(job.getOutputFormatClass(),
job.getConfiguration());
output.checkOutputSpecs(job);
} else {
jConf.getOutputFormat().checkOutputSpecs(jtFs, jConf);
}
}
{code}
The code does handle the situation when reduceTaskNums is set to -1 or any other value less than zero.
When use hive, the "mapred.reduce.tasks" has a default value -1.
was:
JobSubmitter#checkSpecs doesn not judge whether reduceNums is less than zero.
{code:java}
private void checkSpecs(Job job) throws ClassNotFoundException,
InterruptedException, IOException {
JobConf jConf = (JobConf)job.getConfiguration();
// Check the output specification
if (jConf.getNumReduceTasks() == 0 ?
jConf.getUseNewMapper() : jConf.getUseNewReducer()) {
org.apache.hadoop.mapreduce.OutputFormat<?, ?> output =
ReflectionUtils.newInstance(job.getOutputFormatClass(),
job.getConfiguration());
output.checkOutputSpecs(job);
} else {
jConf.getOutputFormat().checkOutputSpecs(jtFs, jConf);
}
}
{code}
The code does handle the situation when reduceTaskNums is set to -1 or any other value less than zero.
> JobSubmitter#checkSpecs doesn not handle the situation when reduceNums is less than zero.
> -----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7186
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7186
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
> Reporter: Hongyuan Li
> Priority: Minor
>
> JobSubmitter#checkSpecs doesn not judge whether reduceNums is less than zero.
> {code:java}
> private void checkSpecs(Job job) throws ClassNotFoundException,
> InterruptedException, IOException {
> JobConf jConf = (JobConf)job.getConfiguration();
> // Check the output specification
> if (jConf.getNumReduceTasks() == 0 ?
> jConf.getUseNewMapper() : jConf.getUseNewReducer()) {
> org.apache.hadoop.mapreduce.OutputFormat<?, ?> output =
> ReflectionUtils.newInstance(job.getOutputFormatClass(),
> job.getConfiguration());
> output.checkOutputSpecs(job);
> } else {
> jConf.getOutputFormat().checkOutputSpecs(jtFs, jConf);
> }
> }
> {code}
> The code does handle the situation when reduceTaskNums is set to -1 or any other value less than zero.
> When use hive, the "mapred.reduce.tasks" has a default value -1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org