[jira] [Created] (MAPREDUCE-7068) Fix Reduce Exception was overwrited by ReduceTask

["tartarus (JIRA)" <ji...@apache.org>]

tartarus created MAPREDUCE-7068:
-----------------------------------

             Summary: Fix Reduce Exception was overwrited by ReduceTask
                 Key: MAPREDUCE-7068
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7068
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv1
    Affects Versions: 2.7.1
         Environment: CentOS 7

Hadoop-2.7.1

Hive-1.2.1
            Reporter: tartarus


 
{code:java}
    try {
      //increment processed counter only if skipping feature is enabled
      boolean incrProcCount = SkipBadRecords.getReducerMaxSkipGroups(job)>0 &&
        SkipBadRecords.getAutoIncrReducerProcCount(job);
      
      ReduceValuesIterator<INKEY,INVALUE> values = isSkipping() ? 
          new SkippingReduceValuesIterator<INKEY,INVALUE>(rIter, 
              comparator, keyClass, valueClass, 
              job, reporter, umbilical) :
          new ReduceValuesIterator<INKEY,INVALUE>(rIter, 
          comparator, keyClass, valueClass,
          job, reporter);
      values.informReduceProgress();
      while (values.more()) {
        reduceInputKeyCounter.increment(1);
        reducer.reduce(values.getKey(), values, collector, reporter);
        if(incrProcCount) {
          reporter.incrCounter(SkipBadRecords.COUNTER_GROUP, 
              SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS, 1);
        }
        values.nextKey();
        values.informReduceProgress();
      }


      reducer.close();
      reducer = null;
      
      out.close(reporter);
      out = null;
    } finally {
      IOUtils.cleanupWithLogger(LOG, reducer);
      closeQuietly(out, reporter);
    }
  }
{code}
if  {color:#d04437}reducer.close();{color} throw Exception , {color:#d04437}reducer = null;{color} will not run, then {color:#d04437}IOUtils.cleanupWithLogger(LOG, reducer); {color}

 

will throw Exception and overwrite the Exception of reducer.close();

so we should catch it and print log to help targeting issues

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org