You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Junping Du (JIRA)" <ji...@apache.org> on 2014/11/25 16:11:12 UTC
[jira] [Commented] (MAPREDUCE-6168) Old MR client is still broken
when receiving new counters from MR job
[ https://issues.apache.org/jira/browse/MAPREDUCE-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224654#comment-14224654 ]
Junping Du commented on MAPREDUCE-6168:
---------------------------------------
Hi [~zjshen], Thanks for reporting this.
>From the track stack below:
{noformat}
org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
at org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
{noformat}
It show that the client is still in 2.2 (https://github.com/apache/hadoop/blob/branch-2.2/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/FrameworkCounterGroup.java), and we fix the problem in 2.6 which means the client after 2.6 can be compatible with new counters in future (forward compatibility) but not means existing broken compatibility can be recovered (except we want to fix the code in branch-2.2).
[~zjshen], I would like to fix this JIRA as "won't fix". Do you agree?
> Old MR client is still broken when receiving new counters from MR job
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-6168
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6168
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Zhijie Shen
> Assignee: Junping Du
> Priority: Blocker
>
> In the following scenarios:
> 1. Either insecure or secure;
> 2. MR 2.2 with new shuffle on NM;
> 3. Submitting via old client.
> We will see the following console exception:
> {code}
> 14/11/17 14:56:19 INFO mapreduce.Job: Job job_1416264695865_0003 completed successfully
> java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.mapreduce.JobCounter.MB_MILLIS_REDUCES
> at java.lang.Enum.valueOf(Enum.java:236)
> at org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.valueOf(FrameworkCounterGroup.java:148)
> at org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.findCounter(FrameworkCounterGroup.java:182)
> at org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
> at org.apache.hadoop.mapreduce.TypeConverter.fromYarn(TypeConverter.java:240)
> at org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:370)
> at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:511)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:756)
> at org.apache.hadoop.mapreduce.Job$7.run(Job.java:753)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:753)
> at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1361)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1289)
> at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:306)
> at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> {code}
> The problem is supposed to be fixed by MAPREDUCE-5831, however, it seems that we haven't cover all the problematic code path.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)