You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "rangjiaheng (JIRA)" <ji...@apache.org> on 2017/10/10 02:33:00 UTC
[jira] [Issue Comment Deleted] (MAPREDUCE-6922) MapReduce jobs may
fail during rolling upgrade due to MAPREDUCE-6829
[ https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
rangjiaheng updated MAPREDUCE-6922:
-----------------------------------
Comment: was deleted
(was: We solve this problem simply by adding an judgment after read ordinal, but if someone add an Enumeration value in the middle position of TaskCounter.class, the AM will read confused Enumeration values from RPC.
I support counters should serialized through protocol buffer.
{code:java}
public void readFields(DataInput in) throws IOException {
clear();
int len = WritableUtils.readVInt(in);
T[] enums = enumClass.getEnumConstants();
for (int i = 0; i < len; ++i) {
int ord = WritableUtils.readVInt(in);
if(ord >= enums.length) {
WritableUtils.readVLong(in);
continue;
}
Counter counter = newCounter(enums[ord]);
counter.setValue(WritableUtils.readVLong(in));
counters[ord] = counter;
}
}
{code}
)
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
> --------------------------------------------------------------------
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Miklos Szegedi
> Assignee: Miklos Szegedi
> Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187] org.apache.hadoop.ipc.Server: Unable to read call parameters for client 10.17.242.22on connection protocol org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org