[jira] [Created] (MAPREDUCE-6387) Fix Task.java to serialize encryptedSpillKey at the end to to allow rolling upgrades without distributed cache

["Arun Suresh (JIRA)" <ji...@apache.org>]

Arun Suresh created MAPREDUCE-6387:
--------------------------------------

             Summary: Fix Task.java to serialize encryptedSpillKey at the end to to allow rolling upgrades without distributed cache
                 Key: MAPREDUCE-6387
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6387
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Arun Suresh
            Assignee: Arun Suresh
            Priority: Minor


There was a recent addition of an {{encryptedSpillKey}} to the Task object. And when serialized, this field was written out somewhere in the middle. This caused deployments that do not use DistributedCache to push job jars before running the job to fail rolling upgrade.

Although deploying via Distributed Cache is the recommended method, there might still be deployments that use the node local classpath to pick up the MR framework classes (eg. for efficiency purposes, since this does not require the jar being copied to hdfs and then to all the nodes)

Ensuring that it is the last field written and read when the Task object is serialized would alleviate this issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)