[jira] [Updated] (MAPREDUCE-7290) ShuffleHeader should be compatible between client&server when rolling update

["Wei-Chiu Chuang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wei-Chiu Chuang updated MAPREDUCE-7290:
---------------------------------------
    Target Version/s: 3.3.1  (was: 3.3.0)

> ShuffleHeader should be compatible between client&server when rolling update
> ----------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-7290
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7290
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>    Affects Versions: 2.6.0
>            Reporter: chaoli
>            Priority: Major
>             Fix For: 2.9.0, 3.3.0
>
>         Attachments: 0001-MAPREDUCE-7290-ShuffleHeader-should-be-compatible-be.patch, 0001-MAPREDUCE-7290-ShuffleHeader-should-be-compatible-be.patch
>
>
> Recently, we need to add a property in ShuffleHeader class. And there were a problem that client&server may not compatible because of serialization/deserialization when we were rolling update NodeManager instance. 
> To solve the problem, we made an improvement to help shuffle header version upgrade smoothly. 
> In HeaderVersion class, shuffle client uses *targetVersion(eg 1.2.0)* to negotiate with shuffle server. In that case, shuffle server has three cases:
>  # shuffle server does not have the newest code which is able to support version chosen, and we have to use *defaultVersion(eg 1.0.0)* instead*.*
>  # shuffle server can support older version(eg 1.1.0), then it can reply to shuffle client with *compatibleVersion(eg 1.1.0),* and finally, client and server can use the same version.
>  # shuffle server can support older version(eg 1.3.0), then it can reply to shuffle client with *compatibleVersion(eg 1.2.0),* and finally, client and server can use the same version.
> Then, client and server can serialize&deserialize property using the available version.
>  
> {code:java}
> /**
>  * unified hard code header version for new properties.
>  * @throws IOException
>  */
> private void writeByVersion(DataOutput out) throws IOException {
>   if (headerVersion == null || headerVersion.compareTo(DEFAULT_HEADER_VERSION_INSTANCE) == 0) {
>     return;
>   }
>   // if current version larger then target version,
>   // we should serialize the properties owned by the version in order.
>   if (headerVersion.compareTo(HEADER_VERSION_INSTANCE_V1_1) >= 0) {
>     // here write version properties
>   }
> }
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org