[jira] [Updated] (MAPREDUCE-3730) Allow restarted NM to rejoin cluster before RM expires it

["Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vinod Kumar Vavilapalli updated MAPREDUCE-3730:
-----------------------------------------------

       Fix Version/s: 0.23.2
    Target Version/s: 0.23.1, 0.24.0  (was: 0.24.0, 0.23.1)
              Status: Open  (was: Patch Available)

Running through the review queue, arrived at this finally.

First thing, the patch has gone stale again, sorry for the review delay.

Second, I still maintain that you should send the RECONNECTED event to a single (old) RMNode. All the conditional logic depending on whether the old node was healthy or not, is simply out of place.

Even the race condition that you outlined above clearly hints to me that we should have a single node. If we have a single node, we will be able to handle the RECONNECTED event at a LOST state correctly by letting the new node in.

If the RECONNECT event goes to the same node, sure, you have to handle the event in all states, but it makes it that much easier to reason there instead of an unrelated place like ResourceTrackerService.
                
> Allow restarted NM to rejoin cluster before RM expires it
> ---------------------------------------------------------
>
>                 Key: MAPREDUCE-3730
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3730
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2, resourcemanager
>    Affects Versions: 0.23.1, 0.24.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>             Fix For: 0.23.2
>
>         Attachments: MAPREDUCE-3730.patch, MAPREDUCE-3730.patch
>
>
> When a node in the RUNNING state (healthy or unhealthy) is rebooted, the resourcemanager rejects the nodemanager's registration request as a duplicate because it is convinced that the nodemanager is already running on that node.  It won't allow that node to rejoin the cluster until the node expiration time elapses which is 10min+ by default.  We should allow the NM to rejoin the cluster if it re-registers within the expiration timeout.
> Note that this problem occurs with NMs that are configured to specific ports.  If ephemeral ports are used then a NM reboot "works" because the RM thinks the NM registration is for a new node.  See the discussions in MAPREDUCE-3070 and MAPREDUCE-3363.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira