[jira] [Updated] (MAPREDUCE-4671) AM does not tell the RM about container requests that are no longer needed

["Siddharth Seth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siddharth Seth updated MAPREDUCE-4671:
--------------------------------------

     Target Version/s: 2.0.3-alpha
    Affects Version/s: 0.23.3
                       2.0.0-alpha
               Status: Open  (was: Patch Available)

bq. // ??? shoudl this assert that the new value >= 0 ???
That does need to be checked before the decrement. There's a comment further up in the code which mentions a decrement coming in after this value has gone to 0. See {{containerFailedOnHost}}.

Looking at this and YARN-103 again, we're not really gaining much by keeping entries in the main request table - (remoteRequestsTable). To avoid duplicate requests, does it make sense to change ResourceRequest to ignore containerCount when computing {{equals}} and {{hashCode}}, or defining a local structure with the required implementation.
                
> AM does not tell the RM about container requests that are no longer needed
> --------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4671
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4671
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha, 0.23.3
>            Reporter: Bikas Saha
>            Assignee: Bikas Saha
>         Attachments: MAPREDUCE-4671.1.patch, MAPREDUCE-4671.2.patch
>
>
> Say the AM wanted a container at hosts h1, h2, h3. After getting a container at h1 it should tell RM that it no longer needs containers at h2, h3. Otherwise on the RM h2, h3 remain valid allocation locations.
> The AM RMContainerAllocator does remove these resource requests internally. When the resource request container count drops to 0 then it drops the resource request from its tables but forgets to send the 0 sized request to the RM.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira