[jira] [Commented] (REEF-877) Consider rounding issues in ProcessNewEvaluator() in AllocatedEvaluator.cs

["Markus Weimer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/REEF-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977756#comment-14977756 ] 

Markus Weimer commented on REEF-877:
------------------------------------

I don't think we can ever make {{EvaluatorBatchID}} safe for use. Shall we just deprecate it?

> Consider rounding issues in ProcessNewEvaluator() in AllocatedEvaluator.cs
> --------------------------------------------------------------------------
>
>                 Key: REEF-877
>                 URL: https://issues.apache.org/jira/browse/REEF-877
>             Project: REEF
>          Issue Type: Bug
>          Components: REEF.NET Driver
>            Reporter: Andrew Chung
>
> In the method, we specifically check that the memory of the request matches the allocated evaluator before removing it from EvaluatorRequestor.Evaluators. That means that if it never matches, the Evaluator stays there forever and an the AllocatedEvaluatorHandler is never called with an AllocatedEvaluator that matches the EvaluatorBatchID. This is dangerous because resource managers such as YARN, for example, can round the memory up for the AllocatedEvaluator. It is thus likely that if the user doesn't know what memory multiple to round it to, the user will never get an AllocatedEvaluator matching the EvaluatorBatchID, and EvaluatorRequestor. Evaluators can fill up with pending requests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)