You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by "Yingda Chen (JIRA)" <ji...@apache.org> on 2014/11/03 19:22:34 UTC
[jira] [Commented] (REEF-6) Improve allocation performance
[ https://issues.apache.org/jira/browse/REEF-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194830#comment-14194830 ]
Yingda Chen commented on REEF-6:
--------------------------------
The new PR looks good. I have tested various scenarios, and most of them are working just fine.
There is one little glitch though: in one case we get more containers allocated than requested, although we correctly ignore the unexpected container (log below). [~johnyangk], could you please confirm if this is the expected behavior? It happens when [2 x 1.5G and 3G] containers are requested.
Other than that, there are some cosmetic changes I commented in PR, but I think this is ready for merge once [~johnyangk] can confirm the above question.
===========
2014-11-03 10:08:30,079 FINE reef.runtime.yarn.driver.YarnContainerManager.onContainersAllocated AMRM Callback Handler Thread | TIME: Allocated Containers AMRM_Callback_Handler_Thread:1415038110079 1 of 1
2014-11-03 10:08:30,080 FINE reef.runtime.yarn.driver.YarnContainerManager.handleNewContainer AMRM Callback Handler Thread | allocated container: id[ container_1415033648762_0010_01_000004 ]
2014-11-03 10:08:30,080 WARNING reef.runtime.yarn.driver.YarnContainerManager.handleNewContainer AMRM Callback Handler Thread | Got an extra container container_1415033648762_0010_01_000004 that doesnt match, releasing...
2014-11-03 10:08:30,080 FINE reef.runtime.yarn.driver.YarnContainerManager.onContainersAllocated AMRM Callback Handler Thread | TIME: Processed Containers AMRM_Callback_Handler_Thread:1415038110079
> Improve allocation performance
> ------------------------------
>
> Key: REEF-6
> URL: https://issues.apache.org/jira/browse/REEF-6
> Project: REEF
> Issue Type: Improvement
> Components: REEF-Runtime-YARN
> Reporter: Markus Weimer
> Assignee: Yingda Chen
> Priority: Critical
> Attachments: driver.stderr, empty-rack-driver.stderr
>
>
> REEF today uses a very conservative protocol when interacting with YARN, which only allows for one allocation request to be in flight at a time.
> We can improve upon this by allowing one kind of allocation request to be in flight at a time without giving up the safety of the current approach.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)