[jira] [Created] (REEF-1653) Request for evaluator with multiple cores always return evaluator with 1 core only in IMRU

["Dhruv Mahajan (JIRA)" <ji...@apache.org>]

Dhruv Mahajan created REEF-1653:
-----------------------------------

             Summary: Request for evaluator with multiple cores always return evaluator with 1 core only in IMRU
                 Key: REEF-1653
                 URL: https://issues.apache.org/jira/browse/REEF-1653
             Project: REEF
          Issue Type: Bug
          Components: REEF.NET
         Environment: C#
            Reporter: Dhruv Mahajan


Currently, on submitting request for evaluators with multiple cores in IMRU to YARN cluster, we always get back evaluators with 1 core only. Memory allocated is right and higher rounded off value than requested which is the desired behavior. However, number of cores has issues. Find below relevant logs.

I asked for evaluator with 7000MB memory and 7 cores. In return I got one with around 9000MB memory and 1 core only which does not seem good right? Find below relevant logs from driver:

Org.Apache.REEF.IMRU.OnREEF.Driver.IMRUDriver`4[[TLCPP.LBFGS.MapFunctionInputOutput.MapFunctionInput, TLCPP, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[TLCPP.LBFGS.MapFunctionInputOutput.MapFunctionOutput, TLCPP, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null],[Microsoft.MachineLearning.Data.VBuffer`1[[System.Single, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Microsoft.MachineLearning.Core, Version=3.4.15.0, Culture=neutral, PublicKeyToken=d353f9ba84f0e281],[Microsoft.MachineLearning.Data.RoleMappedData, Microsoft.MachineLearning.Core, Version=3.4.15.0, Culture=neutral, PublicKeyToken=d353f9ba84f0e281]] Information: 0 : 2016-10-20T20:32:04.0144450+00:00 0001

{color:red}
INFO: map task memory:7000, update task memory:7000, map task cores:7, update task cores:7, maxRetry 10, allowedFailedEvaluators 4.
{color}

INFO: *** Start time is 10/20/2016 8:32:04 PM
Org.Apache.REEF.Driver.Bridge.Events.EvaluatorRequestor Information: 0 : 2016-10-20T20:32:04.0925881+00:00 0001
{color:red}
INFO: Submitting request for 1 evaluators and 7000 MB memory and  7 core to rack  and runtime .
{color}
INFO: Allocated Evaluator: container_1475922122639_0016_01_000002, total running running 0
Oct 20, 2016 8:32:06 PM org.apache.reef.javabridge.AllocatedEvaluatorBridge getEvaluatorDescriptorString

{color:red}
INFO: allocated evaluator - serialized evaluator descriptor: IP=xyz, Port=45454, HostName=xyz, Memory=9216, Core=1, RuntimeName=Yarn
{color}
Oct 20, 2016 8:32:08 PM org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl populateNMTokens




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)