[jira] [Created] (MYRIAD-251) ZERO size NodeManager fail to obtain resource from Mesos Offer

["Tao Jie (JIRA)" <ji...@apache.org>]

Tao Jie created MYRIAD-251:
------------------------------

             Summary: ZERO size NodeManager fail to obtain resource from Mesos Offer
                 Key: MYRIAD-251
                 URL: https://issues.apache.org/jira/browse/MYRIAD-251
             Project: Myriad
          Issue Type: Bug
          Components: Scheduler
            Reporter: Tao Jie


I tried Fine-grained Scaling and flexed up zero size NodeManager, then I run a MR job which request for resource.
However zero size NM did not obtain resource from mesos offer. RM logs like:
{code}
2016-12-14 16:58:23,929 INFO org.apache.myriad.scheduler.fgs.NMHeartBeatHandler: Did not update bdi13.cmss.com with 10 cores and 5888 memory, over max cpu cores and/or max memory
2016-12-14 16:58:23,931 WARN org.apache.myriad.scheduler.fgs.YarnNodeCapacityManager: Asked to set Node bdi13.cmss.com:31905 to a value less than zero!  Had <memory:0, vCores:0>, setting to <memory:0, vCores:0>.
2016-12-14 16:58:23,931 WARN org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler: Update resource on node: bdi13.cmss.com with the same resource: <memory:0, vCores:0>
{code}
It seems that mesos offer with memory larger than 2252.8mb would be denied, and 2252.8mb is fixed value in code :
{code}
private Double generateNodeManagerMemory() {
    return (NodeManagerConfiguration.DEFAULT_JVM_MAX_MEMORY_MB) * (1 + NodeManagerConfiguration.JVM_OVERHEAD);
  }
{code}
where DEFAULT_JVM_MAX_MEMORY_MB=2048 and NodeManagerConfiguration.JVM_OVERHEAD=0.1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)