You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@myriad.apache.org by "Swapnil Daingade (JIRA)" <ji...@apache.org> on 2015/10/13 00:04:05 UTC
[jira] [Commented] (MYRIAD-139) CapacityScheduler doesn't work with
Fine Grained Scaling
[ https://issues.apache.org/jira/browse/MYRIAD-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14953839#comment-14953839 ]
Swapnil Daingade commented on MYRIAD-139:
-----------------------------------------
Filed yarn JIRA for this here https://issues.apache.org/jira/browse/YARN-4257.
Please do voice your opinion on that JIRA.
> CapacityScheduler doesn't work with Fine Grained Scaling
> --------------------------------------------------------
>
> Key: MYRIAD-139
> URL: https://issues.apache.org/jira/browse/MYRIAD-139
> Project: Myriad
> Issue Type: Bug
> Components: Executor, Scheduler
> Reporter: DarinJ
> Assignee: Swapnil Daingade
>
> When trying to flex up a zero profile NM you get the following error:
> {quote}
> 15/09/21 20:31:19 FATAL nodemanager.NodeManager: Error starting NodeManager
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from
> Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from doesn't satis
> fy minimum allocations, Sending SHUTDOWN signal to the NodeManager.
> at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:264)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:463)
> at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:509)
> Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager
> failed, Message from ResourceManager: NodeManager from doesn't satisfy minimum allocations, Sending SHUTDOWN sign
> al to the NodeManager.
> at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:265)
> at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:191)
> ... 6 more
> 15/09/21 20:31:19 INFO nodemanager.NodeManager: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down NodeManager at
> ************************************************************/
> {quote}
> This can be fixed by setting
> yarn.scheduler.minimum-allocation-vcores and yarn.scheduler.minimum-allocation-mb to 0 in yarn-site.xml.
> At this point com.ebay.myriad.scheduler.yarn.MyriadFairScheduler works as expected. However, when start the resource manager with com.ebay.myriad.scheduler.yarn.MyriadCapacityScheduler we get:
> {quote}
> RuntimeException: Invalid resource scheduler memory allocation configuration, yarn.scheduler.minimum-allocation-mb=0, yarn.scheduler.maximum-allocation-mb=8192, min and max should be greater than 0, max should be no smaller than min.
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Invalid resource scheduler memory allocation configuration, yarn.scheduler.minimum-allocation-mb=0, yarn.scheduler.maximum-allocation-mb=8192, min and max should be greater than 0, max should be no smaller than min.
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.validateConf(CapacityScheduler.java:163)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:285)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:324)
> at com.ebay.myriad.scheduler.yarn.MyriadCapacityScheduler.serviceInit(MyriadCapacityScheduler.java:52)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:558)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:989)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:255)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1210)
> 15/09/21 21:42:26 INFO resourcemanager.ResourceManager: Transitioning to standby state
> 15/09/21 21:42:26 INFO resourcemanager.ResourceManager: Transitioned to standby state
> 15/09/21 21:42:26 FATAL resourcemanager.ResourceManager: Error starting ResourceManager
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Invalid resource scheduler memory allocation configuration, yarn.scheduler.minimum-allocation-mb=0, yarn.scheduler.maximum-allocation-mb=8192, min and max should be greater than 0, max should be no smaller than min.
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.validateConf(CapacityScheduler.java:163)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initScheduler(CapacityScheduler.java:285)
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.serviceInit(CapacityScheduler.java:324)
> at com.ebay.myriad.scheduler.yarn.MyriadCapacityScheduler.serviceInit(MyriadCapacityScheduler.java:52)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:558)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:989)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:255)
> at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1210)
> 15/09/21 21:42:26 INFO resourcemanager.ResourceManager: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down ResourceManager at
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)