You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "Benjamin Mahler (JIRA)" <ji...@apache.org> on 2019/02/27 19:32:00 UTC
[jira] [Commented] (MESOS-7883) Quota heuristic check not
accounting for mount volumes
[ https://issues.apache.org/jira/browse/MESOS-7883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779689#comment-16779689 ]
Benjamin Mahler commented on MESOS-7883:
----------------------------------------
Linking in quota "capacity heuristic" testing work.
> Quota heuristic check not accounting for mount volumes
> ------------------------------------------------------
>
> Key: MESOS-7883
> URL: https://issues.apache.org/jira/browse/MESOS-7883
> Project: Mesos
> Issue Type: Bug
> Components: allocation
> Reporter: Vincent Roy
> Priority: Major
> Labels: resource-management
>
> This may be expected but came as a surprise to us. We are unable to create a quota bigger than the root disk space on slaves.
> Given two clusters with the same number of slaves and root disk size, but one that also has mount volumes, is what the disk resources look like:
> {noformat}
> [root@fin-fang-foom-master-1 ~]# curl -s master.mesos:5050/state | jq '.slaves[] .resources .disk'
> 28698
> 28699
> 28698
> 28698
> 28697
> {noformat}
> {noformat}
> [root@hydra-master-1 ~]# curl -s master.mesos:5050/state | jq '.slaves[] .resources .disk'
> 50817
> 50817
> 50814
> 50819
> 50817
> {noformat}
> In {{fin-fang-foom}}, I was able to create a quota for {{143490mb}} which is the total of available disk resources, root in this case, as reported by Mesos. For {{hydra}}, I am only able to create a quota for {{143489mb}}. This is equivalent to the total of root disks available in {{hydra}} rather than the total available disks reported by Mesos resources which is {{254084mb}}.
> With a modified Mesos that adds logging to {{quota_handler}}, we can see that only the {{disk(*)}} number increases in {{nonStaticClusterResources}} after every iteration. The final iteration is {{disk(*):143489}} which is the maximum quota I was able to create on {{hydra}}. We expected that quota heuristic check would also include resources such as {{disk(*)[MOUNT:/dcos/volume2]:7373}}
> {noformat}
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763764 24902 quota_handler.cpp:71] Performing capacity heuristic check for a set quota request
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763783 24902 quota_handler.cpp:87] heuristic: total quota 'disk(*):143489'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763870 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763923 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.763989 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764022 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):57396; cpus(*):8; mem(*):30046; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764077 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):28695; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764119 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):86091; cpus(*):12; mem(*):45069; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764225 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):28700; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764307 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):114791; cpus(*):16; mem(*):60092; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764434 24902 quota_handler.cpp:111] heuristic: nonStaticAgentResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):28698; cpus(*):4; mem(*):15023'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764492 24902 quota_handler.cpp:113] heuristic: nonStaticClusterResources = 'ports(*):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-8180, 8182-32000]; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*):143489; cpus(*):20; mem(*):75115; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373; disk(*)[MOUNT:/dcos/volume0]:7373; disk(*)[MOUNT:/dcos/volume1]:7373; disk(*)[MOUNT:/dcos/volume2]:7373'
> Aug 11 12:54:18 hydra-master-1 mesos-master[24896]: I0811 12:54:18.764562 24902 quota_handler.cpp:118] heuristic: nonStaticClusterResources.contains(totalQuota)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)