You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Alexander Rukletsov <ru...@gmail.com> on 2016/01/20 18:01:49 UTC
Review Request 42559: Allocated resources as revocable beyond quota.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/
-----------------------------------------------------------
Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
Bugs: MESOS-4441
https://issues.apache.org/jira/browse/MESOS-4441
Repository: mesos
Description
-------
See summary.
Diffs
-----
docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
Diff: https://reviews.apache.org/r/42559/diff/
Testing
-------
On Mac OS 10.10.4:
`GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
`make check`
Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
Thanks,
Alexander Rukletsov
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Guangya Liu <gy...@gmail.com>.
> On 一月 21, 2016, 1:48 a.m., Guangya Liu wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1310
> > <https://reviews.apache.org/r/42559/diff/1/?file=1203488#file1203488line1310>
> >
> > The `resource` will include some `reserved` resources, here even if you call `resource.mutable_revocable();`, the `reserved` resources will still be considered as `reserved`.
> >
> > Take a look at isReserved()
> >
> > bool Resources::isReserved(
> > const Resource& resource,
> > const Option<string>& role)
> > {
> > if (role.isSome()) {
> > return !isUnreserved(resource) && role.get() == resource.role();
> > } else {
> > return !isUnreserved(resource);
> > }
> > }
> >
> >
> > bool Resources::isUnreserved(const Resource& resource)
> > {
> > return resource.role() == "*" && !resource.has_reservation();
> > }
> >
> > For a resources such as `cpus(r1):100`, after call `resource.mutable_revocable();`, it will be `cpus(r1){REV}:100`, and the helper function `isReserved(resources, role)` still treate this as `reserved` resources but not `revocable` resources.
> >
> > This has highly dependency and interaction with MESOS-1607,the optimisitic offer phase 1, in phase 1, we are planning to treate the `reserved but not used` as `allocation slack`, it is not right if you simply translating all `resources` to `revocable`, you may want at least filter out the `reserved` resources when translating to `revocalbe` resources.
> >
> > I would suggest you also invite bmahler as review as he may be the shephard for MESOS-1607
Correct one typo: `cpus(r1){REV}:100` will be treated as both `reserved` and `revocable` resources.
diff --git a/src/tests/resources_tests.cpp b/src/tests/resources_tests.cpp
index b42610f..517c70d 100644
--- a/src/tests/resources_tests.cpp
+++ b/src/tests/resources_tests.cpp
@@ -1968,6 +1968,10 @@ TEST(RevocableResourceTest, Filter)
EXPECT_EQ(r2, r2.nonRevocable());
EXPECT_TRUE(r2.revocable().empty());
Resources r3 = createRevocableResource("cpus", "1", "r1", true); <<<<
EXPECT_EQ(r3, r3.revocable()); <<<<
EXPECT_EQ(r3, r3.reserved("r1")); <<<<
EXPECT_EQ(r1, (r1 + r2).revocable());
EXPECT_EQ(r2, (r1 + r2).nonRevocable());
}
- Guangya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115537
-----------------------------------------------------------
On 一月 20, 2016, 5:01 p.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated 一月 20, 2016, 5:01 p.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115537
-----------------------------------------------------------
src/master/allocator/mesos/hierarchical.cpp (line 1310)
<https://reviews.apache.org/r/42559/#comment176515>
The `resource` will include some `reserved` resources, here even if you call `resource.mutable_revocable();`, the `reserved` resources will still be considered as `reserved`.
Take a look at isReserved()
bool Resources::isReserved(
const Resource& resource,
const Option<string>& role)
{
if (role.isSome()) {
return !isUnreserved(resource) && role.get() == resource.role();
} else {
return !isUnreserved(resource);
}
}
bool Resources::isUnreserved(const Resource& resource)
{
return resource.role() == "*" && !resource.has_reservation();
}
For a resources such as `cpus(r1):100`, after call `resource.mutable_revocable();`, it will be `cpus(r1){REV}:100`, and the helper function `isReserved(resources, role)` still treate this as `reserved` resources but not `revocable` resources.
This has highly dependency and interaction with MESOS-1607,the optimisitic offer phase 1, in phase 1, we are planning to treate the `reserved but not used` as `allocation slack`, it is not right if you simply translating all `resources` to `revocable`, you may want at least filter out the `reserved` resources when translating to `revocalbe` resources.
I would suggest you also invite bmahler as review as he may be the shephard for MESOS-1607
- Guangya Liu
On 一月 20, 2016, 5:01 p.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated 一月 20, 2016, 5:01 p.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Klaus Ma <kl...@gmail.com>.
> On Jan. 21, 2016, 1:20 a.m., Klaus Ma wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1313
> > <https://reviews.apache.org/r/42559/diff/1/?file=1203488#file1203488line1313>
> >
> > Who will kill this `revocable` resources? So we'll have a patch for evicting?
>
> Alexander Rukletsov wrote:
> Someday, someway... : )
>
> Actually, I thought you guys are working on this (oversubscription for reservations, incl. quota)!
>
> Klaus Ma wrote:
> In "oversubscription for reservations", only `reserved & !persistentVolume` resources are considered to be revocable to other framework; the un-reserved resources are not handled for now.
>
> To me, the key of eviction is which resources shoulbe be evicted for its owner: 1.) in "oversubscription for reservations", it's `reserved & !persistentVolume`, 2.) in "oversubscription" (Estimator & QoSController), it dependent on modules;
>
> When I polishing the design doc of Phase 2 (Simultaneous offer), I'm thinking to do fair-share un-reserved resources by Quota/Role's Weight; for example, 100 CPUS in cluster and two Quota: 30 CPUS (role1) & 20 CPUS (role2); when only framework1 in role1 registered, it will get 30 CPUS (reserved) + 30 CPUS (unreserved) + 20 CPUS (revocable), this offer is Quota + (total - Quota1 - Quota2) * (Quota1/(Quota1 + Quota2)) + (total - Quota1 - Quota2) * (Quota2/(Quota1 + Quota2)); when framework2 in role2 registered, the revocable 20CPUS of framework1 will be evicted by master/allocator.
>
> I logged MESOS-4303 to trace this idea; but did not get enough time on the detail for now :). Maybe we can draft a doc on this, any comments?
typo: the offer should be "30 CPUS (quota) + 30 CPUS (unreserved) + 20 CPUS (revocable)".
- Klaus
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115443
-----------------------------------------------------------
On Jan. 21, 2016, 1:01 a.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2016, 1:01 a.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Alexander Rukletsov <ru...@gmail.com>.
> On Jan. 20, 2016, 5:20 p.m., Klaus Ma wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1313
> > <https://reviews.apache.org/r/42559/diff/1/?file=1203488#file1203488line1313>
> >
> > Who will kill this `revocable` resources? So we'll have a patch for evicting?
Someday, someway... : )
Actually, I thought you guys are working on this (oversubscription for reservations, incl. quota)!
- Alexander
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115443
-----------------------------------------------------------
On Jan. 20, 2016, 5:01 p.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated Jan. 20, 2016, 5:01 p.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Klaus Ma <kl...@gmail.com>.
> On Jan. 21, 2016, 1:20 a.m., Klaus Ma wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1313
> > <https://reviews.apache.org/r/42559/diff/1/?file=1203488#file1203488line1313>
> >
> > Who will kill this `revocable` resources? So we'll have a patch for evicting?
>
> Alexander Rukletsov wrote:
> Someday, someway... : )
>
> Actually, I thought you guys are working on this (oversubscription for reservations, incl. quota)!
In "oversubscription for reservations", only `reserved & !persistentVolume` resources are considered to be revocable to other framework; the un-reserved resources are not handled for now.
To me, the key of eviction is which resources shoulbe be evicted for its owner: 1.) in "oversubscription for reservations", it's `reserved & !persistentVolume`, 2.) in "oversubscription" (Estimator & QoSController), it dependent on modules;
When I polishing the design doc of Phase 2 (Simultaneous offer), I'm thinking to do fair-share un-reserved resources by Quota/Role's Weight; for example, 100 CPUS in cluster and two Quota: 30 CPUS (role1) & 20 CPUS (role2); when only framework1 in role1 registered, it will get 30 CPUS (reserved) + 30 CPUS (unreserved) + 20 CPUS (revocable), this offer is Quota + (total - Quota1 - Quota2) * (Quota1/(Quota1 + Quota2)) + (total - Quota1 - Quota2) * (Quota2/(Quota1 + Quota2)); when framework2 in role2 registered, the revocable 20CPUS of framework1 will be evicted by master/allocator.
I logged MESOS-4303 to trace this idea; but did not get enough time on the detail for now :). Maybe we can draft a doc on this, any comments?
- Klaus
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115443
-----------------------------------------------------------
On Jan. 21, 2016, 1:01 a.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2016, 1:01 a.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Klaus Ma <kl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115443
-----------------------------------------------------------
src/master/allocator/mesos/hierarchical.cpp (line 1313)
<https://reviews.apache.org/r/42559/#comment176433>
Who will kill this `revocable` resources? So we'll have a patch for evicting?
- Klaus Ma
On Jan. 21, 2016, 1:01 a.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2016, 1:01 a.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Qian Zhang <zh...@cn.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115605
-----------------------------------------------------------
src/master/allocator/mesos/hierarchical.cpp (lines 1342 - 1344)
<https://reviews.apache.org/r/42559/#comment176596>
So here we offer `offerableResources` to framework and mark `resources` as allocated resource of the slave. But for a framework in a quota'ed role, `offerableResources` and `resources` may be different, the former is revocable and the later is non-revocable, so later when `recoverResources()` is called (e.g., the offer is declined by framework), we will subtract offered resources from allocated resources (`slaves[slaveId].allocated -= resources;`), I think the subtraction will not work as we expect since the offered resources are revocable but allocated resources are non-revocable.
- Qian Zhang
On Jan. 21, 2016, 1:01 a.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2016, 1:01 a.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115539
-----------------------------------------------------------
There maybe risk if we want to catch up with 0.27.0 for this.
src/master/allocator/mesos/hierarchical.cpp (lines 1311 - 1314)
<https://reviews.apache.org/r/42559/#comment176519>
I would suggest adding a new revocable type here, according to the design of MESOS-1607, this API will set resources as `USAGE SLACK` which means `resources allocated but under-utilized`, but here giving `USAGE SLACK` another meaning: resources beyond quota which does not fit into the `USAGE SLACK` well.
src/master/allocator/mesos/hierarchical.cpp (line 1313)
<https://reviews.apache.org/r/42559/#comment176518>
I would suggest adding a new revocable type here, according to the design of MESOS-1607, this API will set resources as `USAGE SLACK` which means `resources allocated but under-utilized`, but here giving `USAGE SLACK` another meaning: resources beyond quota which does not fit into the `USAGE SLACK` well.
src/master/allocator/mesos/hierarchical.cpp (line 1329)
<https://reviews.apache.org/r/42559/#comment176516>
The filter will not work after you set the resources as revocable.
src/tests/hierarchical_allocator_tests.cpp (line 1621)
<https://reviews.apache.org/r/42559/#comment176517>
You may encounter some problem if you add some static reserved resources.
- Guangya Liu
On 一月 20, 2016, 5:01 p.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated 一月 20, 2016, 5:01 p.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Mesos ReviewBot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115504
-----------------------------------------------------------
Patch looks great!
Reviews applied: [42559]
Passed command: export OS=ubuntu:14.04;export CONFIGURATION="--verbose";export COMPILER=gcc; ./support/docker_build.sh
- Mesos ReviewBot
On Jan. 20, 2016, 5:01 p.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated Jan. 20, 2016, 5:01 p.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>
Re: Review Request 42559: Allocated resources as revocable beyond
quota.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42559/#review115541
-----------------------------------------------------------
Another problem is with Qos Controller, the slave may found that the resource estimator report 10 revocable cpus but the slave is using 20 revocable cpus, if the qos controller has logic of killing executors in such condition, then even if you offer the resources to those tasks, it will still be killed by qos controller.
src/master/allocator/mesos/hierarchical.cpp (line 408)
<https://reviews.apache.org/r/42559/#comment176520>
When slave recover, the `revocable used` resources will also be reported back as `allocated`, this also cause some problem, as when `allocate()`, you are adding the resources which not coverted to `revocable`.
- Guangya Liu
On 一月 20, 2016, 5:01 p.m., Alexander Rukletsov wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42559/
> -----------------------------------------------------------
>
> (Updated 一月 20, 2016, 5:01 p.m.)
>
>
> Review request for mesos, Bernd Mathiske, Joris Van Remoortere, Joseph Wu, and Qian Zhang.
>
>
> Bugs: MESOS-4441
> https://issues.apache.org/jira/browse/MESOS-4441
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> docs/quota.md 1a6d2f07fb74d168a7eb30764ab9ff80cea5e3b6
> src/master/allocator/mesos/hierarchical.cpp e32ee4aa3ed9793bb5a99233e699e5cc2bdd796b
> src/tests/hierarchical_allocator_tests.cpp 953712149bd951789beb29c72779c4ac65aa48dc
>
> Diff: https://reviews.apache.org/r/42559/diff/
>
>
> Testing
> -------
>
> On Mac OS 10.10.4:
>
> `GTEST_FILTER="HierarchicalAllocatorTest.*" ./bin/mesos-tests.sh --gtest_repeat=100 --gtest_break_on_failure --gtest_shuffle`
> `make check`
>
> Updated `HierarchicalAllocatorTest.DRFWithQuota` succeeds with the change and fails without.
>
>
> Thanks,
>
> Alexander Rukletsov
>
>