You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by ya...@apache.org on 2017/05/23 07:44:27 UTC
[1/4] mesos git commit: Update XFS disk isolator documentation.
Repository: mesos
Updated Branches:
refs/heads/master 5d4faaa9b -> 3135a9d04
Update XFS disk isolator documentation.
Review: https://reviews.apache.org/r/55903/
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/3135a9d0
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/3135a9d0
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/3135a9d0
Branch: refs/heads/master
Commit: 3135a9d0452a87f6ab96a090203d4cf5f5047cf5
Parents: e86f547
Author: James Peach <jp...@apache.org>
Authored: Tue May 23 00:35:17 2017 -0700
Committer: Jiang Yan Xu <xu...@apple.com>
Committed: Tue May 23 00:44:23 2017 -0700
----------------------------------------------------------------------
docs/mesos-containerizer.md | 30 +++++++++++++-----------------
1 file changed, 13 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/3135a9d0/docs/mesos-containerizer.md
----------------------------------------------------------------------
diff --git a/docs/mesos-containerizer.md b/docs/mesos-containerizer.md
index d0b9d76..feaaa55 100644
--- a/docs/mesos-containerizer.md
+++ b/docs/mesos-containerizer.md
@@ -89,19 +89,20 @@ minute. The default interval is 15 seconds.
### XFS Disk Isolator
-The XFS Disk isolator uses XFS project quotas to track the disk
-space used by each container sandbox and to enforce the corresponding
-disk space allocation. Write operations performed by tasks exceeding
-their disk allocation will fail with an `EDQUOT` error. The task
-will not be terminated by the containerizer.
-
-The XFS disk isolator is functionally similar to Posix Disk isolator
-but avoids the cost of repeatedly running the `du`. Though they will
-not interfere with each other, it is not recommended to use them together.
+The XFS Disk isolator uses XFS project quotas to track the disk space
+used by each container sandbox and to enforce the corresponding disk
+space allocation. When quota enforcement is enabled, write operations
+performed by tasks exceeding their disk allocation will fail with an
+`EDQUOT` error. The task will not be terminated by the containerizer.
To enable the XFS Disk isolator, append `disk/xfs` to the `--isolation`
flag when starting the agent.
+The XFS Disk isolator supports the `--enforce_container_disk_quota` flag.
+If enforcement is enabled, the isolator will set both the hard and soft
+quota limit. Otherwise, no limits will be set, Disk usage accounting
+will be performed but the task will be allowed to exceed its allocation.
+
The XFS Disk isolator requires the sandbox directory to be located
on an XFS filesystem that is mounted with the `pquota` option. There
is no need to configure
@@ -110,15 +111,11 @@ or [projid](http://man7.org/linux/man-pages/man5/projid.5.html)
files. The range of project IDs given to the `--xfs_project_range`
must not overlap any project IDs allocated for other uses.
-The XFS disk isolator does not natively support an accounting-only mode
-like that of the Posix Disk isolator. Quota enforcement can be disabled
-by mounting the filesystem with the `pqnoenforce` mount option.
-
The [xfs_quota](http://man7.org/linux/man-pages/man8/xfs_quota.8.html)
command can be used to show the current allocation of project IDs
and quota. For example:
- $ xfs_quota -x -c "report -a -n -L 5000 -U 1000"
+ $ xfs_quota -x -c "report -a -n -L 5000 -U 10000"
To show which project a file belongs to, use the
[xfs_io](http://man7.org/linux/man-pages/man8/xfs_io.8.html) command
@@ -126,9 +123,8 @@ to display the `fsxattr.projid` field. For example:
$ xfs_io -r -c stat /mnt/mesos/
-Note that the Posix Disk isolator flags `--enforce_container_disk_quota`,
-`--container_disk_watch_interval` and `--enforce_container_disk_quota` do
-not apply to the XFS Disk isolator.
+Note that the Posix Disk isolator `--container_disk_watch_interval`
+does not apply to the XFS Disk isolator.
### Docker Runtime Isolator
[4/4] mesos git commit: Stop storing agent flags in the XFS disk
isolator.
Posted by ya...@apache.org.
Stop storing agent flags in the XFS disk isolator.
Review: https://reviews.apache.org/r/55896/
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/bb048eca
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/bb048eca
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/bb048eca
Branch: refs/heads/master
Commit: bb048ecabf30abd56a091ce54fa8abad1651ad62
Parents: 5d4faaa
Author: James Peach <jp...@apache.org>
Authored: Tue May 23 00:33:11 2017 -0700
Committer: Jiang Yan Xu <xu...@apple.com>
Committed: Tue May 23 00:44:23 2017 -0700
----------------------------------------------------------------------
src/slave/containerizer/mesos/isolators/xfs/disk.cpp | 8 ++++----
src/slave/containerizer/mesos/isolators/xfs/disk.hpp | 4 ++--
2 files changed, 6 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/bb048eca/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/xfs/disk.cpp b/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
index 40f1049..72bef87 100644
--- a/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
+++ b/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
@@ -157,15 +157,15 @@ Try<Isolator*> XfsDiskIsolatorProcess::create(const Flags& flags)
}
return new MesosIsolator(Owned<MesosIsolatorProcess>(
- new XfsDiskIsolatorProcess(flags, totalProjectIds.get())));
+ new XfsDiskIsolatorProcess(flags.work_dir, totalProjectIds.get())));
}
XfsDiskIsolatorProcess::XfsDiskIsolatorProcess(
- const Flags& _flags,
+ const std::string& _workDir,
const IntervalSet<prid_t>& projectIds)
: ProcessBase(process::ID::generate("xfs-disk-isolator")),
- flags(_flags),
+ workDir(_workDir),
totalProjectIds(projectIds),
freeProjectIds(projectIds)
{
@@ -188,7 +188,7 @@ Future<Nothing> XfsDiskIsolatorProcess::recover(
// for project IDs that we have not recovered and make a best effort to
// remove all the corresponding on-disk state.
Try<list<string>> sandboxes = os::glob(path::join(
- paths::getSandboxRootDir(flags.work_dir),
+ paths::getSandboxRootDir(workDir),
"*",
"frameworks",
"*",
http://git-wip-us.apache.org/repos/asf/mesos/blob/bb048eca/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/xfs/disk.hpp b/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
index 52f0459..8f008f3 100644
--- a/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
+++ b/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
@@ -72,7 +72,7 @@ public:
private:
XfsDiskIsolatorProcess(
- const Flags& flags,
+ const std::string& workDir,
const IntervalSet<prid_t>& projectIds);
// Take the next project ID from the unallocated pool.
@@ -91,7 +91,7 @@ private:
const prid_t projectId;
};
- const Flags flags;
+ const std::string workDir;
const IntervalSet<prid_t> totalProjectIds;
IntervalSet<prid_t> freeProjectIds;
hashmap<ContainerID, process::Owned<Info>> infos;
[2/4] mesos git commit: Extract a BasicBlocks class for disk block
arithmetic.
Posted by ya...@apache.org.
Extract a BasicBlocks class for disk block arithmetic.
Extract a BasicBlocks class to handle disk blocks to clarify block-based
arithmetic in the XFS disk isolator.
Review: https://reviews.apache.org/r/55895/
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/80b652c3
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/80b652c3
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/80b652c3
Branch: refs/heads/master
Commit: 80b652c3342120cb476fda6d16eb5ba231cab10a
Parents: bb048ec
Author: James Peach <jp...@apache.org>
Authored: Tue May 23 00:34:30 2017 -0700
Committer: Jiang Yan Xu <xu...@apple.com>
Committed: Tue May 23 00:44:23 2017 -0700
----------------------------------------------------------------------
.../containerizer/mesos/isolators/xfs/utils.cpp | 26 +++++++---------
.../containerizer/mesos/isolators/xfs/utils.hpp | 32 ++++++++++++++++++++
src/tests/containerizer/xfs_quota_tests.cpp | 16 ++++++++++
3 files changed, 59 insertions(+), 15 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/80b652c3/src/slave/containerizer/mesos/isolators/xfs/utils.cpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/xfs/utils.cpp b/src/slave/containerizer/mesos/isolators/xfs/utils.cpp
index 8018ad3..2708524 100644
--- a/src/slave/containerizer/mesos/isolators/xfs/utils.cpp
+++ b/src/slave/containerizer/mesos/isolators/xfs/utils.cpp
@@ -59,11 +59,6 @@ namespace mesos {
namespace internal {
namespace xfs {
-// The quota API defines space limits in terms of in basic
-// blocks (512 bytes).
-static constexpr Bytes BASIC_BLOCK_SIZE = Bytes(512u);
-
-
// Although XFS itself doesn't define any invalid project IDs,
// we need a way to know whether or not a project ID was assigned
// so we use 0 as our sentinel value.
@@ -154,12 +149,13 @@ static Try<Nothing> setProjectQuota(
quota.d_id = projectId;
quota.d_flags = FS_PROJ_QUOTA;
- // Set both the hard and the soft limit to the same quota, just
- // for consistency. Functionally all we need is the hard quota.
+ // Set both the hard and the soft limit to the same quota. Functionally
+ // all we need is the hard limit. The soft limit has no effect when it
+ // is the same as the hard limit but we set it for explicitness.
quota.d_fieldmask = FS_DQ_BSOFT | FS_DQ_BHARD;
- quota.d_blk_hardlimit = limit.bytes() / BASIC_BLOCK_SIZE.bytes();
- quota.d_blk_softlimit = limit.bytes() / BASIC_BLOCK_SIZE.bytes();
+ quota.d_blk_hardlimit = BasicBlocks(limit).blocks();
+ quota.d_blk_softlimit = BasicBlocks(limit).blocks();
if (::quotactl(QCMD(Q_XSETQLIM, PRJQUOTA),
devname.get().c_str(),
@@ -250,8 +246,8 @@ Result<QuotaInfo> getProjectQuota(
}
QuotaInfo info;
- info.limit = BASIC_BLOCK_SIZE * quota.d_blk_hardlimit;
- info.used = BASIC_BLOCK_SIZE * quota.d_bcount;
+ info.limit = BasicBlocks(quota.d_blk_hardlimit).bytes();
+ info.used = BasicBlocks(quota.d_bcount).bytes();
return info;
}
@@ -266,10 +262,10 @@ Try<Nothing> setProjectQuota(
return nonProjectError();
}
- // A 0 limit deletes the quota record. Since the limit is in basic
- // blocks that effectively means > 512 bytes.
- if (limit < BASIC_BLOCK_SIZE) {
- return Error("Quota limit must be >= " + stringify(BASIC_BLOCK_SIZE));
+ // A 0 limit deletes the quota record. If that's desired, the
+ // caller should use clearProjectQuota().
+ if (limit == 0) {
+ return Error( "Quota limit must be greater than 0");
}
return internal::setProjectQuota(path, projectId, limit);
http://git-wip-us.apache.org/repos/asf/mesos/blob/80b652c3/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/xfs/utils.hpp b/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
index eddd4c3..e5567c5 100644
--- a/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
+++ b/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
@@ -37,6 +37,38 @@ struct QuotaInfo
};
+// Quota operations are defined in terms of basic blocks (512 byte units).
+class BasicBlocks
+{
+public:
+ // Convert from Bytes to basic blocks. Note that we round up since a partial
+ // block costs a full block to store on disk.
+ explicit BasicBlocks(const Bytes& bytes)
+ : blockCount((bytes.bytes() + BASIC_BLOCK_SIZE - 1) / BASIC_BLOCK_SIZE) {}
+
+ explicit constexpr BasicBlocks(uint64_t _blockCount)
+ : blockCount(_blockCount) {}
+
+ bool operator==(const BasicBlocks& that) const
+ {
+ return blockCount == that.blockCount;
+ }
+
+ bool operator!=(const BasicBlocks& that) const
+ {
+ return blockCount != that.blockCount;
+ }
+
+ uint64_t blocks() const { return blockCount; }
+ Bytes bytes() const { return Bytes(BASIC_BLOCK_SIZE) * blockCount; }
+
+private:
+ uint64_t blockCount;
+
+ static constexpr unsigned BASIC_BLOCK_SIZE = 512;
+};
+
+
inline bool operator==(const QuotaInfo& left, const QuotaInfo& right)
{
return left.limit == right.limit && left.used == right.used;
http://git-wip-us.apache.org/repos/asf/mesos/blob/80b652c3/src/tests/containerizer/xfs_quota_tests.cpp
----------------------------------------------------------------------
diff --git a/src/tests/containerizer/xfs_quota_tests.cpp b/src/tests/containerizer/xfs_quota_tests.cpp
index 7beb60b..78a8fce 100644
--- a/src/tests/containerizer/xfs_quota_tests.cpp
+++ b/src/tests/containerizer/xfs_quota_tests.cpp
@@ -937,6 +937,22 @@ TEST_F(ROOT_XFS_QuotaTest, CheckQuotaEnabled)
EXPECT_SOME_EQ(true, xfs::isQuotaEnabled(mountPoint.get()));
}
+
+TEST(XFS_QuotaTest, BasicBlocks)
+{
+ // 0 is the same for blocks and bytes.
+ EXPECT_EQ(BasicBlocks(0).bytes(), Bytes(0u));
+
+ EXPECT_EQ(BasicBlocks(1).bytes(), Bytes(512));
+
+ // A partial block should round up.
+ EXPECT_EQ(Bytes(512), BasicBlocks(Bytes(128)).bytes());
+ EXPECT_EQ(Bytes(1024), BasicBlocks(Bytes(513)).bytes());
+
+ EXPECT_EQ(BasicBlocks(1), BasicBlocks(1));
+ EXPECT_EQ(BasicBlocks(1), BasicBlocks(Bytes(512)));
+}
+
} // namespace tests {
} // namespace internal {
} // namespace mesos {
[3/4] mesos git commit: Add support for not enforcing XFS quotas.
Posted by ya...@apache.org.
Add support for not enforcing XFS quotas.
Add XFS disk isolator support for not enforcing disk quotas on
containers. While there is a global filesystem configuration option
to turn off quota enforcement, we should not automatically toggle
that because we don't know why the operator might have changed that
configuration. Instead, we just apply an unlimited (0) quota, which
engages XFS space accounting without enforcing any limit.
Review: https://reviews.apache.org/r/55897/
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/e86f547e
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/e86f547e
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/e86f547e
Branch: refs/heads/master
Commit: e86f547e334092de7884db6f32a9cdd396862769
Parents: 80b652c
Author: James Peach <jp...@apache.org>
Authored: Tue May 23 00:34:48 2017 -0700
Committer: Jiang Yan Xu <xu...@apple.com>
Committed: Tue May 23 00:44:23 2017 -0700
----------------------------------------------------------------------
.../containerizer/mesos/isolators/xfs/disk.cpp | 54 ++++--
.../containerizer/mesos/isolators/xfs/disk.hpp | 2 +
.../containerizer/mesos/isolators/xfs/utils.hpp | 6 +
src/tests/containerizer/xfs_quota_tests.cpp | 167 ++++++++++++++++++-
4 files changed, 212 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/e86f547e/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/xfs/disk.cpp b/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
index 72bef87..2113f86 100644
--- a/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
+++ b/src/slave/containerizer/mesos/isolators/xfs/disk.cpp
@@ -156,15 +156,22 @@ Try<Isolator*> XfsDiskIsolatorProcess::create(const Flags& flags)
return Error(status->message);
}
+ xfs::QuotaPolicy quotaPolicy =
+ flags.enforce_container_disk_quota ? xfs::QuotaPolicy::ENFORCING
+ : xfs::QuotaPolicy::ACCOUNTING;
+
return new MesosIsolator(Owned<MesosIsolatorProcess>(
- new XfsDiskIsolatorProcess(flags.work_dir, totalProjectIds.get())));
+ new XfsDiskIsolatorProcess(
+ quotaPolicy, flags.work_dir, totalProjectIds.get())));
}
XfsDiskIsolatorProcess::XfsDiskIsolatorProcess(
+ xfs::QuotaPolicy _quotaPolicy,
const std::string& _workDir,
const IntervalSet<prid_t>& projectIds)
: ProcessBase(process::ID::generate("xfs-disk-isolator")),
+ quotaPolicy(_quotaPolicy),
workDir(_workDir),
totalProjectIds(projectIds),
freeProjectIds(projectIds)
@@ -322,23 +329,37 @@ Future<Nothing> XfsDiskIsolatorProcess::update(
return Nothing();
}
- // Only update the disk quota if it has changed.
- if (needed.get() != info->quota) {
- Try<Nothing> status =
- xfs::setProjectQuota(info->directory, info->projectId, needed.get());
+ switch (quotaPolicy) {
+ case xfs::QuotaPolicy::ACCOUNTING: {
+ Try<Nothing> status = xfs::clearProjectQuota(
+ info->directory, info->projectId);
+
+ if (status.isError()) {
+ return Failure("Failed to clear quota for project " +
+ stringify(info->projectId) + ": " + status.error());
+ }
- if (status.isError()) {
- return Failure("Failed to update quota for project " +
- stringify(info->projectId) + ": " + status.error());
+ break;
}
- info->quota = needed.get();
+ case xfs::QuotaPolicy::ENFORCING: {
+ Try<Nothing> status = xfs::setProjectQuota(
+ info->directory, info->projectId, needed.get());
+
+ if (status.isError()) {
+ return Failure("Failed to update quota for project " +
+ stringify(info->projectId) + ": " + status.error());
+ }
- LOG(INFO) << "Set quota on container " << containerId
- << " for project " << info->projectId
- << " to " << info->quota;
+ LOG(INFO) << "Set quota on container " << containerId
+ << " for project " << info->projectId
+ << " to " << needed.get();
+
+ break;
+ }
}
+ info->quota = needed.get();
return Nothing();
}
@@ -361,9 +382,14 @@ Future<ResourceStatistics> XfsDiskIsolatorProcess::usage(
return Failure(quota.error());
}
+ // If we didn't set the quota (ie. we are in ACCOUNTING mode),
+ // the quota limit will be 0. Since we are already tracking
+ // what the quota ought to be in the Info, we just always
+ // use that.
+ statistics.set_disk_limit_bytes(info->quota.bytes());
+
if (quota.isSome()) {
- statistics.set_disk_limit_bytes(quota.get().limit.bytes());
- statistics.set_disk_used_bytes(quota.get().used.bytes());
+ statistics.set_disk_used_bytes(quota->used.bytes());
}
return statistics;
http://git-wip-us.apache.org/repos/asf/mesos/blob/e86f547e/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/xfs/disk.hpp b/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
index 8f008f3..aced6d9 100644
--- a/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
+++ b/src/slave/containerizer/mesos/isolators/xfs/disk.hpp
@@ -72,6 +72,7 @@ public:
private:
XfsDiskIsolatorProcess(
+ xfs::QuotaPolicy quotaPolicy,
const std::string& workDir,
const IntervalSet<prid_t>& projectIds);
@@ -91,6 +92,7 @@ private:
const prid_t projectId;
};
+ xfs::QuotaPolicy quotaPolicy;
const std::string workDir;
const IntervalSet<prid_t> totalProjectIds;
IntervalSet<prid_t> freeProjectIds;
http://git-wip-us.apache.org/repos/asf/mesos/blob/e86f547e/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
----------------------------------------------------------------------
diff --git a/src/slave/containerizer/mesos/isolators/xfs/utils.hpp b/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
index e5567c5..e034133 100644
--- a/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
+++ b/src/slave/containerizer/mesos/isolators/xfs/utils.hpp
@@ -69,6 +69,12 @@ private:
};
+enum class QuotaPolicy {
+ ENFORCING,
+ ACCOUNTING,
+};
+
+
inline bool operator==(const QuotaInfo& left, const QuotaInfo& right)
{
return left.limit == right.limit && left.used == right.used;
http://git-wip-us.apache.org/repos/asf/mesos/blob/e86f547e/src/tests/containerizer/xfs_quota_tests.cpp
----------------------------------------------------------------------
diff --git a/src/tests/containerizer/xfs_quota_tests.cpp b/src/tests/containerizer/xfs_quota_tests.cpp
index 78a8fce..b33fe8b 100644
--- a/src/tests/containerizer/xfs_quota_tests.cpp
+++ b/src/tests/containerizer/xfs_quota_tests.cpp
@@ -32,8 +32,6 @@
#include <stout/os.hpp>
#include <stout/path.hpp>
-#include <stout/os/constants.hpp>
-
#include "linux/fs.hpp"
#include "master/master.hpp"
@@ -170,6 +168,7 @@ public:
// don't mind that other flags refer to a different temp directory.
flags.work_dir = mountPoint.get();
flags.isolation = "disk/xfs";
+ flags.enforce_container_disk_quota = true;
return flags;
}
@@ -441,6 +440,69 @@ TEST_F(ROOT_XFS_QuotaTest, DiskUsageExceedsQuota)
}
+// This is the same logic as DiskUsageExceedsQuota except we turn off disk quota
+// enforcement, so exceeding the quota should be allowed.
+TEST_F(ROOT_XFS_QuotaTest, DiskUsageExceedsQuotaNoEnforce)
+{
+ Try<Owned<cluster::Master>> master = StartMaster();
+ ASSERT_SOME(master);
+
+ Owned<MasterDetector> detector = master.get()->createDetector();
+
+ slave::Flags flags = CreateSlaveFlags();
+ flags.enforce_container_disk_quota = false;
+
+ Try<Owned<cluster::Slave>> slave = StartSlave(detector.get(), flags);
+ ASSERT_SOME(slave);
+
+ MockScheduler sched;
+ MesosSchedulerDriver driver(
+ &sched, DEFAULT_FRAMEWORK_INFO, master.get()->pid, DEFAULT_CREDENTIAL);
+
+ EXPECT_CALL(sched, registered(&driver, _, _));
+
+ Future<vector<Offer>> offers;
+ EXPECT_CALL(sched, resourceOffers(&driver, _))
+ .WillOnce(FutureArg<1>(&offers))
+ .WillRepeatedly(Return()); // Ignore subsequent offers.
+
+ driver.start();
+
+ AWAIT_READY(offers);
+ EXPECT_FALSE(offers->empty());
+
+ const Offer& offer = offers.get()[0];
+
+ // Create a task which requests 1MB disk, but actually uses more
+ // than 2MB disk.
+ TaskInfo task = createTask(
+ offer.slave_id(),
+ Resources::parse("cpus:1;mem:128;disk:1").get(),
+ "dd if=/dev/zero of=file bs=1048576 count=2");
+
+ Future<TaskStatus> status1;
+ Future<TaskStatus> status2;
+ EXPECT_CALL(sched, statusUpdate(&driver, _))
+ .WillOnce(FutureArg<1>(&status1))
+ .WillOnce(FutureArg<1>(&status2));
+
+ driver.launchTasks(offer.id(), {task});
+
+ AWAIT_READY(status1);
+ EXPECT_EQ(task.task_id(), status1->task_id());
+ EXPECT_EQ(TASK_RUNNING, status1->state());
+
+ // We expect the task to succeed even though it exceeded
+ // the disk quota.
+ AWAIT_READY(status2);
+ EXPECT_EQ(task.task_id(), status2->task_id());
+ EXPECT_EQ(TASK_FINISHED, status2->state());
+
+ driver.stop();
+ driver.join();
+}
+
+
// Verify that we can get accurate resource statistics from the XFS
// disk isolator.
TEST_F(ROOT_XFS_QuotaTest, ResourceStatistics)
@@ -528,7 +590,106 @@ TEST_F(ROOT_XFS_QuotaTest, ResourceStatistics)
}
ASSERT_FALSE(timeout.expired());
- os::sleep(Milliseconds(1));
+ os::sleep(Milliseconds(100));
+ }
+
+ driver.stop();
+ driver.join();
+}
+
+
+// This is the same logic as ResourceStatistics, except the task should
+// be allowed to exceed the disk quota, and usage statistics should report
+// that the quota was exceeded.
+TEST_F(ROOT_XFS_QuotaTest, ResourceStatisticsNoEnforce)
+{
+ Try<Owned<cluster::Master>> master = StartMaster();
+ ASSERT_SOME(master);
+
+ Fetcher fetcher;
+ Owned<MasterDetector> detector = master.get()->createDetector();
+
+ slave::Flags flags = CreateSlaveFlags();
+ flags.enforce_container_disk_quota = false;
+
+ Try<MesosContainerizer*> _containerizer =
+ MesosContainerizer::create(flags, true, &fetcher);
+
+ ASSERT_SOME(_containerizer);
+ Owned<MesosContainerizer> containerizer(_containerizer.get());
+
+ Try<Owned<cluster::Slave>> slave =
+ StartSlave(detector.get(), containerizer.get(), flags);
+ ASSERT_SOME(slave);
+
+ MockScheduler sched;
+
+ MesosSchedulerDriver driver(
+ &sched, DEFAULT_FRAMEWORK_INFO, master.get()->pid, DEFAULT_CREDENTIAL);
+ EXPECT_CALL(sched, registered(_, _, _));
+
+ Future<vector<Offer>> offers;
+ EXPECT_CALL(sched, resourceOffers(_, _))
+ .WillOnce(FutureArg<1>(&offers))
+ .WillRepeatedly(Return()); // Ignore subsequent offers.
+
+ driver.start();
+
+ AWAIT_READY(offers);
+ EXPECT_FALSE(offers->empty());
+
+ Offer offer = offers.get()[0];
+
+ // Create a task that uses 4MB of 3MB disk and fails if it can't
+ // write the full amount.
+ TaskInfo task = createTask(
+ offer.slave_id(),
+ Resources::parse("cpus:1;mem:128;disk:3").get(),
+ "dd if=/dev/zero of=file bs=1048576 count=4 && sleep 1000");
+
+ Future<TaskStatus> status;
+ EXPECT_CALL(sched, statusUpdate(&driver, _))
+ .WillOnce(FutureArg<1>(&status))
+ .WillRepeatedly(Return()); // Ignore subsequent updates.
+
+ driver.launchTasks(offers.get()[0].id(), {task});
+
+ AWAIT_READY(status);
+ EXPECT_EQ(task.task_id(), status->task_id());
+ EXPECT_EQ(TASK_RUNNING, status->state());
+
+ Future<hashset<ContainerID>> containers = containerizer.get()->containers();
+ AWAIT_READY(containers);
+ ASSERT_EQ(1u, containers->size());
+
+ ContainerID containerId = *(containers->begin());
+ Duration diskTimeout = Seconds(5);
+ Timeout timeout = Timeout::in(diskTimeout);
+
+ while (true) {
+ Future<ResourceStatistics> usage = containerizer.get()->usage(containerId);
+ AWAIT_READY(usage);
+
+ ASSERT_TRUE(usage->has_disk_limit_bytes());
+ EXPECT_EQ(Megabytes(3), Bytes(usage->disk_limit_bytes()));
+
+ if (usage->has_disk_used_bytes()) {
+ if (usage->disk_used_bytes() >= Megabytes(4).bytes()) {
+ break;
+ }
+ }
+
+ // The stopping condition for this test is that the isolator is
+ // able to report that we wrote the full amount of data without
+ // being constrained by the task disk limit.
+ EXPECT_LE(usage->disk_used_bytes(), Megabytes(4).bytes());
+
+ ASSERT_FALSE(timeout.expired())
+ << "Used " << Bytes(usage->disk_used_bytes())
+ << " of expected " << Megabytes(4)
+ << " within the " << diskTimeout << " timeout";
+
+ os::sleep(Milliseconds(100));
}
driver.stop();