You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by an...@apache.org on 2018/03/05 22:17:26 UTC
[1/4] mesos git commit: Windows: Fixed location of Docker's
`config.json` file.
Repository: mesos
Updated Branches:
refs/heads/1.5.x 35716408c -> 0de1cce2e
Windows: Fixed location of Docker's `config.json` file.
Per MESOS-8619, Docker checks `$USERPROFILE/.docker/config.json`
instead of `$HOME`. Mesos overrides this environment variable in order
to point Docker to a `config.json` file in another location, so we
have to fix the assumption we made about Docker.
We do not add this constant to stout, because it is not consistent
across Windows applications. This particular logic is specific to the
implementation of Docker. Other applications might check `$HOME` or
`$HOMEPATH` on Windows.
Review: https://reviews.apache.org/r/65872
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/048b7df0
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/048b7df0
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/048b7df0
Branch: refs/heads/1.5.x
Commit: 048b7df0e08a61e84619f29fede300d9455ffdb6
Parents: 3571640
Author: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Authored: Wed Feb 28 16:45:49 2018 -0800
Committer: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Committed: Mon Mar 5 13:26:17 2018 -0800
----------------------------------------------------------------------
src/docker/docker.cpp | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/048b7df0/src/docker/docker.cpp
----------------------------------------------------------------------
diff --git a/src/docker/docker.cpp b/src/docker/docker.cpp
index 722a54a..c2feebe 100644
--- a/src/docker/docker.cpp
+++ b/src/docker/docker.cpp
@@ -1669,10 +1669,19 @@ Future<Docker::Image> Docker::__pull(
// provided which is a docker config file we want docker to be
// able to pick it up from the sandbox directory where we store
// all the URI downloads.
+ //
+ // NOTE: On Windows, Docker users $USERPROFILE instead of $HOME.
+ // See MESOS-8619 for more details.
+ //
// TODO(gilbert): Deprecate the fetching docker config file
// specified as URI method on 0.30.0 release.
+#ifdef __WINDOWS__
+ const std::string HOME = "USERPROFILE";
+#else
+ const std::string HOME = "HOME";
+#endif // __WINDOWS__
map<string, string> environment = os::environment();
- environment["HOME"] = directory;
+ environment[HOME] = directory;
bool configExisted =
os::exists(path::join(directory, ".docker", "config.json")) ||
@@ -1683,7 +1692,7 @@ Future<Docker::Image> Docker::__pull(
// and another docker config file is specified using the
// '--docker_config' agent flag.
if (!configExisted && home.isSome()) {
- environment["HOME"] = home.get();
+ environment[HOME] = home.get();
}
Try<Subprocess> s_ = subprocess(
[2/4] mesos git commit: Added MESOS-8619 to the 1.5.1 CHANGELOG.
Posted by an...@apache.org.
Added MESOS-8619 to the 1.5.1 CHANGELOG.
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/77e4e7ee
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/77e4e7ee
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/77e4e7ee
Branch: refs/heads/1.5.x
Commit: 77e4e7ee4fb5d38b0df965fede82a91bff10ab51
Parents: 048b7df
Author: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Authored: Mon Mar 5 13:28:27 2018 -0800
Committer: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Committed: Mon Mar 5 13:28:27 2018 -0800
----------------------------------------------------------------------
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/77e4e7ee/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index 47bfa77..91479a0 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -16,6 +16,7 @@ Release Notes - Mesos - Version 1.5.1 (WIP)
* [MESOS-8598] - Allow empty resource provider selector in `UriDiskProfileAdaptor`.
* [MESOS-8601] - Master crashes during slave reregistration after failover.
* [MESOS-8604] - Quota headroom tracking may be incorrect in the presence of hierarchical reservation.
+ * [MESOS-8619] - Docker on Windows uses `USERPROFILE` instead of `HOME` for credentials.
Release Notes - Mesos - Version 1.5.0
[3/4] mesos git commit: Windows: Fixed bug with CPU isolator not
checking the max boundary.
Posted by an...@apache.org.
Windows: Fixed bug with CPU isolator not checking the max boundary.
The `set_job_cpu_limit` function does not sanitize its input, e.g.
`cpus`, causing a bug when more CPUs were requested than exist on the
machine.
For instance, in Mesos it was not possible to start a single task
which consumed every CPU, because the executor overcommit would cause
the task to ask for 0.1 more CPUs than available. So on a quad-core
machine, it'd ask for 4 CPUs, but that would cause the limiter to set
a limit at 4.1 CPUs, which exceeds the maximum allowed, causing an
"Invalid parameter" error when setting the CPU limit. We can avoid
this by ensuring the `cpu_rate` value is in the range of [1, 10000],
regardless of the input.
Review: https://reviews.apache.org/r/65891
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/e0b5046f
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/e0b5046f
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/e0b5046f
Branch: refs/heads/1.5.x
Commit: e0b5046fdb99dbef920615492a6f568523778a88
Parents: 77e4e7e
Author: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Authored: Fri Mar 2 13:22:09 2018 -0800
Committer: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Committed: Mon Mar 5 13:28:59 2018 -0800
----------------------------------------------------------------------
3rdparty/stout/include/stout/windows/os.hpp | 21 +++++++++++++--------
1 file changed, 13 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/e0b5046f/3rdparty/stout/include/stout/windows/os.hpp
----------------------------------------------------------------------
diff --git a/3rdparty/stout/include/stout/windows/os.hpp b/3rdparty/stout/include/stout/windows/os.hpp
index 8c6f7a3..94cbbab 100644
--- a/3rdparty/stout/include/stout/windows/os.hpp
+++ b/3rdparty/stout/include/stout/windows/os.hpp
@@ -15,6 +15,7 @@
#include <sys/utime.h>
+#include <algorithm>
#include <list>
#include <map>
#include <memory>
@@ -795,14 +796,18 @@ inline Try<Nothing> set_job_cpu_limit(pid_t pid, double cpus)
// `(cpus / os::cpus()) * 100 * 100`, or the requested `cpus` divided by the
// number of CPUs to obtain a fractional representation, multiplied by 100 to
// make it a percentage, multiplied again by 100 to become a `CpuRate`.
- Try<long> total_cpus = os::cpus();
- control_info.CpuRate =
- static_cast<DWORD>((cpus / total_cpus.get()) * 100 * 100);
- // This must not be set to 0, so 1 is the minimum.
- if (control_info.CpuRate < 1) {
- control_info.CpuRate = 1;
- }
-
+ //
+ // Mathematically, we're normalizing the requested CPUS to a range
+ // of [1, 10000] cycles. However, because the input is not
+ // sanitized, we have to handle the edge case of the ratio being
+ // greater than 1. So we take the `min(max(ratio * 10000, 1),
+ // 10000)`. We don't consider going out of bounds an error because
+ // CPU limitations are inherently imprecise.
+ const long total_cpus = os::cpus().get(); // This doesn't fail on Windows.
+ // This must be constrained. We don't care about perfect precision.
+ const long cycles = static_cast<long>((cpus / total_cpus) * 10000L);
+ const long cpu_rate = std::min(std::max(cycles, 1L), 10000L);
+ control_info.CpuRate = static_cast<DWORD>(cpu_rate);
Try<SharedHandle> job_handle = os::open_job(
JOB_OBJECT_SET_ATTRIBUTES,
false,
[4/4] mesos git commit: Added MESOS-8631 to the 1.5.1 CHANGELOG.
Posted by an...@apache.org.
Added MESOS-8631 to the 1.5.1 CHANGELOG.
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/0de1cce2
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/0de1cce2
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/0de1cce2
Branch: refs/heads/1.5.x
Commit: 0de1cce2e521f13590e3f406c931bf5f0b2d2d15
Parents: e0b5046
Author: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Authored: Mon Mar 5 13:30:27 2018 -0800
Committer: Andrew Schwartzmeyer <an...@schwartzmeyer.com>
Committed: Mon Mar 5 13:30:27 2018 -0800
----------------------------------------------------------------------
CHANGELOG | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/0de1cce2/CHANGELOG
----------------------------------------------------------------------
diff --git a/CHANGELOG b/CHANGELOG
index 91479a0..f544ced 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -17,6 +17,7 @@ Release Notes - Mesos - Version 1.5.1 (WIP)
* [MESOS-8601] - Master crashes during slave reregistration after failover.
* [MESOS-8604] - Quota headroom tracking may be incorrect in the presence of hierarchical reservation.
* [MESOS-8619] - Docker on Windows uses `USERPROFILE` instead of `HOME` for credentials.
+ * [MESOS-8631] - Agent should be able to start a task with every CPU on a Windows machine.
Release Notes - Mesos - Version 1.5.0