You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by dl...@apache.org on 2015/10/09 17:35:58 UTC
mesos git commit: This closes: #73 Review:
https://github.com/apache/mesos/pull/73
Repository: mesos
Updated Branches:
refs/heads/master 6ac4b6cdd -> 95c9f9571
This closes: #73
Review: https://github.com/apache/mesos/pull/73
Project: http://git-wip-us.apache.org/repos/asf/mesos/repo
Commit: http://git-wip-us.apache.org/repos/asf/mesos/commit/95c9f957
Tree: http://git-wip-us.apache.org/repos/asf/mesos/tree/95c9f957
Diff: http://git-wip-us.apache.org/repos/asf/mesos/diff/95c9f957
Branch: refs/heads/master
Commit: 95c9f95715a8bc97c0479525681c5bf7fa8dc07d
Parents: 6ac4b6c
Author: Jonathon Rossi <jo...@jonorossi.com>
Authored: Fri Oct 9 16:34:59 2015 +0100
Committer: Dave Lester <da...@davelester.org>
Committed: Fri Oct 9 16:34:59 2015 +0100
----------------------------------------------------------------------
docs/architecture.md | 4 +-
docs/fetcher-cache-internals.md | 12 +++---
docs/images/containerizer_isolator_api.png | Bin 0 -> 89106 bytes
site/.gitignore | 3 ++
site/Rakefile | 39 ++++++++-----------
.../img/documentation/architecture-example.jpg | Bin 98210 -> 0 bytes
.../assets/img/documentation/architecture3.jpg | Bin 112655 -> 0 bytes
.../containerizer_isolator_api.png | Bin 89106 -> 0 bytes
8 files changed, 27 insertions(+), 31 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/docs/architecture.md
----------------------------------------------------------------------
diff --git a/docs/architecture.md b/docs/architecture.md
index 92538b7..df750c7 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -4,7 +4,7 @@ layout: documentation
# Mesos Architecture
-
+
The above figure shows the main components of Mesos. Mesos consists of a *master* daemon that manages *slave* daemons running on each cluster node, and *mesos applications* (also called *frameworks*) that run *tasks* on these slaves.
@@ -16,7 +16,7 @@ A framework running on top of Mesos consists of two components: a *scheduler* th
The figure below shows an example of how a framework gets scheduled to run a task.
-
+
Let's walk through the events in the figure.
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/docs/fetcher-cache-internals.md
----------------------------------------------------------------------
diff --git a/docs/fetcher-cache-internals.md b/docs/fetcher-cache-internals.md
index e8a68d1..b71d225 100644
--- a/docs/fetcher-cache-internals.md
+++ b/docs/fetcher-cache-internals.md
@@ -45,7 +45,7 @@ Besides minor complications such as archive extraction and execution rights sett
Based on this setup, the main program flow in the fetcher process is concerned with assembling a list of parameters to the mesos-fetcher program that describe items to be fetched. This figure illustrates the high-level collaboration of the fetcher process with mesos-fetcher program runs. It also depicts the next level of detail of the fetcher process, which will be described in the following section.
-
+
### Cache state representation and manipulation
@@ -58,7 +58,7 @@ A cache entry corresponds directly to a cache file on disk throughout the entire
This figure illustrates the different states which a cache entry can be in.
-
+
While a cache entry is referenced it cannot be evicted by a the current or any other concurrent fetch attempt in order to make space for a download of a new cache file.
@@ -78,7 +78,7 @@ Every increment is recorded in a list. At the very end of the fetch procedure, n
As menitoned above, the fetcher process' main control flow concerns sorting out what to do with each URI presented to it in a fetch request. An overview of the ensuing control flow for a given URI is depicted in this figure.
-
+
After going through this procedure for each URI, the fetcher process assembles the gathered list of per-URI actions into a JSON object (`FetcherInfo`), which is passed to the mesos-fetcher program in an environment variable. The possible fetch actions for a URI are shown at the bottom of the flow chart. After they are determined, the fetcher process invokes mesos-fetcher.
@@ -101,7 +101,7 @@ Besides, everything touched in 1/a and 1/b needs to be prevented from being cach
### Cache eviction
-
+
The resources named "A" and "B" have been fetched with caching into sandbox 1 and 2 below. In the course of this, two cache entries have been created and two files have been downloaded into the cache and named "1" and "2". (Cache file names have unique names that comprise serial numbers.)
@@ -110,8 +110,8 @@ The next figure illustrates the state after fetching a different cached URI into
1. Remove the cache entry for "A" from the fetcher process' cache entry table. Its faded depiction is supposed to indicate this. This immediately makes it appear as if the URI has never been cached, even though the cache file is still around.
2. Proceed with fetching "C". This creates a new cache file, which has a different unique name. (The fetcher process remembers in its cache entry which file name belongs to which URI.)
-
+
The next figure then shows what happens if the first URI is fetched once again. Here we also assume the cache being so filled up that eviction is necessary and this time the entry and file for "B" are the victims.
-
+
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/docs/images/containerizer_isolator_api.png
----------------------------------------------------------------------
diff --git a/docs/images/containerizer_isolator_api.png b/docs/images/containerizer_isolator_api.png
new file mode 100644
index 0000000..0e15b91
Binary files /dev/null and b/docs/images/containerizer_isolator_api.png differ
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/site/.gitignore
----------------------------------------------------------------------
diff --git a/site/.gitignore b/site/.gitignore
new file mode 100644
index 0000000..987a5d3
--- /dev/null
+++ b/site/.gitignore
@@ -0,0 +1,3 @@
+source/documentation
+source/assets/img/documentation
+publish
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/site/Rakefile
----------------------------------------------------------------------
diff --git a/site/Rakefile b/site/Rakefile
index 6281f06..9fe4730 100644
--- a/site/Rakefile
+++ b/site/Rakefile
@@ -4,10 +4,7 @@ require 'fileutils'
task :default => [:update_docs, :build, :clean]
-tmp_dir = File.join(File.dirname(__FILE__), "tmp")
-mesos_dir = File.join(tmp_dir, "mesos")
-
-CLEAN.include(tmp_dir, "**/.DS_Store")
+CLEAN.include("**/.DS_Store")
desc "Build the website from source"
task :build do
@@ -20,50 +17,46 @@ task :build do
end
end
-desc "Clone the Mesos code base"
-task :clone_mesos do
- puts "Cloning Apache Mesos codebase"
-
- if File.exists?(mesos_dir)
- system("pushd #{mesos_dir} && git pull origin master && popd")
- else
- FileUtils.mkdir_p(mesos_dir)
- system("git clone --depth 1 http://git-wip-us.apache.org/repos/asf/mesos.git #{mesos_dir}")
- end
-end
-
desc "Update the latest docs from the Apache Mesos codebase"
-task :update_docs => :clone_mesos do
+task :update_docs do
puts "Updating latest documentation from the Apache Mesos codebase"
docs_dir = File.join(File.dirname(__FILE__), "source/documentation")
puts "Updating docs to the latest version"
+ FileUtils.mkdir_p("source/documentation/latest/")
FileUtils.rm_f(Dir.glob("source/documentation/latest/*.md"))
- FileUtils.cp_r(Dir.glob("#{mesos_dir}/docs/*.md"), File.expand_path("source/documentation/latest/"))
+ FileUtils.cp_r(Dir.glob("../docs/*.md"), File.expand_path("source/documentation/latest/"))
puts "Parse documentation files to removing md extension in any links"
Dir.chdir("#{docs_dir}/latest/") {
Dir.glob('*.md').each { |doc|
puts "working on: #{doc}"
IO.write(doc, File.open(doc) { |f|
- f.read.gsub(/\((.*)(\.md)\)/, '(/documentation/latest/\1/)')
+ f.read
+ .gsub(/\((.*)(\.md)\)/, '(/documentation/latest/\1/)')
+ .gsub(/\(images\/(.*)\)/, '(/assets/img/documentation/\1)')
})
}
}
puts "Moving documentation index to its own 'latest' directory"
FileUtils.mv("source/documentation/latest/home.md", "source/documentation/latest.html.md")
puts "Documentation updated"
+
+ # Copy the images from the docs
+ FileUtils.mkdir_p("source/assets/img/documentation/")
+ FileUtils.rm_f(Dir.glob("source/assets/img/documentation/*"))
+ FileUtils.cp_r(Dir.glob("../docs/images/*"), File.expand_path("source/assets/img/documentation/"))
end
desc "Generate javadoc from the Java source files in the codebase"
-task :javadoc => :clone_mesos do
+task :javadoc do
# TODO: add version to the path. Uses 'current' for now.
- system("javadoc -d source/api/latest/java -sourcepath #{mesos_dir}/src/java/src org.apache.mesos")
+ system("javadoc -d publish/api/latest/java -sourcepath ../src/java/src org.apache.mesos")
end
desc "Generate doxygen from the C++ source files in the codebase"
-task :doxygen => :clone_mesos do
- system("pushd #{mesos_dir} && doxygen && popd && rsync -avz #{mesos_dir}/docs/html/* source/api/latest/c++")
+task :doxygen do
+ system("pushd .. && doxygen && popd && rsync -avz ../docs/html/* publish/api/latest/c++")
end
desc "Run the site in development mode. Preview available at http://localhost:4567/"
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/site/source/assets/img/documentation/architecture-example.jpg
----------------------------------------------------------------------
diff --git a/site/source/assets/img/documentation/architecture-example.jpg b/site/source/assets/img/documentation/architecture-example.jpg
deleted file mode 100644
index e93e027..0000000
Binary files a/site/source/assets/img/documentation/architecture-example.jpg and /dev/null differ
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/site/source/assets/img/documentation/architecture3.jpg
----------------------------------------------------------------------
diff --git a/site/source/assets/img/documentation/architecture3.jpg b/site/source/assets/img/documentation/architecture3.jpg
deleted file mode 100644
index e962af6..0000000
Binary files a/site/source/assets/img/documentation/architecture3.jpg and /dev/null differ
http://git-wip-us.apache.org/repos/asf/mesos/blob/95c9f957/site/source/assets/img/documentation/containerizer_isolator_api.png
----------------------------------------------------------------------
diff --git a/site/source/assets/img/documentation/containerizer_isolator_api.png b/site/source/assets/img/documentation/containerizer_isolator_api.png
deleted file mode 100644
index 0e15b91..0000000
Binary files a/site/source/assets/img/documentation/containerizer_isolator_api.png and /dev/null differ